Compiler Internals

Under the hood

The Recoil compiler is written in Rebol and turns .rcl source into C, then hands that C to a target toolchain. This page sketches the pipeline and the intermediate forms for the curious and for contributors.

source

→

AST

→

monomorphize

→

borrow check

→

facts

→

optimize

→

C → link

AST and IR are Rebol block/object structures and are compiler internals, not a stable external API. The shapes below are representative samples and can change between versions.

End-to-end stages

Orchestrated by compile in src/compiler.r3; the CLI wrapper is recoil.r3.

Stage	Role
Prepare	load modules, parse (`to-ast`), parse the header, alpha-rename bindings → statement AST
Monomorphize	instantiate generic functions/types where required
Borrow check	`analyze` — ownership / move / borrow state over the AST
Fact analysis	`analyze-facts` — flow of refined types / constraints (post-borrow)
AST → IR	`ast-to-ir` — typed, C-oriented IR blocks (`ir-*` tags)
Optimize	`optimize` — constant folding, dead-code elimination, expression cleanup, optional self-tail-call rewrite
Codegen prep	collect constants, register series/global/rule info, assemble header + C helpers, select the entry wrapper
Emit C	`emit-c` / `emit-expr` — IR → C source plus include/link metadata; entry shape (`main`, `app_main`, emscripten) chosen here
Build	write a temp `.c` and invoke the target toolchain (GCC-compatible, `emcc`, MSVC, or ESP-IDF) and link the runtime

Each stage has its own page in the repository docs: 01 prepare, 02 monomorphize, 03 borrow check, 04 fact analysis, 05 AST to IR, 06 optimize, 07 codegen and 08 link and CLI (index: pipeline overview). The node model is described in AST model.

The AST

The parser produces a top-level block of statement nodes. Ordinary bindings are assignment-shaped (x: make i32! 1 becomes a set path, not an old decl form); if/either/match are expression-capable; type definitions use make-def.

Representative statement nodes

Tag	Shape
`func-def`	`[func-def name spec body]`
`set`	`[set name value ...]`
`return`	`[return value]`
`while` / `repeat`	`[while cond body]` / `[repeat var limit body]`
`go`	`[go body]` (statement only)
`defer` / `defer-error`	`[defer body]`
`make-def`	`[make-def name spec]` (struct/enum/sum/c-struct/fsm)

Representative expression nodes

Tag	Shape
`call`	`[call func-name args ...]`
`indirect-call`	`[indirect-call fn-ptr args]`
`ns-call`	`[ns-call head selector c-name args ...]` (foreign)
`make` / `cast`	`[make type payload]` / `[cast type value]`
`path-get` / `path-set`	field/index read & write
`enum-get`	`[enum-get enum-type variant]` (enum/sum/FSM tag)

The IR

IR is a normalized, more C-oriented form produced by ast-to-ir after borrow checking and fact analysis. Tags are prefixed ir-*.

Representative statement IR

Tag	Shape
`ir-func`	`[ir-func name spec body]`
`ir-assign`	`[ir-assign name value type ...]`
`ir-if` / `ir-while` / `ir-for`	statement conditional & loops
`ir-go`	`[ir-go task-name captures body]`
`ir-match`	`[ir-match subject sum-type branches]`
`ir-defer` / `ir-defer-error`	deferred cleanup

Representative expression IR

Tag	Shape
`ir-binop`	`[ir-binop op left right]`
`ir-call` / `ir-indirect-call`	direct & fn-ptr/closure calls
`ir-either-expr` / `ir-match-expr`	value-position conditional & match
`ir-block-expr`	expression block with prefix statements and a final value
`ir-vector-get/set`, `ir-struct-get/set`	indexed and field access
`ir-parse` / `ir-rule`	compiled parse expression & rule
`ir-const`	`[ir-const type value]`

Function spec objects

Function specs are object-based and shared across the parser, borrow checker, and codegen. Closures and fn-ptr! values use the same model plus capture metadata.

make object! [
    name: 'function-name
    params: [
        {name: 'a type: 'i32! borrowed?: false}
        {name: 'msg type: 'string! borrowed?: true}
    ]
    return-spec: [i32!]
    annotations: [#export]
]

C identifier names

There is no universal mapping from a source word to a C name. The emitted name depends on the symbol’s role, its module origin, and any renaming done during preparation. The shared helpers live in src/core/tools.r3.

Helper	Rule
`clean-extern-name`	Removes a leading `:`, replaces `-` with `_` and `?` with `_q`; no keyword renaming. Foreign C `getenv` stays `getenv`.
`clean-name`	Same substitutions, then prefixes `_` for names on an explicit historical keyword list (C keywords plus `getenv`, so Recoil `getenv` becomes `_getenv`). Not a complete C identifier validator.
`clean-symbol-name`	`clean-name`, then `_` for the reserved runtime names `string`, `vec`, `value`, `recoil_dict`, `recoil_map`, `strlen`.
`clean-var-name`	`clean-name`, then the `__recoil_` prefix unless already present.
`clean-global-name`	With a module origin: `<clean-module>__<clean-name>`, where `/` in the module name becomes `__`. Without one, falls back to `clean-symbol-name`.

Where each rule applies

Locals and parameters use clean-var-name. Alpha renaming adds a __bN binding suffix that stays in C: item-count__b3 becomes __recoil_item_count__b3. The number identifies a binding; it is not a stable source-level name.
Ordinary functions use clean-global-name; #export functions use clean-symbol-name with no module prefix; ABI c-fn! declarations use clean-name. A module’s Exports: list and C ABI #export are separate mechanisms.
Module globals: exported storage uses clean-global-name, private storage clean-var-name; the storage emitter strips ! from either.
Fields use clean-name without the local prefix. Types handle ! separately. Enum variants are the cleaned enum name without !, then __, then the cleaned variant name.

These substitutions are not collision-proof: foo-bar and foo_bar both clean to foo_bar. Generated internal names are not a stable ABI. For FFI or debugging, inspect the real C with r3 -s recoil.r3 --transpile file.rcl, and read the generated header when consuming an exported library.

Diagnostics

Every compile-time failure is both a human-readable string and a structured object carrying a stage (parser, module, borrow-checker, ir, codegen, cli, build), a kind, a stable code, and an optional location. The user-facing format and code ranges are documented in the Reference.

Introspection

You can inspect each stage's output:

# print the parsed AST
r3 -s recoil.r3 -a file.rcl

# emit the generated C without building a binary
r3 -s recoil.r3 --transpile file.rcl

Within the compiler, compile/ir returns IR (/optimized? selects pre- vs post-optimize). For the per-stage source map and the full ir-* inventory, see the repository docs: compiler pipeline, AST reference, and IR reference.

Testing with RUT

Compiler tests are declared in tests/*.reb and run with rut.r3, the only supported test runner. Prefer focused runs while working on a change:

# one test file
r3 -s rut.r3 -F core.reb

# tests whose title matches (partial match)
r3 -s rut.r3 -t "closure basic"

# a group, or several tags
r3 -s rut.r3 --group types
r3 -s rut.r3 -tg closure -tg fn-ptr

# list matching tests without running them; stop at the first failure
r3 -s rut.r3 --list
r3 -s rut.r3 --fail-fast

-t, -g, -tg and -F are repeatable; -j N sets the worker count (use -j 1 for serial debugging). Test sources are mostly .rcl files under tests/, each with a matching entry in a .reb declaration file.

Never run two RUT runs that share a working directory and cache. RUT holds a lock in cache/rut.lock/ and rejects an overlapping run; stale locks from dead processes are reclaimed automatically. Runs from separate working directories with independent caches may run in parallel. Do not commit or edit sources while a run is in progress, or the report is marked inconsistent.

The runner's execution model, caches, locks and failure modes are documented in the RUT guide.