FS3: CkLang StdoutLog provider — Log::info runs freestanding #43

Manually merged
hkowalski merged 9 commits from feat/043-fs3-cklang-stdoutlog-provider-log-info-runs-fr into master 2026-02-24 12:52:10 +01:00
Owner

FS3: CkLang StdoutLog provider — Log::info runs freestanding

Along the way:

  • FS4: the Memory floor freestanding (merged FS4a+FS4b)
  • infra(test): shared Alpine container for the gates — ~45% faster suite
  • docs: record the fn rec<N> recursion-depth spelling (owner decision)
  • FS5.0: compile-time per-actor arena-size calculation (pays M1 debt)
  • docs: schedule FS5.1 (arena assembly) + FS5.2 (fn rec recursion)
  • FS5.1: arena assembly — body-allocation, deepest call-chain, per-actor query
  • FS5.2: fn rec recursion — parse + cycle-check + arena depth sizing
  • fix(arena): opaque nominals report unbounded, not a fabricated 0 (review Finding 1)

Touches 94 files: current-state.md, docs/errors/E0901.md, docs/errors/E0902.md, docs/impl/memory-model/arenas.md, scripts/clean.sh, src/rust/crates/ck_driver/src/compile/fn_main.rs and 88 more.

FS3: CkLang StdoutLog provider — Log::info runs freestanding Along the way: - FS4: the Memory floor freestanding (merged FS4a+FS4b) - infra(test): shared Alpine container for the gates — ~45% faster suite - docs: record the `fn rec<N>` recursion-depth spelling (owner decision) - FS5.0: compile-time per-actor arena-size calculation (pays M1 debt) - docs: schedule FS5.1 (arena assembly) + FS5.2 (fn rec<N> recursion) - FS5.1: arena assembly — body-allocation, deepest call-chain, per-actor query - FS5.2: fn rec<N> recursion — parse + cycle-check + arena depth sizing - fix(arena): opaque nominals report unbounded, not a fabricated 0 (review Finding 1) Touches 94 files: `current-state.md`, `docs/errors/E0901.md`, `docs/errors/E0902.md`, `docs/impl/memory-model/arenas.md`, `scripts/clean.sh`, `src/rust/crates/ck_driver/src/compile/fn_main.rs` and 88 more.
Author
Owner

Nothing outside calls exec. It is pub(crate) in src/rust/crates/ck_driver/src/container.rs now.

Nothing outside calls `exec`. It is pub(crate) in `src/rust/crates/ck_driver/src/container.rs` now.
hkowalski approved these changes 2026-02-24 08:00:00 +01:00
hkowalski left a comment

Approving. I would have split src/rust/crates/ck_driver/src/container.rs differently, but not enough to hold this up.

Approving. I would have split `src/rust/crates/ck_driver/src/container.rs` differently, but not enough to hold this up.
@ -111,3 +159,3 @@
/// [`ContainerError`] if `docker` cannot be spawned. A non-zero *program* exit
/// is **not** an error — it is the returned code (the program's value).
pub fn exec(name: &str, bin: &str) -> Result<i32, ContainerError> {
pub fn exec(name: &str, work_subdir: &str, bin: &str) -> Result<i32, ContainerError> {
Owner

exec is public. Does anything outside the crate call it, or is it pub because that was easier? Every public item is a promise.

`exec` is public. Does anything outside the crate call it, or is it pub because that was easier? Every public item is a promise.
The payoff of Phase FS-2: `fn main() -> Unit effect Log { Log::info("Hello,
world") }` compiles, runs freestanding (no libc, no ck_rt), prints `Hello, world`
via a real write syscall, exits 0.

- stdlib (promote into live tests/corpus/stdlib): StdoutLog.ck rewritten from six
  `= intrinsic` ops to real CkLang — `fn info(msg) { Sys::write(1, msg) }` (raw
  message; the level prefix needs String::concat, rides the operators arc). Log.ck
  + Log.ckh (effect Log + LogLevel/LogLine) promoted. String.ck/Bytes.ck stay in
  stdlib_old (String is builtin; both carry ck_rt-backed = intrinsic bodies that
  don't lower freestanding — the doc's Promotes list is aspirational for FS3).
- codegen/call.rs (new): the SHARED intrinsic-call site emit_intrinsic_call +
  emit_syscall_leaf — checks syscall_spec before intrinsic_target, emits
  `call ck_syscall(nr, …)`. Lifted out of the freestanding-only module so a
  syscall leaf emits uniformly from ck_entry, a user fn, or (later) an actor.
- codegen/freestanding.rs: emit_fn_main_object_freestanding now REUSES
  user_fn::emit_user_fns + emit_effect — only ck_entry + the LinkMode predicate
  (fn_main_is_freestanding: freestanding iff no reachable leaf is a ck_rt symbol)
  are freestanding-specific. THIN FORK held (201 lines).
- driver/compile/fn_main.rs (new): try_emit_fn_main + resolve_fn_main +
  lower_dispatched_providers (cross-module lowering of CkLang provider fns) +
  emit_by_link_mode. Split from compile/mod.rs (no LoC cap raised).
- Gate: tests/corpus/freestanding_log + freestanding_log_run.rs; in RUNTIME_TESTED
  + LIVE_STDLIB_APPS. Corpus reconcile self-closes the live stdlib.

Two-link-mode contract: Log/StdoutLog deliberately live in BOTH stdlib dirs during
the transition (= intrinsic form for the libc apps which have no ck_syscall; CkLang
form for freestanding) — FS8 deletes the = intrinsic form. stdlib_old/README.md
corrected to state this real invariant (was: "never hold the same live unit").

Observed: freestanding `Hello, world`, exit 0, 9040-byte static ELF, empty NEEDED,
empty `nm -u`, no ck_rt/printf/malloc/__libc. Libc path (helloWorld 2.1 MB,
greeter) confirmed green.

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (no
libtest failures); R1-R4 arch caps pass (split, not raised); Docker e2e green.

Docs: stdlib.md §8 FS3 -> DONE; current-state.md FS3 note; stdlib_old/README.md.
A freestanding -> Int fn main mmaps a region, stores then loads 42 through the
machine-op leaves, munmaps, and the loaded value IS the exit code. No libc, no
ck_rt.

Merged per owner (FS4a couldn't be gated alone — no Region source without mmap;
a store-then-load round-trip must hold the same address).

- codegen/call.rs: machine-op leaf emit (the THIRD leaf kind) — size_of ->
  iconst(width), handle_at -> iadd, load -> Cranelift load, store -> store; PLUS
  custom mmap/munmap marshalling (Memory::os_acquire -> ck_syscall(9, 0, bytes,
  3, 0x22, -1, 0); os_release -> ck_syscall(11, region, bytes)). All syscall
  leaves now declare ck_syscall at its fixed 7-param arity (Cranelift requires
  one signature per imported symbol; unused slots zero-padded, trampoline loads
  a6 off the stack regardless).
- Turbofish -> width map: driver build_mem_types reads each turbofished Memory
  call's T off the CST via new ck_types::read_call_turbofish_type, threads a
  MemTypes (keyed by callee span) to codegen like VariantTags (ck_lower ↛
  ck_resolve held).
- Value-returning user fns (bounded R3-sliver, single scalar word): roundtrip
  -> Int + main -> Int return their value so exit code == loaded value. NO
  let/if/match/for (R2 stays out).
- freestanding.rs: -> Int ck_entry (returns the computed value, not 0); thin
  fork held (230 lines; shares emit_intrinsic_call/emit_user_fns/emit_effect/
  emit_block_value with the libc path).
- Round-trip needs no let: the region rides an S1 user-fn parameter (stable
  address); emit_arg/emit_value gained a Call arm for intrinsic leaves as values.
- Memory.ck/.ckh promoted to live stdlib but ALSO kept in stdlib_old (Buffer/
  StringBuf/Arena there still consume Memory::* — deleting it would trap E0301);
  byte-identical dual presence, deleted with stdlib_old at FS8 (Log/StdoutLog
  precedent). Recorded in both docs + stdlib_old/README contract.
- errno convention: returned-raw; a -errno detection + nonzero exit needs an
  if-branch (R2) and the rich OOM Panic::raise/R11 upcall isn't freestanding yet
  — both explicitly deferred to FS9, not faked. Success path fully implemented.
- Gate: tests/corpus/freestanding_mem + freestanding_mem_run.rs (exit 42 +
  no-libc), in RUNTIME_TESTED + LIVE_STDLIB_APPS.

Observed: exit code 42, 5072-byte static ELF, empty NEEDED, empty `nm -u`, no
ck_rt/printf/malloc/__libc. Libc path (greeter/helloWorld) green.

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0
(no libtest failures); R1-R4 arch caps pass (split, not raised); Docker e2e green.

Docs: stdlib.md §8 FS4 -> DONE (merged row); current-state.md FS4 note.
The *_run.rs gates each booted their own Alpine container (container::ensure's
reuse path never hit, because each used a unique name + per-test bind-mount, and
docker pins mounts at creation). Now the ~18 fixture gates share ONE long-lived
`ck-test` container.

- container.rs: TEST_CONTAINER = "ck-test"; ensure() is now idempotent under the
  parallel-startup race (docker "name already in use" -> re-check is_running in a
  bounded 50x20ms poll -> Reused, not error; other failures propagate); exec()
  takes a work_subdir and runs /work/<subdir>/<bin> with
  CK_META_STREAM=/work/<subdir>/meta.stream.
- run.rs: mount the /tmp/ck-work ROOT once at /work; each run keeps its own
  unique work SUBDIR and execs the subdir-prefixed path -> isolation preserved
  (distinct binary + distinct meta.stream per run; parallel docker exec is safe).
  Entry points gained a work_key param.
- lib.rs: new global `--work <key>` flag (defaults to the container name, so a
  plain `ckc run` is unchanged); USAGE + docs updated.
- 24 fixture gates: `--container <uniq> run` -> `--container ck-test --work <uniq>
  run`; their Teardown reclaims the per-run subdir but no longer `docker rm -f`s
  the shared container. execution.rs/meta.rs keep per-name containers (they test
  the container mechanism itself). clean.sh + `ckc stop` clean `ck-test`.

Timing (full cargo test --workspace): ~15s -> ~9s; 3 starting / 18 reused.
Isolation confirmed: two_tests_sharing_an_effect, greeter_two_tests,
soft_asserts, string_equality all green (a bled meta.stream would flip one).
CLI unregressed (ckc run greeter / freestanding_log verified). Test-infra only —
no compiler feature behavior changed, not an FS-cascade slice.

Gates: fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (no libtest
failures).
arenas.md §"Three nested bounded regions" flagged the recursion-depth bound
spelling as open. Owner decided 2026-07-01: `fn rec<N> name(...)` — a rec<N>
marker between `fn` and the name; rec<N> reserves N+1 stack frames (initial call
+ N re-entries), so `fn rec<10> fib(...)` needs 11 frames; a non-recursive fn is
depth 0 (1 frame). Not yet implemented (parser/resolver + depth×frame sizing is
a future language slice); recursion is skipped in the FS5.0/FS5.1 arena calc
until then.
The compiler statically CALCULATES an actor's arena size from its reachable
bounded types, per docs/impl/memory-model/arenas.md. The reusable, correctness-
critical composition core + function-frame sizing; recursion/mailbox/captures +
per-actor assembly are FS5.1, --mem surfacing is FS5.

- ck_types/src/arena_size/: type_size(&Ty) -> TypeSize { bounded_bytes,
  unbounded: Vec<UnboundedComponent{path,ty,reason}> }; frame_size (Σ of a fn's
  locals); arena_size -> ArenaSize { state, call_stack{frame,recursion_depth},
  mailbox{max_messages,message_size}, unbounded }. Lives in ck_types (owns type
  layout); InternerLayout adapter is the driver-consumable query FS5's --mem
  calls. recursion_depth (default 1) + mailbox factors (default 0) exposed as
  NAMED factors so FS5.1 fills them without reshaping the API.
- Composition: record = Σ fields; variant/data = max of cases + one tag word;
  bounded List<T,N> = handle + N×stride(T); scalars fixed. Widths reconciled with
  codegen::record::value_width (Int=8, String handle=16, tag=8); the calc adds
  the out-of-line N×stride list buffer codegen allocs.
- HONESTY: an unbounded type (bare String, List<T> without N, Unknown, generic
  Var, Fn, named-payload variant not collected by the front-end, recursive-type
  cycle) yields an UnboundedComponent with a reason — NEVER a fabricated number.
  Bounded List<T,N>'s N IS representable today (ConstValue::Int in the nominal
  args), so bounded lists are fully sizable.
- 77 exact-byte unit tests: scalars, records (sum, mixed/sub-word widths, field-
  order independence), variants (max+tag, the sum-vs-max distinguisher 24-not-32,
  not-first-wins), bounded lists (N=0/1/4/12, record stride), deep nesting,
  recursion cycle-detection, frames + assembly + named factors, and the
  InternerLayout real adapter end-to-end. Combined with FS5.1 this exceeds the
  owner's ≥100 floor.
- Invariant (documented): calculated bounded_bytes ≥ runtime ArenaHighWater for a
  fully-bounded actor; today's corpus uses bare String (growable interim) so it
  reports unbounded components — not a clean validation target until String<N>.

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (no
libtest failures); R3/R4 caps ok (mod.rs 633 + tests split, no cap raised).

Docs: stdlib.md §8 FS5.0 -> DONE; current-state.md arena section.
Owner split/scheduling 2026-07-01. FS5.0's scope (arena-size calc) was too large
for one slice, so:
- FS5.1: per-actor "arena = entire size of everything" assembly (body-allocation
  analysis, deepest call-chain, mailbox + capture factors via synthetic tests,
  per-actor total wired to real actors). Recursion skipped here.
- FS5.2: fn rec<N> recursion end to end (parser + cycle-check + arena depth×frame,
  rec<N> => N+1 frames) — "extremely important but needs special care."
Combined FS5.0+FS5.1 test floor: >=100 real exact-byte tests. Sequence:
FS5.0(done) -> FS5.1 -> FS5.2 -> FS5(Arena + --mem) -> FS6 ...
The per-actor "arena = entire size of everything" assembly on FS5.0's core
(additive API, no reshape). 116 combined arena tests (≥100 owner floor).

- ck_lower/arena.rs: frame_alloc_footprint(body) — body-allocation analysis
  ("each method's allocations get space"). Counts EXACTLY codegen's arena sites:
  Record (static frame), List (ck_rt_alloc buffer), String-typed BinaryOp::Add
  concat. NOT counted: literals (.rodata), param/local refs (registers), scalar
  Int + (iadd), projections, index-reads, calls (each callee is its own chain
  node — no double-count). Run-path lowering has empty TypeckResults so a value's
  type is often absent → reported UntypedAllocation (conservative/honest, never a
  fabricated number).
- ck_types/arena_size/assembly.rs: CallGraph + deepest_chain (bounded = MAX over
  root→leaf paths of Σ frame bytes — a stack holds one active chain; unbounded
  collected graph-wide; cycles surfaced Recursive, DAG-only). capture_size +
  assemble_arena. Pure (Floor-IR-free) so the path math is exactly unit-testable.
- ck_driver/compile/mem.rs: actor_arena_size(app, stdlib) -> ArenaSize — the
  driver query FS5's --mem consumes.
- Mailbox + captures: synthetic plumbing only (no send/lambdas in the front-end)
  — max×size / capture Σ, tested with labelled synthetic inputs, documented as
  not-wired-from-real-programs. Recursion left at depth 1 (FS5.2).
- Real actors: helloWorld → fully bounded, 0 arena bytes (its string is a rodata
  literal, not an allocation — honest); greeter → unbounded (its greet concat,
  UntypedAllocation on the run path). Both asserted.
- 39 new exact-byte tests (20 ck_types + 16 ck_lower + 3 driver): body-allocation,
  deepest-chain incl. the max-path-vs-sum distinguisher (106 not 116), mailbox/
  capture synthetic, assembly, real-actor queries. Total 116.

Note for FS5: the query lowers with empty TypeckResults → UntypedAllocation for
typed allocs; --mem should feed the type-checked HIR to get real numbers.

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (no
libtest failures); R3/R4 caps ok (split, not raised).

Docs: stdlib.md §8 FS5.1 -> landed; current-state.md arena section.
The compile-time recursion story end to end (owner: "extremely important, needs
special care"). Runtime overflow trap (FS9) and recursive-fn execution (R2) are
correctly out of scope.

- Parser: `rec` is a hard keyword; a `RecBound` node wraps `rec < IntLit >` in a
  NEW slot between `fn` and the name (ck_parser/fn_decl.rs), distinct from
  post-name generics. CST reader FnDecl::recursion_depth(). Malformed markers →
  E0120 (missing `<`) / E0121 (missing/non-int depth) / E0122 (missing `>`).
- Cycle detection: pure Tarjan-SCC in ck_types/recursion.rs (analyze_recursion),
  wired per-module in ck_driver/compile/recursion.rs off NAME RESOLUTION (so a
  self-call inside a match arm is seen without R2). SAFETY PROPERTY: a recursive
  cycle (self-loop or SCC>1) with no rec<N> member → E0901 (cites the cycle's
  fns); a rec<N> on a non-recursive fn → E0902 (hard error, no-warnings rule).
  Intra-module (cross-module cycles are already E0601). Full docs docs/errors/.
- Arena sizing: deepest_chain_rec (SCC-condensation longest path). Direct self →
  frame × (N+1); mutual/transitive SCC → (Σ member frames) × (max N + 1), the max
  rule a documented conservative over-approximation. Depth flows CST →
  FnDecl::recursion_depth → CallGraph → deepest_chain_rec. Frame modelled by the
  fn's parameter slots (honest run-path lower bound; body-local types need
  TypeckResults).
- fib acceptance: fn rec<10> fib(n: Int) → clean (self-cycle bound-declared) +
  stack size 88 bytes = 8 (one Int param) × 11 frames — the N+1 semantics exact.
- 38 new tests: parser (9), cycle analysis (13, incl. 100k-node no-stack-overflow),
  arena sizing (10, incl. differing-bounds→max), driver e2e (6). All exact-byte.
- Ripple fixed, NO caps raised: moved Token to ck_lex/token.rs (syntax_kind.rs was
  at the 809 cap); extracted build_fn_entries (R4). ck_highlight gained Rec.

Surfaced (not blocking): u64 isn't a lifted primitive yet, so fib(n: u64) would
size unbounded — the exact-byte demo uses Int (noted for the primitive-set slice).
The existing tests/corpus/fibonacci app is the natural first rec<N> adopter (left
untouched — can't run yet, out of scope).

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (the
2 Docker soft-assert "FAILED" lines are the inner-CkLang harness, identical on
clean master via git stash). R3/R4 caps ok.

Docs: arenas.md (rec<N> implemented + SCC/mutual sizing rule); docs/errors/
E0901.md + E0902.md (new); stdlib.md §8 FS5.2 → landed; current-state.md.
Adversarial review of the arena calc found a silent under-count — the single most
dangerous failure class for this subsystem. walk_nominal's `else -> walk_record`
fallback returned bounded(0) for any nominal that wasn't a list/variant but also
wasn't a field-bearing record: real stdlib newtypes (Region, Handle, Bytes, Span),
`type` aliases, `data` parents. A newtype field/param silently sized to 0 instead
of its 8-byte handle — a fabricated number (honesty-rule violation), inconsistent
with codegen's newtype ABI, breaking bounded_bytes() >= ArenaHighWater. Latent
(fixtures use Int) but live via frame_size in compile/recursion.rs.

Root cause: record_fields alone can't tell an empty *registered record* from an
opaque nominal — both yield empty fields. Fix adds a real discriminator:
- ck_hir: new `record_types: HashSet<TypeId>` + `Interner::is_record`;
  register_record_fields inserts unconditionally (before its empty-field early
  return), so even a fieldless record is marked. Newtypes/aliases/data are bare
  alloc_type calls → not records. No ck_resolve change.
- TypeLayout gains `is_record(id)`; InternerLayout delegates to Interner::is_record,
  FakeLayout to its records map.
- walk_nominal: list -> variant -> is_record?walk_record : unsizable(Opaque). A
  declared empty record still -> bounded(0); an unrecognized nominal -> unbounded
  (Opaque), NEVER a fabricated 0.
- F3: corrected the false InternerLayout::new docstring (list_id=None now genuinely
  reports unbounded, was silently 0).

5 new tests, verified fail-pre-fix / pass-post-fix: opaque_nominal_is_unbounded_
not_a_fabricated_zero, empty_record_and_opaque_nominal_are_distinguished,
interner_layout_newtype_is_opaque_not_zero, is_record_tells_records_from_bare_
nominals (+ an assertion in empty_field_sets_stay_sparse). empty_record_is_zero
stays green (the distinction is real, not a regression). No live number changed
(fib still 88; greeter still UntypedAllocation on the untyped run path).

Review clean bills held on all 8 other axes (sum-vs-max, N+1, double-count,
cycles, width, overflow, graph traversal, remaining unbounded handling).
Findings 4 (N=0 empty-list) + 5 (sub-word padding) are non-live — documented, not
fixed.

Gates (all green): fmt pass; clippy 0 warnings; cargo test --workspace exit 0 (no
libtest failures; ck_types 214 passed).
hkowalski manually merged commit c360abae96 into master 2026-02-24 12:52:10 +01:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
northgale/cklang-bootstrap!43
No description provided.