bootstrap: the compiler becomes a pinned tool — src/cklang stops needing src/rust #214
hkowalski
dferreira
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/214-bootstrap-the-compiler-becomes-a-pinned-tool-s"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
bootstrap: the compiler becomes a pinned tool — src/cklang stops needing src/rust
Along the way:
Touches 55 files:
.gitignore,CLAUDE.md,LICENSE,README.md,scripts/cklang_ai.sh,scripts/editor.shand 49 more.Reads well. One question on
src/rust/crates/ck_resolve/src/const_values.rs, nothing blocking.@ -0,0 +155,4 @@// Reason: the fold threads its collected initializers, the module table, the// interner, and the memo + cycle sets through one recursive walk; bundling them// would only move the unpacking inward.fn eval_expr(eval_exprreads, decides and writes in one function. The deciding part is the piece I would want to test on its own. Worth splitting?Split. The decision in
eval_expris a pure function now, with its own tests insrc/rust/crates/ck_resolve/src/const_values.rs.Second read on
src/rust/crates/ck_resolve/src/const_values.rs. Nothing to add.The bootstrap is not a codebase we live in; it is a tool `ckc.ck` is compiled BY, worked on only when `ckc.ck` needs it. This makes that structural. Both src/cklang gates now resolve their compiler from a declared pin instead of a build tree, and both run GREEN with `src/rust` renamed off the disk and the compiler cache cleared — the split is now a file move, not a leap. The pin is a DIRECTORY, not a binary: `ckc` resolves the freestanding crt beside its own executable (ck_driver/src/link.rs:41) and refuses to link without it, so pinning the compiler alone would pin half a tool and leave the other half floating in a build tree. `bin/ckc-bootstrap-v<N>/{ckc,ck_start.o}`, hashed whole by the same construction corpus-selfhost.sh uses over src/cklang. `[profile.bootstrap]` — optimized, but `debug-assertions` and `overflow-checks` ON. Plain `--release` is 10M and silently drops both; the bootstrap leans on 13 `debug_assert` sites (one documented at ck_parser/src/operator_table.rs:174 as the detector for a bundled-Operators.ck bug), and unchecked overflow in a compiler whose whole job is Law 1 arithmetic turns a loud refusal into a silent miscompile of the program that replaces it. 129M debug -> 9.5M, guards intact. The `src/rust/target/debug/ckc` default is DELETED, not redirected. A build tree is not a pin: it changes underfoot, it is unversioned, and its presence made every gate result quietly a function of whatever `cargo build` last left there. Absent pin, missing binary, sha mismatch -> refusal naming the reason. `--ckc` survives only to validate a CANDIDATE bootstrap, and downgrades the verdict to "CANDIDATE OK — UNPINNED, NOT a gate result" so an override run can never be quoted as a gate pass. The binaries are gitignored: ~9.5M a version, kept by git forever, from a source tree that is not public. So the pin DECLARES an identity and verifies it rather than distributing it. A fresh clone therefore refuses until a bootstrap is built at the commit scripts/.env names — the correct failure. Also, each one a bill this work surfaced rather than a digression: - src/cklang/bin -> src/cklang/.cache. Root bin/ is tracked pinned INPUT; that directory is ignored generated OUTPUT. Two dirs named `bin` with opposite meanings is the carve-out Law 2 forbids. - time-compile.sh joins the pin. It compiles against src/cklang/stdlib and refuses a dirty src/cklang — a src/cklang tool that was resolving its compiler from a build tree, so its reports were keyed by rev but built by whatever happened to be on disk. The report now records the bootstrap version: two bootstraps emit two images from one source. - timing reports become `<stamp>-<rev>.md`, stamp first, so `ls timing/` reads chronologically and re-measuring one commit records a second observation instead of erasing the first. Existing four renamed by git-add time. - tictactoe/ttt_train_report/cklang_ai deliberately do NOT take the pin, and now say so: they drive tests/corpus/, the bootstrap's own oracle, and want the freshest build precisely so a stale binary cannot lag the source they exist to exercise. They travel with the bootstrap. - cklang_ai.sh had `ws` as a local array in one fn and a local string in another. No runtime bug, but a reader trap and a linter error; the string is now `swh`, which is what it holds. Licence: MIT OR Apache-2.0 -> source-available, all rights reserved (read it, do not use it). `license-file` cannot be used — all 20 member crates inherit `license.workspace`, so it is an SPDX `LicenseRef-`, which claims nothing. selfhost-verify: 287 passed, 0 failed. corpus-selfhost: 66 rows, 0 fails. Both with src/rust absent. shellcheck -x: 0 across all seven scripts.The bootstrap is a TOOL that `ckc.ck` is compiled BY, worked on only when `ckc.ck` needs it (owner, 2026-08-17). Keeping it in the same tree as the language made that impossible to see: agents landed in src/rust by default, and two stdlib roots had to be kept apart by a hand-spelled rule whose violation caused the Bounds.ck incident. So the tree splits. This side keeps what implements the v0 compiler and what proves it; ~/Projects/CkLang keeps the language. History is preserved whole — this repo is a clone of the pre-split tree with the other side removed, so `git log --follow` still reaches every commit that shaped these files. Kept here: src/rust (20 crates), crt/ck_start.s, tests/{capability, corpus, corpus_parked, corpus_test_run, frame_pin, lex, parser, resolve, sizes_pin, types}, and the scripts that drive them — test.sh/test-rotate.sh (the cargo gate), deploy.sh, run.sh, clean.sh, ckeditor-integration.sh, and the three ML/Othello stones (cklang_ai.sh, tictactoe.sh, ttt_train_report.sh) which drive tests/corpus and deliberately want the freshest build, not a pin. Removed here: src/cklang, tests/corpus_pure, docs, editors, reports, timing, the plan files, and the three src/cklang scripts with their pin library. docs/ is deliberately NOT duplicated. Two copies of a design doc is two designs; the CkLang repo is the single source of truth and CLAUDE.md §6 says so. What this repo gains instead is its own CLAUDE.md: the demand-driven maintenance rule, the BROKEN-cell queue discipline, and the pin-cutting protocol — including why a pin is built with [profile.bootstrap] and never --release, and why the pin is `ckc` PLUS `ck_start.o`. The machine is still one machine: rotate-exec.sh and cpu_broken.md are present on both sides because both repos can build at once.The pass could already WRITE a record element into a list buffer and had no way to read one back, because an index-read could only ever produce a Load of a scalar and a record is not a scalar. What closes it is not a new node: a record value in the Floor IR IS a frame base pointer, and the element's frame is already laid out in place inside the buffer, so the strided address xs[i] computes IS the value. The read is the address, and every record position the pass already had takes it from there. - lower_element_read: a scalar element Loads its repr; a record element returns the Offset itself, typed with the read's checked type so it classifies back as ValueRepr::Record. - index_receiver: ONE classifier for what counts as a list receiver, a literal or a local, so xs[i] and xs[i].f cannot disagree about what a list is. record_lower's dedicated xs[i].f arm is DELETED — with the index-read able to produce a record value, a second path to the same bytes is a second layout waiting to disagree. - element_frame: a record element's fields are written BY NAME into the element type's declared ABI row, not in the order the literal spells them. That row is the layout the projection, a fn boundary, and a seeded list parameter all already assume; the write now agrees with its readers instead of with the source's spelling. - seed_lists: a record-element list PARAMETER seeds from the same row, so caller and callee agree because they read one table. Codegen gains one arm: emit_record_value accepts an Offset, which is already an address — the same no-copy sub-address the Load arm yields for a nested field. R3: list_lower.rs hit 569 against the 540 cap, so the READ moved out rather than the cap moving up. list_store.rs became list_slot.rs and took the read in beside the write, which is where it belonged — they already shared element_address, and the module header now states that as its reason for existing. lower.rs sits at exactly 540, which is why the read joined an existing module instead of opening a new one; the next ck_lower module has to split lower.rs first. Ratchets, all in this commit: - record_as_list_element: lower: ListLayout -> runs: 41. CELL_COUNT 416 -> 424 for seven further cells — the element bound to a local, crossing a fn boundary, through a list parameter, out-of-order element fields, a nested record two deep, the store door still refusing, and one bug below. - CARRIER_COUNT 12 -> 11: `lists` retired, its gap was exactly this one. It left tests/sizes_pin/unbounded_inventory.txt entirely, 12 apps -> 11. - UNMARKED_CELL_COUNT 149 -> 148. - BROKEN_CELL_COUNT 48 -> 50, for two bugs this UNCOVERED rather than caused — the ListLayout refusal was the lid on both, and both reproduce at the parent commit: record_literal_field_order_is_lost_crossing_a_fn — a plain record literal is laid out in the order its fields are WRITTEN, not DECLARED, so P { b: 41, a: 1 } handed to fn pick(p: P) -> Int { p.b } returns 1. Self consistent inside the caller, wrong at the boundary, and no refusal fires. The fix is the one applied to list elements here, one tier over: lower_record_ctor should take its layout from record_layout_of_ty and store by name, keeping plan_lowered_fields as the row-less fallback. Not done here because it is the record tier's door and would move cells this slice does not own. a_list_literal_in_a_test_fn_body_fails_the_suite — any arena-allocating construct in a test fn body fails the test, with no read of it and a trivially-true predicate, while the identical literal on the run entry is correct. It had never executed: corpus_test_run/lists stopped at ListLayout in a sibling body, so its two Int assertions had never once run. corpus_run does not catch it either — lists/run.md declares exit: 0 and the driver returns 0 even when the tests it ran failed, so that app has been green on a gate that never looked at the tally. The element STORE (xs[i] with mut <record>) still refuses, honestly and on purpose: a record element write is the per-field write construction already does, aimed at a computed address, and that is D1's wide-element half. record_list_element_store_does_not_lower is the pair that pins the asymmetry. Gates: fmt and clippy --all-targets -D warnings clean, zero warnings; capability matrix green at 424 cells; door and unbounded-inventory gates green. Two gates are RED and were RED at the parent commit, both split leftovers pointing at paths that moved to the CkLang repo: ck_docs_linkchecker wants docs/, and architecture::floor::f2 wants src/cklang/stdlib/Floor.ck. They are not this change's and are left for a decision about whether they belong here at all.Point 1 of the Main3.ck header: a size is a number the compiler computes from source (Law 1), so it must be spellable by the `const` that derives it rather than by a literal repeated beside a comment. `ConstScalar` + the interner's registry hold the settled values; `ck_resolve::const_values` is the ONE evaluator, hung inside `link_project` because linking is the pass every harness funnels through and a fold hanging off any single entry point would be a fact one harness had and another did not. `value_slot_const` reads the registry in the lift, `check_value_slot_node` re-runs the same predicate so check and lift cannot disagree, and `fold_closure_consts` stops evaluating anything — it now only admits a registered value against its declared ascription. No operator glyph is matched in Rust; `+` reaches its method name through `standard_operator_table`, exactly as lowering desugars it. That work was not gate-green, and the gate could not say so. `scripts/test.sh` ran `cargo test --workspace` with no `--no-fail-fast`, so cargo stopped at the first failing test binary and every target after it silently never ran. `ck_docs_linkchecker` failed first — it asserts a `docs/` tree that `07a49fef` deleted from this repo — and behind that one stale red sat three more the gate never reached: - R3: ck_types/src/lift.rs at 914 lines against the 862 cap; - R4: ck_resolve/src/link.rs `link_project` at 110 against the 100 cap; - floor::f2, reading src/cklang/stdlib/Floor.ck, which the split moved out. A gate whose output depends on which red comes first is not a gate. It reported one failure when the truth was four. - ck_docs_linkchecker is DELETED (owner call). `docs/` is in neither repo now, so the check could never go green as written, and a gate that cannot pass teaches readers to skip the suite. Its row leaves the R1 crate graph with it. - floor::f2 keeps its teeth on what this repo OWNS: tests/corpus/**/Floor.ck, 65 intrinsic sites. The sibling's stdlib floor answers to the sibling's gate (CLAUDE.md §6) — a path this repo cannot see is not a rule it can enforce. A non-vacuity assert now fails loudly if that corpus file ever disappears, because a silently-empty half A is the one way this rule rots. - R3 is resolved by remedy (a), splitting, never by raising the cap: lift/value_slot.rs takes the generic-argument slotting unit it already was — lift_generic_args, the two value_slot_* recognisers, and the value-param binding they share. lift.rs 914 → 651. - R4 likewise: bind_scalar_aliases_to_fixpoint lifts the alias-transparency second pass out whole. link_project 110 → 85. - test.sh runs --no-fail-fast, so one stale red can never again hide an unbounded number of live ones. Gate: GREEN, 0 failures, 0 warnings, full sweep.The owner removed the app (11 files: the program, its `tests/sizes_pin` and `tests/frame_pin` entries). It was the corpus's largest source of noise — 240 of the 262 rows in the unbounded inventory were its unlowerable bodies — and the request was zero errors, not a quieter report. What makes this more than a `git rm` is that eight other places named it, and each had to be settled honestly rather than edited to compile: - `door::CARRIERS` loses its `ml_othello` row, but the comment left behind records that this is NOT a retired gap. `MixedArmValues` and `MutLocalShape` are still open; they simply have no corpus witness now. The carrier ledger is evidence of what the compiler cannot do, and deleting the demonstration does not close the gap. `CARRIER_COUNT` 11 → 10 with the same distinction spelled out. - `a_listed_carrier_is_downgraded_not_refused` hardcoded the app and died with it. It now walks the whole registry — a test pinning a law over a registry should not have a favourite row. Broadening it immediately found that the law has TWO halves at different scopes: every carrier is DOWNGRADED (never E0760), but only `tests/corpus` is positive-only, and `mock_dispatch` legitimately reports E0503 standalone because supplying that provider is the substrate's job. Conflating them made the broadened test fail on a correct app. That split exposes the real cost of this deletion: ml_othello was the ONLY carrier under `tests/corpus`, so the positive-only half is VACUOUS today. The assertion is kept live and the vacuity is stated in the test, not hidden — it re-arms by itself the moment a corpus carrier appears. - M20 fell 64 → 14 (ml_othello's 8x8 board literal was the corpus worst case for items in one list literal; it is now algorithms_BFS_trapped). The `grammar_structure` reconciliation caught this on its own and printed the corrected row, which is the gate working. The DECLARED bound stays at 128: it is a language limit written into `Grammar.ckx`, not a lint threshold, and lowering it would reject a 30-item literal that is legal today. The seeding note now says that instead of citing a program nobody can read. - The unbounded inventory is re-blessed: 11 apps / 262 components → 10 / 22. Pure shrink, the only direction that harness permits. - Four historical citations (two doc comments, three capability provenance notes) keep their claims — the history is true — and gain the deletion date so no reader hunts for a missing path. `tests/corpus/README.md` is the exception: its "without the floor" bullet was a CLAIM, that the integer-native capabilities do not depend on the carved-region substrate, and ml_othello was its only demonstration. Law 3 says a behavioural claim exists as a runnable cell or it does not exist, so the bullet is rewritten as an open gap rather than quietly dropped from the list.