feat(coherence): the concrete-tag runtime proof lands — the annotation-path E0742 miss fixed, tag_dispatch exits 42 (debt #32) #78

Manually merged
mdoorn merged 8 commits from feat/078-feat-coherence-the-concrete-tag-runtime-proof into master 2026-04-09 14:54:20 +02:00
Owner

feat(coherence): the concrete-tag runtime proof lands — the annotation-path E0742 miss fixed, tag_dispatch exits 42 (debt #32)

Along the way:

  • kill(selectors): of_str/str_len/as_str die — String reads its own record via .value; str_abi_word deleted; R5 widens to call.rs
  • docs(plan): plan_ai.md — the capability path: training you can interrogate
  • feat(tensor): tensor_attention — integer-native hard attention; the sparse gradient is a bit-assertion
  • docs: the truth-up — register hygiene (this run's closures + the falsely-open sweep), stale-comment scrub, stdlib.md syscall tables, ratification queue recorded
  • fix(traits): impl heads resolve traits project-globally — app modules implement Prelude traits; .value covers scalar reps
  • docs(handoff): close #66 (54ba82a+f6bc995 landed), mint #68 (project_traps vs run check-depth divergence)
  • feat(tensor): tensor_compose — C1 certified block injection; certification composes

Touches 60 files: docs/impl/08-diagnostics.md, docs/impl/type-system/README.md, docs/impl/type-system/TY2-synthesis-core.md, docs/impl/type-system/TY3-generics-and-instantiation.md, docs/impl/type-system/inference.md, docs/impl/type-system/type-representation.md and 54 more.

feat(coherence): the concrete-tag runtime proof lands — the annotation-path E0742 miss fixed, tag_dispatch exits 42 (debt #32) Along the way: - kill(selectors): of_str/str_len/as_str die — String reads its own record via .value; str_abi_word deleted; R5 widens to call.rs - docs(plan): plan_ai.md — the capability path: training you can interrogate - feat(tensor): tensor_attention — integer-native hard attention; the sparse gradient is a bit-assertion - docs: the truth-up — register hygiene (this run's closures + the falsely-open sweep), stale-comment scrub, stdlib.md syscall tables, ratification queue recorded - fix(traits): impl heads resolve traits project-globally — app modules implement Prelude traits; .value covers scalar reps - docs(handoff): close #66 (54ba82a+f6bc995 landed), mint #68 (project_traps vs run check-depth divergence) - feat(tensor): tensor_compose — C1 certified block injection; certification composes Touches 60 files: `docs/impl/08-diagnostics.md`, `docs/impl/type-system/README.md`, `docs/impl/type-system/TY2-synthesis-core.md`, `docs/impl/type-system/TY3-generics-and-instantiation.md`, `docs/impl/type-system/inference.md`, `docs/impl/type-system/type-representation.md` and 54 more.
mdoorn approved these changes 2026-04-09 10:42:43 +02:00
mdoorn left a comment

Happy with this. I ran the suite against the branch and it is green, src/rust/crates/ck_driver/tests/cross_impl_add_run.rs included.

Happy with this. I ran the suite against the branch and it is green, `src/rust/crates/ck_driver/tests/cross_impl_add_run.rs` included.
@ -0,0 +195,4 @@
.is_ok_and(|s| s.success());
assert!(ok, "failed to build the `ckc` binary");
});
find_near_exe("ckc").expect("`ckc` binary next to the test executable")
Owner

The expect message on find_near_exe is the only thing a user sees if this fires. Does it say what they should do, or only what we assumed?, at src/rust/crates/ck_driver/tests/cross_impl_add_run.rs:198.

The expect message on `find_near_exe` is the only thing a user sees if this fires. Does it say what they should do, or only what we assumed?, at src/rust/crates/ck_driver/tests/cross_impl_add_run.rs:198.
@ -0,0 +130,4 @@
}
/// `nm <binary>` output (all symbols; scanned for forbidden libc/`ck_rt` names).
fn nm_all(binary: &Path) -> String {
Owner

nm_all reads, decides and writes in one function. The deciding part is the piece I would want to test on its own. Worth splitting?, at src/rust/crates/ck_driver/tests/tag_dispatch_run.rs:133.

`nm_all` reads, decides and writes in one function. The deciding part is the piece I would want to test on its own. Worth splitting?, at src/rust/crates/ck_driver/tests/tag_dispatch_run.rs:133.
@ -623,0 +623,4 @@
let tag_name = match segments.as_slice() {
[tag] => *tag,
[head, tag] if *head == interner.resolve(interner.type_data(parent).name) => *tag,
_ => return None,
Owner

The catch-all arm means a variant added later compiles and lands here without a word. Can the match on _ name every variant so the compiler tells us?, at src/rust/crates/ck_types/src/lift.rs:626.

The catch-all arm means a variant added later compiles and lands here without a word. Can the match on `_` name every variant so the compiler tells us?, at src/rust/crates/ck_types/src/lift.rs:626.
Author
Owner

It said what I assumed. Reworded in src/rust/crates/ck_driver/tests/cross_impl_add_run.rs so it names the input that broke it, which is what the bug report will need.

It said what I assumed. Reworded in `src/rust/crates/ck_driver/tests/cross_impl_add_run.rs` so it names the input that broke it, which is what the bug report will need.
Author
Owner

Named them all. A new variant is a compile error in src/rust/crates/ck_types/src/lift.rs now, which is where we want to find out, at src/rust/crates/ck_types/src/lift.rs:626.

Named them all. A new variant is a compile error in `src/rust/crates/ck_types/src/lift.rs` now, which is where we want to find out, at src/rust/crates/ck_types/src/lift.rs:626.
Author
Owner

Split. The decision in nm_all is a pure function now, with its own tests in src/rust/crates/ck_driver/tests/tag_dispatch_run.rs.

Split. The decision in `nm_all` is a pure function now, with its own tests in `src/rust/crates/ck_driver/tests/tag_dispatch_run.rs`.
Follow-up to the tag-discriminating coherence key (debt #32, landed b1334df):
close the annotation-path miss the coordinator's end-to-end probe surfaced, and
land the blocked RUNTIME proof.

ROOT CAUSE of the annotation-path miss. `value_slot_tag` (the lift's
bare-variant→ConstValue::Tag rule) WAS reached and resolved correctly, but a
SECOND, independent pass — `param_args::check_type_expr`, the value-parameter
argument-form validator — fires **E0742** ("USD is not a literal") on ANY node
in a value slot, walking the CST directly and never consulting the tag rule. So
every `let a: Money<USD>` annotation (and turbofish) was rejected before the lift
mattered, cascading E0728/E0718. The fix: `check_type_expr` now asks the SAME
`value_slot_tag` predicate — a bare (or declared-type-qualified) nullary variant
of the parameter's declared `data` type is a concrete tag, legal; only a genuine
non-literal (a binder `n`, a type `Int`) still fires E0742. `value_slot_tag` also
gained the qualified `Currency::USD` spelling (verified by the declared type's
name), so both spellings work in every type-annotation and turbofish position.

RUNTIME PROOF (`tests/corpus/tag_dispatch`, exit 42, freestanding, in
RUNTIME_TESTED). `newtype Money<currency: Currency> = i64` with `data Currency {
USD, EUR }`; a local `trait Code<T>` with `impl Code<Money<USD>> { … = 20 }` and
`impl Code<Money<EUR>> { … = 22 }`. `main` CONSTRUCTS `Money::<USD>(0)` /
`Money::<EUR>(0)` and dispatches `a.code()`→20, `b.code()`→22 — each routed by
the concrete tag in the operand's type. A mis-route flips the exit (EUR→USD gives
20→exit 2; USD→EUR gives 22→exit 1). Green 42 IS the discrimination, end to end.
Before the tag-discriminating key both impls keyed as `Money` and collided.

Two UNRELATED gaps found and worked around (reported, not fixed here — outside
debt #32/#37):
  1. Cross-module trait-impl resolution: an app module cannot `impl Add<…>` for
     a stdlib-Prelude trait — `Add` does not resolve in the app scope
     (`register_impl`'s `lookup_trait` misses), so the impl silently drops. The
     proof uses a LOCAL trait `Code` (same-module → resolves), exercising the
     identical tag-dispatch mechanism.
  2. `.value` on a SCALAR-rep newtype (`= i64`): the newtype convention (a2901fd)
     only registers component fields for a `(| … |)` tuple rep, so `.value` on a
     scalar-rep newtype is E0718. The proof returns discriminating constants from
     the impl bodies, so it needs no `.value`.

Gates: fmt clean; clippy -D warnings clean; build clean; workspace suite — only
the 3 pre-existing reds (one_equals_two, strings_differ canaries + the
tensor_* runtime-coverage red, all present at a2901fd, none mine); architecture
22/22 (check_type_expr's tag arm extracted to `check_value_slot_node`, the test
helper split to `insert_money_over_currency`, the registry row kept terse — all
under R3/R4); conformance ratchet ≤94 (2 passed, not loose); stdlib_type_clean
green; tag_dispatch_run green (exit 42).
The three condemned Bytes String word-selectors retire to pure CkLang over the
landed newtype convention (a2901fd). Per-selector spelling in Bytes.ck:

  of_str(s) = Region { offset: s.value.position, size: s.value.len }
  str_len(s) = s.value.len::Int
  as_str(r, len) = String((| r.offset, len::usize |))

`= intrinsic` bodies deleted; the shim fn names + .ckh contracts are unchanged,
so all ~44 `Bytes::`/`Meta::` call sites are untouched.

Rust killed:
  - ck_resolve/intrinsics.rs: the ("Bytes", of_str|str_len|as_str) condemned arm
  - ck_codegen/resolution.rs: is_machine_op's trailing name-keyed Bytes arm
  - ck_codegen/call.rs: the str_abi_word fn + its dispatch arm (~55 lines)

One generic law added (NOT a per-name special case): ReturnMode::ValueWords now
materializes the return value's flat words via the shared arg::store_value_at —
the same path args/lets use — instead of assuming a frame-base pointer. This
lets a `String` value (a two-word `Str`, produced by the newtype ctor's
frame→flat bridge) return by value, which the old base-only assumption could not.
Extracted as emit_value_words_return to hold R4.

Ratchet:
  - deleted the 3 Bytes CONDEMNED_COMPOUNDS entries (arch floor.rs)
  - R5_ABI_PATH widened to cover call.rs; its bare opcode/symbol literals
    (load/store/ck_syscall) hoisted to named constants in format.rs/resolution.rs
    (off the R5 path) so no bare-identifier literal remains for R5 to flag

Gates: fmt 0, clippy 0 (-D warnings), build clean, architecture 22/22,
conformance ratchet 94 (honest), stdlib_type_clean green. Full suite: only the
3 designed canaries + the pre-existing tensor runtime-coverage red fail.
Net production Rust: -53 LOC.
The owner's ruling made law: the old math was the license, the
destination is capabilities only this substrate can hold. Two axes
separated (better learning — not our race; better knowing — ours). The
five capability stairs, ordered with dependencies: C1 tensor_compose
(certified block injection — bit-identity of function, provable
freezing, certification composes), C2 tensor_bisect (divergence as a
bit-event, culprit found in log2 replays), C3 tensor_unlearn (exact
unlearning by replay + the influence delta as artifact — the
regulator-grade headline), C4 tensor_isolate (run-long non-interference,
ATT-2 generalized), C5 fed v2 (shard replay-checks, Byzantine detection
by verification). Rules of the path restated; non-goals named (no
scale/throughput/GPU chase); the horizon: the micro-LM whose training
run is itself a verified artifact.
Argmax attention over the matmul substrate with NO softmax, exp,
normalization, or division anywhere — add/sub/mul/shift/compare only,
by design. Scores S = Q·Kᵀ are ONE existing matmul call over K's
swapped stride pair (the free transpose reused as a claim); selection
is first-max-wins argmax with the tie rule pinned by a deliberate 6-6
tie in the hand table (strict > to replace, ties break low); output
gathers V rows; margins are the integer-native confidence signal (the
tie row's margin 0 is the ambiguity witness).

The crown (ATT-2): hard attention's sparse backward as a BIT-assertion.
scatter_add routes dO only into selected V rows — proven twice: the
zero-init gradient matches the hand table exactly (row2 == [0,0], not
epsilon-small), and a sentinel-seeded dV survives backward with the
non-selected row BIT-UNCHANGED. Float autodiff cannot state this claim.
ATT-3: with Q,K frozen, V trains to EXACT convergence (loss 8843 →
3368 → … → 0; routed rows land on target, the un-routed row provably
never moves). The stone even caught its author: a hand-arithmetic slip
(8842) was named by exit 16 on the first run — the methodology policing
the mathematician.

New reusable kernels in Tensor.ck/.ckh: max2, fill, argmax_row/rows,
gather_rows, scatter_add_rows; Att public module for the E0302
cross-module read law; attention.results ships 20 deterministic words
into the runner's cross-invocation act. Spec: ATT-1/2/3 PROVEN, ATT-4
(2^x soft scoring, stacking, multi-head) DEFERRED.

Verified (orchestrator-run): cklang_ai.sh exit 0 — six suites
13/9/7/10/6/7 all green, race+fed+backprop+attention byte-identical
across 3 invocations; tensor_attention run 42 + 7 green; siblings
unchanged.
PART 1 — handoff.md debt register, archaeology-verified against landing
commits rather than trusted carry-forward prose:

- CLOSED with citation: #12/#14/#16/#36 (844c262, four silent-hole
  diagnostics E0751-E0753), #15 (NamedTuple ruled + 291bf9a projection),
  #19 (e449d67 killed the transitional +-as-concat), #28 (9b7407f return-area
  + 291bf9a named-tuple projection), #29 (e80d162 per-stamp widths +
  ccb8d1c test-path fix), #32/#37 (b1334df tag-discriminating coherence key;
  #37 verdict MOOT), #34 (already "effectively closed" at d068dfe, re-lost
  to carry-forward drift), #38/#63 (291bf9a), #50 (acd3f07/8cadc9b args +
  9b7407f returns — the guard recommendation is moot), #51 (f4b2457 citation
  added), #56/#59 (d35ec86 letter-form coercion on any postfix receiver),
  #61 (this commit's own stale-comment scrub), #62 (5e3b067 citation added).
- Falsely-open carry-forward bug fixed: #18/#26/#27/#31 were each closed in
  their own session's handoff but a later "carry forward unchanged" range
  re-absorbed them — struck for good, with the bug named so it isn't
  repeated. #43 independently re-verified already correctly excluded.
- #57 narrowed: the mmap/generic-call ROOT wall is CLOSED (ccb8d1c); the
  test-fn-parameter injection wall is the residual, now its own entry (#65).
- #58 confirmed IN FLIGHT on sibling slice w3-traits-key (6850bbe/dace54b),
  not yet merged into this base — left open, not prematurely closed.
- New entries #64-#67: surface `if` parses but lowers to Unsupported (no
  HIR arm); #57's residual (test-fn params are effect/mock-only); tag-
  parameterized annotation-lift E0742 in app modules, in flight on
  w3-traits-key (af3db65/54ba82a) with two more unrelated gaps it found;
  the R5 bare-literal scanner's opcode-string blind spot (pending, not
  landed, per w2-selectors/w2c-selectors).
- Awaiting-ratification list recorded: MAX_FILE_LOC 822->840->845,
  E0748/E0749/E0750-53 (batch ratification wanted, one mid-flight
  collision already rebase-resolved), the ShortCircuit-vs-Match deviation
  (w3-shortcircuit blocker analysis), the sret law-premise note
  (slice_function_calls.md owner decision 2's "Cranelift handles the
  split" is empirically false per 9b7407f - the return-area convention is
  CkLang's own compiler mechanism), dead_code="deny" re-enable timing.

PART 2 — stale comment/doc scrub, each instance verified by grep/read
before editing:

- ck_types/src/narrow.rs: Ty::EmptyString variant references corrected to
  the well-known-id nominal (no such Ty variant exists).
- ck_lower/src/floor_ir.rs, floor_ir/scalar.rs, lower.rs, arena.rs: dead
  ExprKind::BinaryOp/UnaryOp (HIR and Floor IR) references removed; the
  "unchecked mock sliver" framing narrowed to what's still true (only a
  mock seed-fn/state-initializer body is untyped; mock OP bodies are typed
  since the operators-B2 finishing pass); lower.rs's module docs rewritten
  to describe the actual current mechanism (MethodCall -> Call ->
  codegen's intrinsic-target fold, not a direct HIR-BinaryOp-to-ScalarOp
  fold that no longer exists).
- ck_driver/src/compile/mock.rs: stale `[Ty::String]` spelling fixed; the
  module doc's "untyped sliver" claim narrowed to seed-fn/state-init only.
- ck_codegen/src/value.rs: the "until the slice-8 Ty::String collapse"
  comment corrected (Ty::String is already collapsed; the residual is the
  shape-keyed ValueRepr::Str arm).
- ck_driver/tests/architecture/main.rs: EXPECTED_GRAPH comment's
  "Primitive/Ty" direct-read claim corrected (no Primitive type exists).
- ck_driver/tests/freestanding_mem_run.rs, ck_load/src/loader.rs: stale
  Memory.ck/Memory::*/Sys::* naming corrected to the live Alloc.ck/Floor::*
  naming.
- ck_driver/tests/dogfood_tests_run.rs + tests/corpus/dogfood_tests/Main.ck:
  the "os_acquire in a test fn is a wall" prose reworded to the true
  residual (the ROOT wall lifted by ccb8d1c; the PARAMETER-injection wall
  remains, debt #65).
- tests/corpus/alloc_roundtrip/Main.ck: the "await the cross-module
  generic-stamps fix" / "named-tuple slice" comments updated to landed
  fact (e80d162, 291bf9a).
- docs/stdlib.md: the layered-model diagram, floor-ISA table, and per-OS
  syscall table renamed from the historical Memory::/Sys:: split to the
  live single Floor:: module (+ Alloc:: on top), widened to the actual
  leaf roster (full scalar ISA, network syscalls) per tests/corpus/stdlib/
  Floor.ck; the FS-cascade session-ledger narrative (SS7-9) is left
  untouched as historical record, by design.
- docs/types.md: "literal size arguments only" updated against the landed
  const-param mechanism (OQ#25, owner ruling B).
- docs/impl/**: the pre-SS1b `::<T>` turbofish spelling for scalar-value
  coercion corrected to the landed letter form `::T` (no angle brackets)
  across type-system/{README,inference,TY2,TY3,type-representation}.md and
  08-diagnostics.md's E0739/E0742 rows; genuine specialization turbofish
  examples (generic calls, variant refs, empty literals) are left
  untouched — the two forms are named as distinct CST productions.

Gates: fmt 0, clippy -D warnings 0, workspace build 0/0, architecture
22/22, full workspace suite green (only the 3 pre-declared soft-assert
canaries + the pre-existing tensor runtime-coverage red, all allowed).
PART A — cross-module trait-impl resolution (slice-8 blocker). An app-owned
module could not `impl Add<...>` for a trait declared in the Prelude: the
coherence pass's `lookup_trait` consulted only the app module's scope, missed,
and SILENTLY DROPPED the impl — the miss surfaced only as a confusing E0728 at
the operator/method call site.

The fix is ONE generic resolution rule, no Prelude special case: `lookup_trait`
now falls back to the project-global trait roster (`Interner::trait_id`, the
binding sibling of the existence-only `trait_name_declared`) after a per-module
`scope.lookup` miss. Sound because trait names are project-globally unique (no
shadowing — strict-orphan rule, first-declaration-wins), so at most one TraitId
per name; a module that declares its own trait still binds it locally first. The
four duplicated `lookup_trait` copies (coherence / impl_conformance /
trait_methods / fields) are consolidated into the one roster-aware resolver.

A genuinely-unknown trait in an impl head now DIAGNOSES (E0753, the impl-head
twin of the `composes` check) instead of dropping the row.

PART B — `.value` on a scalar-rep newtype. A scalar-rep newtype
(`newtype Money<c> = i64`, `newtype Handle = usize`) previously got E0718 on
`.value`. Extended the same generic rep-type law: the scalar-rep newtype now
stores its underlying leaf (`newtype_scalar_target`), `.value` types as that
scalar and lowers as a zero-instruction identity retype (the value IS the one
word). `ty_repr` classifies a scalar-leaf newtype by its repr regardless of
type arguments (a tag param like `Money<USD>` is compile-time only), and the
lowering rep-read recognizer handles the scalar (identity) shape alongside the
named-tuple (frame-bridge) shape.

PROOF: `tests/corpus/cross_impl_add` — an app-module `impl Add<Money<USD>,
Money<USD>, Money<USD>>` over the Prelude's `Add`, body using `.value` and
`Money::<USD>(...)`, resolves through `+` and runs exit 42
(`cross_impl_add_run.rs`, registered in RUNTIME_TESTED). Negative golden
`tests/types/impl_head_unknown_trait.cktest` — an impl head naming a
nonexistent trait diagnoses E0753. Seven parser/impl fixtures whose isolated
`impl <UndeclaredTrait>` now correctly E0753s: trap added + re-blessed
(ratchet holds at 94).
The anti-recertification stone. Stage 1 trains a 2-lane block to exact
convergence (W: 0→8→12→14→15→16, loss 1024→0) and CERTIFIES it — proof
chain + weights witness (words [16,16], certified output fb(x)=32) kept
on disk and in a certified-copy region. Stage 2 injects the block BY
REFERENCE (same region, no copy — injection is a pointer, not a
migration) into a two-stage model and trains only the surrounding Wn
(0→2→3→4, loss 16384→4096→1024→0, lr shift 12).

CMP-1 · certificate survival: after all of stage 2's training the
block's words bit-equal the witness AND its certification forward
re-passes in situ — with the reasoning stated in prose: deterministic
kernels ⇒ bit-identical weights ⇒ identical function ⇒ the prior claims
ledger remains valid; the re-check is the witness of the implication,
not its source. CMP-2 · provable freezing, the ATT-2 discipline: a
777-sentinel guard lane immediately adjacent to Wn's single trained
lane survives bit-unchanged — the update provably stayed inside its
fence. CMP-3 · the composition learns to exactly zero. CMP-4 deferred
(adapters/mismatch — empirical, not claimed). Exit 42, codes 10..18.

Pure reuse: Tensor.ck byte-unchanged — the stone composes dot/loss/
backward_dot/sgd_update through its own Cmp module (E0302 precedent).
compose.results ships 12 deterministic words into the runner's
cross-invocation act; scoreboard grows to SEVEN suites; plan_ai.md §2/§3
mark C1 LANDED.

Verified (orchestrator-run): cklang_ai.sh exit 0 — seven suites
13/9/7/10/6/7/7 all green, five-stone cross-invocation byte-identical;
tensor_compose run 42 + 7 green; siblings unchanged.
mdoorn manually merged commit a3fa3caabf into master 2026-04-09 14:54:20 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
northgale/cklang-bootstrap!78
No description provided.