diff --git a/.mermaid/01-stack-overview.md b/.mermaid/01-stack-overview.md new file mode 100644 index 000000000..ec9602de3 --- /dev/null +++ b/.mermaid/01-stack-overview.md @@ -0,0 +1,145 @@ +# 01 — Stack overview + +Status: describes what is in this checkout. Nothing here claims Foundry parity or a graphical low-code builder. Neither exists. + +The stack is a set of layers. Each layer depends on the ones below it. Values stay in source-owned planes. Layers above the contract pass addresses, masks and witnesses. + +Legend: green = in code on main, yellow = lab or probe crate (excluded from the root workspace), grey = another repository, red dashed = gap (not built). + +## 1.1 Layers + +An arrow from A to B means "A depends on B" in a `Cargo.toml` (see [02](02-crate-dependencies.md)). Dashed red boxes are work that no code does yet. + +```mermaid +flowchart TB + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef ext fill:#eeeeee,stroke:#666,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + subgraph L0["contract: zero-dep types"] + CON["lance-graph-contract: ClassView, WideFieldMask, ScopeSpec, Marking"]:::shipped + end + + subgraph L1["dt64: DR and DT64 binding"] + DT["lance-graph-dt64: BindStore, RouteLog, MapTable, FrameStore, LaneBinding"]:::shipped + end + + subgraph L2["execution"] + MR["lance-graph-mask-risc: Program, MaskOp, execute"]:::shipped + QK["lance-graph-quack: Binder, Query, lower"]:::shipped + R2["lance-graph-r2il: FoldDialect, prepare, run"]:::shipped + XL["lance-graph-xlog: Cube, stage, commit"]:::shipped + RA["lance-graph-relalg: Rel, normalize, run"]:::lab + end + + subgraph L3["authorization"] + RBAC["lance-graph-rbac: authorize_scoped"]:::shipped + OGA["OGAR: ogar-auth, ogar-rbac"]:::ext + end + + subgraph L4["write and scenario"] + TG["thought-git-probe: commit_cycle, SlotPool, Receipt"]:::lab + end + + subgraph L5["lab"] + LAB["foundry-parity-lab: 64-store read-only lab"]:::lab + end + + G2["apply a decision to a binding, G2"]:::gap + G3["scenario overlay frame, G3"]:::gap + G4["write-back composed end to end, G4"]:::gap + + DT --> CON + QK --> MR + QK --> CON + R2 --> MR + R2 --> DT + XL --> QK + XL --> DT + RA --> R2 + RA --> MR + RBAC --> CON + OGA -->|git| RBAC + TG --> CON + LAB --> DT + LAB --> MR + + RBAC -.-> G2 + G2 -.-> DT + TG -.-> G3 + G3 -.-> DT + R2 -.-> G4 + XL -.-> G4 + TG -.-> G4 +``` + +Reading notes: + +- The contract has no runtime dependencies. Every other layer builds on it. +- DT64 is the join. It ties a class, a route, a frame and a lane together and holds addresses only. +- The execution crates evaluate masks. `lance-graph-mask-risc` is the one bulk evaluator. Quack and r2il build programs for it. Quack's module doc says it may build a `Program` and must never evaluate one. +- Authorization is split. `lance-graph-rbac` has the generic kernel. OGAR supplies the login and the grant data, and calls that kernel. +- `thought-git-probe` is a probe. Its own doc says the full thought git API is a proposal and that it has no production land, rebase or persistence pipeline. +- `foundry-parity-lab` is read-only. Its doc says it has no write-back, no actions and no Foundry comparator. + +Gaps in the diagram: + +- **G2.** A `ScopedDecision` exists. Nothing applies it to a DT64 binding. See [04](04-read-path.md). +- **G3.** `SlotPool::branch` and `Cube::stage_closure` exist. A scenario overlay frame does not. +- **G4.** r2il can run an admitted method's effects through its fold dialect and returns a receipt. xlog has `stage` and `commit` for edges. thought-git has `commit_cycle`. No crate read here depends on more than one of the three. They are not composed into one write-back. + +Sources: + +- `crates/lance-graph-contract/Cargo.toml:10-17` (no runtime dependencies) +- `crates/lance-graph-dt64/src/lib.rs:1-30` (what DT64 binds and what it does not hold) +- `crates/lance-graph-mask-risc/src/lib.rs:1-20` (executor, oracle, tiled execution) +- `crates/lance-graph-quack/src/lib.rs:1-25` (operators lower to a `Program`; never evaluates) +- `crates/lance-graph-r2il/src/lib.rs:1-14` (fold reading of R2IL over mask-risc) +- `crates/lance-graph-xlog/src/lib.rs:1-20` (admitted chain query over Quack and mask-risc) +- `crates/lance-graph-relalg/src/lib.rs:1-27` (what it holds) +- `crates/lance-graph-rbac/src/authorize.rs:168-250` (`ScopedDecision`, `authorize_scoped`) +- `OGAR/crates/ogar-rbac/src/lib.rs:120-162` (`OgarRbac::authorize_user`) +- `crates/thought-git-probe/src/lib.rs:1-8` (probe status) +- `crates/foundry-parity-lab/src/lib.rs:1-26` (lab scope: read-only) +- `crates/lance-graph-xlog/src/cube.rs:856-945` (`stage`, `stage_closure`, `commit`) +- `crates/thought-git-probe/src/slots.rs:103-125` (`SlotPool::branch`) +- `crates/thought-git-probe/src/receipt.rs:153` (`commit_cycle`) +- `crates/lance-graph-r2il/src/binding.rs:1-20,321,377` (admitted effects, `prepare`, `run`) +- Cargo edges: see the sources of [02](02-crate-dependencies.md). + +## 1.2 What each layer holds + +| layer | crates | what it holds | what it does not do | +|---|---|---|---| +| contract | `lance-graph-contract` | Shared types: `ClassView`, `WideFieldMask`, `ScopeSpec`, `Marking`, set algebra | Run anything. It has no runtime dependencies | +| dt64 | `lance-graph-dt64` | Class schemas, method bindings (ON x WHEN x DO), the DR route log, the map table, frames, lane bindings | Hold values, evaluate, persist, or decide who may read | +| execution | `lance-graph-mask-risc`, `-quack`, `-r2il`, `-xlog`, `-relalg` | A flat program form, a name-to-number binder, an R2IL fold dialect, an admitted chain query, a relational rewrite | Loop over rows in the front ends. `mask-risc` is the only bulk engine | +| authorization | `lance-graph-rbac`, OGAR `ogar-auth` and `ogar-rbac` | Login to canonical user, grants by class, row scope, field mask | Reach the read path. G2 is open | +| write and scenario | `thought-git-probe` | Rubicon outcomes, a version-store stand-in, commit witnesses, receipts, a slot pool, squash gates | Run in production. It is a probe | +| lab | `foundry-parity-lab` | A 64-store fixture with an independent oracle, three arms (object, mask, INT8 cost) | Write, act, or compare against Foundry | + +Sources: + +- `crates/lance-graph-contract/src/class_view.rs:1057-1061` (`ClassView`) +- `crates/lance-graph-contract/src/rbac.rs:113-132` (`ScopeSpec`) +- `crates/lance-graph-dt64/src/lib.rs:24-30` (stored and not stored) +- `crates/lance-graph-quack/src/lib.rs:12-24` (the shapes it leaves out) +- `crates/lance-graph-r2il/src/lib.rs:1-14` +- `crates/lance-graph-xlog/src/lib.rs:1-26` +- `crates/lance-graph-relalg/src/lib.rs:1-27` +- `crates/thought-git-probe/src/receipt.rs:60-130` (`VersionStore`, `CommitWitness`, `CommitError`) +- `crates/thought-git-probe/src/fixtures.rs:28-40` (`Outcome`) +- `crates/thought-git-probe/src/squash.rs:96-100,290-300` (`squash_gate`, `protected_squash`) +- `crates/foundry-parity-lab/src/lib.rs:1-26` + +## 1.3 Two limits worth knowing + +- **No production issuer for execution permission.** `ExecPermit` in r2il is documented as a lab permission, not a production authorization. Who issues it is not decided. `BindStore::admit` succeeding is never a substitute for it. +- **Several records, no join drawn.** r2il returns a `Receipt` for a run. thought-git has its own `Receipt` and a `CommitWitness`. This overview draws no code that ties them to each other or to an authorization decision. + +Sources: + +- `crates/lance-graph-r2il/src/binding.rs:123-130` (`ExecPermit`) +- `crates/thought-git-probe/src/receipt.rs:80-100,366-380` (`CommitWitness`, `Receipt`) +- `crates/lance-graph-r2il/src/binding.rs:1-20,221-235` (r2il runs admitted effects; its `Receipt`) diff --git a/.mermaid/02-crate-dependencies.md b/.mermaid/02-crate-dependencies.md new file mode 100644 index 000000000..fff05d96c --- /dev/null +++ b/.mermaid/02-crate-dependencies.md @@ -0,0 +1,180 @@ +# 02 — Crate dependencies + +Status: every edge below was read from a `Cargo.toml`. Dev-only edges are listed in the table and left off the diagrams. + +Legend: green = workspace member of the root `Cargo.toml`. Yellow = excluded from the root workspace (the crate is its own workspace root, or is a lab crate). Grey = another repository. + +## 2.1 Edges inside lance-graph + +Solid edges are path dependencies. Edges marked `git` use a git coordinate. + +```mermaid +flowchart LR + classDef member fill:#d9f2d9,stroke:#2d7a2d,color:#111 + classDef excl fill:#fff3cd,stroke:#a67c00,color:#111 + classDef ext fill:#eeeeee,stroke:#666,color:#111 + + subgraph MEM["workspace members"] + CON["lance-graph-contract"]:::member + DT["lance-graph-dt64"]:::member + MR["lance-graph-mask-risc"]:::member + QK["lance-graph-quack"]:::member + R2["lance-graph-r2il"]:::member + XL["lance-graph-xlog"]:::member + RB["lance-graph-rbac"]:::member + end + + subgraph EXC["excluded, own workspace root"] + RA["lance-graph-relalg"]:::excl + TG["thought-git-probe"]:::excl + LAB["foundry-parity-lab"]:::excl + end + + ND["ndarray, git master"]:::ext + OGL["ogar-loco, OGAR git main"]:::ext + OGR["ogar-r2il, OGAR git main"]:::ext + + DT --> CON + RB --> CON + MR -->|git| ND + QK --> MR + QK --> CON + R2 --> MR + R2 --> DT + R2 --> CON + R2 -->|git| OGL + R2 -->|git| OGR + XL --> QK + XL --> MR + XL --> DT + XL --> CON + XL -->|git| ND + RA --> MR + RA --> R2 + RA --> CON + RA -->|git| OGL + RA -->|git| OGR + TG --> CON + LAB --> DT + LAB --> MR + LAB --> CON + LAB -->|git| ND +``` + +Reading notes: + +- `lance-graph-contract` has an empty `[dependencies]` table. It has one build-dependency, `serde_yaml`, used only by `build.rs`. +- `lance-graph-dt64` and `lance-graph-rbac` depend on the contract and nothing else. +- `lance-graph-mask-risc`, `lance-graph-xlog` and `foundry-parity-lab` name ndarray directly. All three use the same git coordinate. The root `[patch]` points that coordinate at the sibling checkout, so the graph holds one ndarray package. +- `lance-graph-r2il` is a workspace member and still depends on two OGAR crates by git. `lance-graph-relalg` does the same and is excluded. +- `thought-git-probe` depends on the contract only. +- In this checkout `foundry-parity-lab` does not depend on `lance-graph-relalg`. + +Sources: + +- `Cargo.toml:6-37` (workspace members: contract line 7, mask-risc 8, quack 9, r2il 15, xlog 21, dt64 24, rbac 35) +- `Cargo.toml:133` (`lance-graph-relalg` in `exclude`) +- `Cargo.toml:221` (`thought-git-probe` in `exclude`) +- `Cargo.toml:240` (`foundry-parity-lab` in `exclude`) +- `Cargo.toml:500-515` (root `[patch]` for the contract and ndarray) +- `crates/lance-graph-contract/Cargo.toml:10-24` (empty dependencies, build-dependencies) +- `crates/lance-graph-dt64/Cargo.toml:8-10` +- `crates/lance-graph-mask-risc/Cargo.toml:19-20` +- `crates/lance-graph-quack/Cargo.toml:12-18` +- `crates/lance-graph-r2il/Cargo.toml:17-30` +- `crates/lance-graph-xlog/Cargo.toml:8-17` +- `crates/lance-graph-rbac/Cargo.toml:7-8` +- `crates/lance-graph-relalg/Cargo.toml:11-28` +- `crates/thought-git-probe/Cargo.toml:10-13` +- `crates/foundry-parity-lab/Cargo.toml:18-27` + +## 2.2 Edges that cross into OGAR and back + +lance-graph depends on OGAR through the r2il crates. OGAR depends on lance-graph through the contract and the rbac crate. Both directions use a git coordinate. + +```mermaid +flowchart LR + classDef member fill:#d9f2d9,stroke:#2d7a2d,color:#111 + classDef ext fill:#eeeeee,stroke:#666,color:#111 + + subgraph LG["lance-graph"] + CON["lance-graph-contract"]:::member + RB["lance-graph-rbac"]:::member + R2["lance-graph-r2il"]:::member + end + + subgraph OG["OGAR"] + LOCO["ogar-loco"]:::ext + OR2["ogar-r2il"]:::ext + AUTH["ogar-auth"]:::ext + ORB["ogar-rbac"]:::ext + ENC["ogar-encryption"]:::ext + VOC["ogar-vocab"]:::ext + DIR["ogar-dir-core"]:::ext + end + + R2 -->|git| LOCO + R2 -->|git| OR2 + OR2 -->|path| LOCO + LOCO -->|git| CON + AUTH -->|path| ENC + AUTH -->|git| CON + ORB -->|path| AUTH + ORB -->|path| VOC + ORB -->|path| DIR + ORB -->|git| CON + ORB -->|git| RB +``` + +Reading notes: + +- The root `[patch]` maps the git coordinate of `lance-graph-contract` to the workspace member. That keeps one contract package in the graph when OGAR crates pull it by git. +- There is no such patch for `lance-graph-rbac`. `ogar-rbac` fetches it from git `main`. +- `ogar-r2il` also lists `lance-graph-contract` by git, as a dev-dependency. It is not drawn. + +Sources: + +- `crates/lance-graph-r2il/Cargo.toml:25-26` (`ogar-loco`, `ogar-r2il` by git) +- `Cargo.toml:500-504` (`[patch."https://github.com/AdaWorldAPI/lance-graph"]`) +- `OGAR/crates/ogar-loco/Cargo.toml:26` (contract by git) +- `OGAR/crates/ogar-r2il/Cargo.toml:11-19` (`ogar-loco` by path; contract by git under `[dev-dependencies]`) +- `OGAR/crates/ogar-auth/Cargo.toml:13,20` (`ogar-encryption` by path, contract by git) +- `OGAR/crates/ogar-rbac/Cargo.toml:14,19,24,29,33` (`ogar-auth`, `ogar-vocab`, contract, `lance-graph-rbac`, `ogar-dir-core`) + +## 2.3 Dependency table + +"Dev" lists dev-dependencies, which are left off the diagrams. + +| crate | in root workspace | direct dependencies | dev-dependencies | +|---|---|---|---| +| lance-graph-contract | member | none (runtime) | serde_yaml, serde, criterion (and `serde_yaml` as a build-dependency) | +| lance-graph-dt64 | member | contract | none | +| lance-graph-mask-risc | member | ndarray (git master) | causal-edge, contract | +| lance-graph-quack | member | mask-risc, contract | lance-graph-arm-discovery, causal-edge, deepnsm-v2 | +| lance-graph-r2il | member | mask-risc, dt64, contract, ogar-loco (git), ogar-r2il (git) | quack | +| lance-graph-xlog | member | quack, mask-risc, dt64, contract, ndarray (git master) | lance-graph-onnx | +| lance-graph-rbac | member | contract | none | +| lance-graph-relalg | excluded, own `[workspace]` | mask-risc, r2il, contract, ogar-loco (git), ogar-r2il (git) | none | +| thought-git-probe | excluded, own `[workspace]` | contract | none | +| foundry-parity-lab | excluded, own `[workspace]` | dt64, mask-risc, contract, ndarray (git master) | none | +| ogar-auth | OGAR | ogar-encryption, contract (git), crypto crates | not read | +| ogar-rbac | OGAR | ogar-auth, ogar-vocab, ogar-dir-core, contract (git), lance-graph-rbac (git) | not read | + +How to build the excluded crates: + +- `relalg`: `cargo test --manifest-path crates/lance-graph-relalg/Cargo.toml`. +- `thought-git-probe`: run from its own directory. +- `foundry-parity-lab`: `cargo test --release --manifest-path crates/foundry-parity-lab/Cargo.toml`. + +Sources: + +- `crates/lance-graph-quack/Cargo.toml:20-30` (dev-dependencies) +- `crates/lance-graph-r2il/Cargo.toml:32-36` (dev: quack) +- `crates/lance-graph-mask-risc/Cargo.toml:32-37` (dev-dependencies) +- `crates/lance-graph-xlog/Cargo.toml:19-21` (dev: onnx) +- `crates/lance-graph-contract/Cargo.toml:26-32` (dev-dependencies) +- `crates/lance-graph-relalg/Cargo.toml:8-11` (own workspace root) +- `crates/thought-git-probe/Cargo.toml:9-10` (own workspace root, run from its directory) +- `crates/foundry-parity-lab/Cargo.toml:6-7,18` (run commands, own workspace root) +- `OGAR/crates/ogar-auth/Cargo.toml:9-45` (dependencies) +- `OGAR/crates/ogar-rbac/Cargo.toml:9-33` (dependencies) diff --git a/.mermaid/03-dr-dt64-binding.md b/.mermaid/03-dr-dt64-binding.md new file mode 100644 index 000000000..7c0b25263 --- /dev/null +++ b/.mermaid/03-dr-dt64-binding.md @@ -0,0 +1,372 @@ +# 03 — DR / DT64 binding + +Status: in code on main (crate `lance-graph-dt64`, a workspace member). + +DT64 is the store that joins four things without copying any value: + +- a class (`ClassId`, read through a `ClassView`), +- a route (which class a DR position points at, and at which version), +- a frame (a sealed reference to the source revisions), and +- a lane (a source-owned plane of values). + +The crate holds addresses and witnesses. It holds no business value, no evaluator and no persistence path. + +Legend: green = in code on main, red dashed = gap (not built). + +## 3.1 DR: reservations, route log, map table + +DR reserves space per class and records where each DR position routes. Both the route log and the map table tick one shared version clock. + +```mermaid +classDiagram + direction LR + class Width { + <> + W64 + W256 + positions() + } + class Axis { + tile Width + tiles u32 + reserved() + for_survivors() + } + class Extent { + identity Axis + property Axis + predicate Axis + reserved_cells() + tile_bytes() + } + class VersionClock { + current() + tick() + fork() + } + class RouteLog { + route(position, class, identity) + check(position, class, identity) + retract(position) + routed_at(version, position) + claimed(class, extent, version) + } + class RouteRefusal { + <> + OutsideReservation + FutureVersion + } + class MapTable { + advance() + advance_at() + seal() Frame + map(frame, tile) TileMap + diagnose_at(version, tile) + } + class LocalMap { + prepare(width, globals) + local(global) u8 + global(local) Option + } + class MapRefusal { + <> + TooMany + Duplicate + Unmapped + OutOfTile + } + Axis --> Width : tile + Extent --> Axis : identity, property, predicate + RouteLog --> VersionClock : ticks + MapTable --> VersionClock : ticks + RouteLog ..> RouteRefusal : refuses with + LocalMap ..> MapRefusal : refuses with + MapTable --> LocalMap : one log per tile +``` + +Reading notes: + +- `Width` is DR64 (`W64`) or DR256 (`W256`). A DR64 tile has 64 positions, a DR256 tile has 256. +- A class reserves an `Extent`: one `Axis` each for identity, property and predicate (methods). A reservation costs nothing. Only identity tiles a route touches count as claimed. +- `RouteLog` is append-only. Only routing is versioned; what a classid means is not. +- `MapTable` changes only by appending. An old frame replays the layout in force when it was sealed. +- `MapTable::diagnose_at` takes a bare version number and returns a read-only diagnostic, not an executable map. Execution goes through `MapTable::map` with a frame. + +Sources: + +- `crates/lance-graph-dt64/src/dr.rs:32-45` (`Width`) +- `crates/lance-graph-dt64/src/dr.rs:47-95` (`Axis`, `for_survivors`) +- `crates/lance-graph-dt64/src/dr.rs:97-140` (`Extent`) +- `crates/lance-graph-dt64/src/dr.rs:165-186` (`VersionClock`) +- `crates/lance-graph-dt64/src/dr.rs:190-208` (`RouteRefusal`) +- `crates/lance-graph-dt64/src/dr.rs:208-336` (`RouteLog`) +- `crates/lance-graph-dt64/src/dr.rs:555-566` (`MapRefusal`) +- `crates/lance-graph-dt64/src/dr.rs:568-660` (`LocalMap`) +- `crates/lance-graph-dt64/src/dr.rs:860-1100` (`MapTable`, `map`, `diagnose_at`) + +## 3.2 Frames and guards + +A frame is a seal over three revisions. Only a `FrameStore` can issue one. A frame compares by the identity of its issue, never by its numbers. + +```mermaid +classDiagram + direction LR + class FrameRevisions { + source u64 + relation_map u64 + coordinates u64 + } + class FrameStore { + seal() Frame + advance(revisions) RouteVersion + issued(frame) RouteVersion + check(frame) + } + class Frame { + version() RouteVersion + revisions() FrameRevisions + same_issue(other) + } + class FrameRefusal { + <> + ForeignFrame + SourceChanged + MapChanged + CoordinatesChanged + FrameResealed + } + class Focus { + revision u64 + } + class Scope { + id u64 + } + class Guards { + frame Frame + focus Focus + scope Scope + check(store, now) + } + class GuardRefusal { + <> + Frame + FocusChanged + ScopeChanged + } + FrameStore --> FrameRevisions : current + FrameStore ..> Frame : issues, no public ctor + FrameStore ..> FrameRefusal : refuses with + Guards --> Frame + Guards --> Focus + Guards --> Scope + Guards ..> GuardRefusal : refuses with + GuardRefusal --> FrameRefusal : Frame variant +``` + +Reading notes: + +- `Frame` has no public constructor. A caller cannot build one from numbers. +- `FrameStore::check` accepts only the store's current seal. A frame from another store is `ForeignFrame` even at equal numbers. A stale frame names the first revision that moved, or `FrameResealed` if none did. +- `Focus` is motion, not reference. A new focus over the same frame is not a new frame. +- `Scope` here is an opaque `u64` id. The source says which scope each lane serves. DT64 compares ids. It does not decide who may read. + +Sources: + +- `crates/lance-graph-dt64/src/dr.rs:355-365` (`FrameRevisions`) +- `crates/lance-graph-dt64/src/dr.rs:365-378` (`FrameRefusal`) +- `crates/lance-graph-dt64/src/dr.rs:379-484` (`FrameStore`) +- `crates/lance-graph-dt64/src/dr.rs:484-511` (`Frame`) +- `crates/lance-graph-dt64/src/dr.rs:513-555` (`Focus`, `Scope`, `Guards`, `GuardRefusal`) + +## 3.3 ClassView, BindStore, lanes + +`BindStore` binds a class to its methods. `ClassView` (a contract trait) supplies the field basis. `LaneSource` is the socket a source owner implements. + +```mermaid +classDiagram + direction LR + class ClassView { + <> + fields(class) + fields_at(class, rev) + lens_fields(lens) + } + class ClassSchema { + schema_rev SchemaRev + field_count usize + extent Extent + } + class MethodBinding { + id MethodId + sig_rev SigRev + on WideFieldMask + when Option~Guard~ + reads WideFieldMask + effects Vec~Effect~ + } + class BindStore { + register(view, class, schema, methods) + route(position, class) + resolve(view, class, rev) Resolved + method_ref(class, method) MethodRef + admit(method_ref) MethodBinding + bind_lanes(view, frames, source, request) + } + class Resolved { + field_view(mask) + field_meaning(mask, schema) + mask_where(schema, pred) + action_ordinals() + } + class LaneSource { + <> + lane_of(class, rev, field, basis, frame) + borrow(addr) + frames() FrameStore + } + class LaneAddr { + space CoordSpace + lane u16 + scope Scope + } + class LaneRequest { + route RouteAt + rev SchemaRev + fields WideFieldMask + frame Frame + scope Scope + selection CoordSpace + } + class LaneBinding { + addr(field) + check(store, view, frames, source) + lane(store, view, frames, source, field) + borrow_all(store, view, frames, source) + } + BindStore --> ClassSchema : per class + BindStore --> MethodBinding : per method + BindStore ..> ClassView : reads basis through + BindStore ..> Resolved : resolve returns + BindStore ..> LaneBinding : bind_lanes makes + LaneRequest ..> BindStore : input to bind_lanes + LaneBinding --> LaneAddr : one per field + LaneBinding ..> LaneSource : asks and borrows +``` + +Reading notes: + +- A `MethodBinding` is ON x WHEN x DO. `on` is the field mask that triggers it, `when` is an optional guard, `effects` is the ordered DO list. +- `BindStore::method_ref` issues a `MethodRef` stamped with the store identity. `BindStore::admit` checks that reference against the binding. It refuses with `SignatureRevisionMismatch`, `GuardDropped`, `GuardAdded`, `GuardMismatch`, `BindingMismatch` or `ForeignStore`. +- `Resolved` is an answer, not a witness. Its accessors give the field view, the meaning of each field joined by predicate to a `PropertySpec` (including its `marking`), a mask chosen by meaning, and the method ordinals. +- `PropertySpec.marking` is one of `Public`, `Internal`, `Pii`, `Financial`, `Restricted`. A field with no spec is never selected by `mask_where`. +- `LaneBinding` holds addresses only. It never holds a value. + +Sources: + +- `crates/lance-graph-contract/src/class_view.rs:1057-1120` (`ClassView`) +- `crates/lance-graph-dt64/src/lib.rs:213-251` (`MethodBinding`) +- `crates/lance-graph-dt64/src/lib.rs:253-262` (`ClassSchema`) +- `crates/lance-graph-dt64/src/lib.rs:352-366` (`MethodRef`) +- `crates/lance-graph-dt64/src/lib.rs:640-770` (`Resolved`, `field_view`, `field_meaning`, `mask_where`, `action_ordinals`) +- `crates/lance-graph-dt64/src/lib.rs:772-795` (`BindStore`) +- `crates/lance-graph-dt64/src/lib.rs:1136-1160` (`method_ref`) +- `crates/lance-graph-dt64/src/lib.rs:1180-1215` (`admit`) +- `crates/lance-graph-contract/src/property.rs:769-783` (`Marking`) +- `crates/lance-graph-dt64/src/lane.rs:26-142` (`CoordSpace`, `LaneAddr`, `LaneSource`, `LaneRequest`, `LaneRefusal`) +- `crates/lance-graph-dt64/src/lane.rs:151-165` (`LaneBinding`) +- `crates/lance-graph-dt64/src/lane.rs:264-396` (`LaneBinding` methods) + +## 3.4 Bind then borrow + +`bind_lanes` is cold and runs once per binding. `borrow_all` re-checks the binding once and then hands out every lane. Each `alt` branch below is a named refusal in the code. + +```mermaid +sequenceDiagram + autonumber + participant C as Caller + participant B as BindStore + participant F as FrameStore + participant V as ClassView + participant S as LaneSource + + C->>B: bind_lanes(view, frames, source, request) + B->>F: same store as source.frames() + alt frames is not the source store + B-->>C: Frame(ForeignFrame) + end + B->>F: check(request.frame) + alt frame stale or resealed + B-->>C: Frame(SourceChanged, MapChanged, CoordinatesChanged, FrameResealed) + end + B->>B: routed_at(route.version, position) + alt position unrouted or version in the future + B-->>C: Bind(Unrouted or FutureVersion) + end + B->>B: route at current version equals bound class + alt re-routed since + B-->>C: RouteChanged + end + B->>V: resolve(class, rev) reads field basis + alt class unknown or revision or basis differs + B-->>C: Bind(UnknownClass, SchemaRevisionMismatch, PredicateBasisChanged) + end + loop each requested field + alt field past the basis + B-->>C: FieldNotInBasis + end + B->>S: lane_of(class, rev, field, basis, frame) + alt source declares no lane + B-->>C: Unbound + end + alt lane row order differs from selection + B-->>C: CoordinateSpace + end + alt lane serves another reader scope + B-->>C: Scope + end + end + B-->>C: LaneBinding with addresses only + + C->>B: binding.borrow_all(store, view, frames, source) + B->>B: check once: frame, route, basis digest, every lane again + alt basis witness differs from bind + B-->>C: StoreBasisMismatch + end + alt source maps a field to another lane + B-->>C: MappingChanged + end + B-->>C: BoundLanes iterator, ascending field order + loop each lane + C->>S: borrow(addr) + alt source no longer holds the lane + S-->>C: Err(Vanished) at that field + end + end +``` + +Reading notes: + +- Every refusal fires before any lane is borrowed. `borrow_all` runs the one check first, so N lanes cost N source lookups and not N squared. +- `LaneBinding::lane(field)` re-runs the whole check on each call and returns `NotBound` for a field outside the binding. +- `check` does not hash the basis again. It compares a `u128` witness and the revision and field count. A same-count reorder of the view after bind is caught only if the source maps lanes by predicate. The doc says to re-bind after any basis change. +- `Vanished` is yielded per lane and the iterator carries on. +- The returned lanes are as current as the check. No lock is held on the source. + +Sources: + +- `crates/lance-graph-dt64/src/lane.rs:104-141` (`LaneRefusal`) +- `crates/lance-graph-dt64/src/lane.rs:171-217` (`bind_lanes`) +- `crates/lance-graph-dt64/src/lane.rs:220-263` (`source_owns`, `current_route`, `admit_lane`) +- `crates/lance-graph-dt64/src/lane.rs:297-350` (`check`) +- `crates/lance-graph-dt64/src/lane.rs:352-418` (`lane`, `borrow_all`, `BoundLanes`) +- `crates/lance-graph-dt64/src/lib.rs:1073-1095` (`resolve`) +- `crates/lance-graph-dt64/src/lib.rs:1220-1245` (`check_basis`) +- `crates/lance-graph-dt64/src/dr.rs:414-475` (`FrameStore::seal`, `issued`, `check`) + +## 3.5 What this layer does not do + +Source: `crates/lance-graph-dt64/src/lib.rs:24-30` (module doc, "What is stored, and what is not"). + +- It does not decide who may read. `Scope` is an id the source stamps on a lane. See [04](04-read-path.md) for the authorization gap. +- It does not evaluate anything. Evaluation belongs to mask-risc. +- It does not persist anything. The module doc says the store keeps bindings and the route log only, with no persistence path. diff --git a/.mermaid/04-read-path.md b/.mermaid/04-read-path.md new file mode 100644 index 000000000..b9a9eeb56 --- /dev/null +++ b/.mermaid/04-read-path.md @@ -0,0 +1,182 @@ +# 04 — Read path + +Status: the read path from a sealed frame to a result mask is in code. The step that applies an authorization decision to that read is not built (gap G2). + +Legend: green = in code on main, yellow = lab crate only, red dashed = gap (not built). + +A governed read here means: the answer is computed over source-owned planes that were bound under a checked frame, and it comes back as a mask over the source's own row order. + +## 4.1 One read, end to end + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + SEAL["FrameStore seal gives a Frame"]:::shipped + BIND["bind_lanes: LaneBinding, addresses only"]:::shipped + BORROW["borrow_all: check once, then borrow every lane"]:::shipped + PLANES["Planes: resident masks and value lanes"]:::shipped + BUILD["Program: MaskOps then Terminal Keep"]:::shipped + EXEC["execute or execute_into"]:::shipped + MASK["result mask: scratch slot or Out Mask"]:::shipped + IDENT["row bit to identity through the id lane"]:::lab + + STANDIN["lab stand-in: NeU32 on the id lane"]:::lab + SCOPEPLANE["per-row scope plane from the decision"]:::gap + FIELDS["visible fields intersect marking mask"]:::gap + DECISION["authorize_scoped gives ScopedDecision"]:::shipped + + SEAL --> BIND --> BORROW --> PLANES --> EXEC + BUILD --> EXEC --> MASK --> IDENT + STANDIN --> BUILD + DECISION -.-> SCOPEPLANE + SCOPEPLANE -.-> PLANES + DECISION -.-> FIELDS + FIELDS -.-> BIND +``` + +What each solid step does: + +- **Seal.** A `FrameStore` issues the `Frame`. See [03](03-dr-dt64-binding.md). +- **Bind.** `bind_lanes` checks the store, frame, route, basis, row order and reader scope, and returns a `LaneBinding` that holds addresses only. +- **Borrow.** `borrow_all` runs the one check and yields every bound lane from the source. No lane is copied. +- **Planes.** The caller sorts borrowed lanes into `Planes`: mask planes and value lanes, all owned by the source. +- **Program.** A `Program` is a flat list of `MaskOp`s and one `Terminal`. `Terminal::Keep` leaves the final mask in place. +- **Execute.** `execute_into` runs the program one tile at a time. A `Keep` mask is written to the caller's `Out::Mask`, or read from its scratch slot when the scratch is one whole-population tile. +- **Identity.** In the lab, bit `r` of the result is physical row `r`, and the id lane gives the global identity at that row (`store_at`). + +What the dashed edges say: + +- A `ScopedDecision` exists (green). Nothing in the read path consumes it. +- The lab filters with a stand-in. `visible` is a `NeU32` predicate on the identity lane that denies exactly one fixed store id. It is a test stand-in, not an authorization. +- Two pieces are missing: a per-row scope plane built from the decision's `ScopeSpec`, and the intersection of `visible_fields()` with a marking mask to limit which fields get bound. + +Sources: + +- `crates/lance-graph-dt64/src/dr.rs:414-420` (`FrameStore::seal`) +- `crates/lance-graph-dt64/src/lane.rs:171-217` (`bind_lanes`) +- `crates/lance-graph-dt64/src/lane.rs:365-394` (`borrow_all`) +- `crates/lance-graph-mask-risc/src/ir.rs:71-81` (`Planes`) +- `crates/lance-graph-mask-risc/src/ir.rs:708-716` (`Program`) +- `crates/lance-graph-mask-risc/src/ir.rs:319-321` (`Terminal::Keep`) +- `crates/lance-graph-mask-risc/src/exec.rs:1318-1351` (`execute_into`, tiling, `Out::Mask`) +- `crates/lance-graph-mask-risc/src/value.rs:55-76` (`Out`) +- `crates/foundry-parity-lab/src/bind.rs:154-178` (lab `bind`: seal, then `bind_lanes`) +- `crates/foundry-parity-lab/src/arm_b.rs:139-188` (`ItemQuery::open`: `borrow_all`, planes sorted into masks and lanes) +- `crates/foundry-parity-lab/src/arm_b.rs:254-265` (`visible`, the stand-in) +- `crates/foundry-parity-lab/src/arm_b.rs:267-289` (`run`, `keep`) +- `crates/foundry-parity-lab/src/arm_b.rs:325-328` (`store_at`) +- `crates/foundry-parity-lab/src/fixture.rs:195` (`DENIED_STORE`) + +## 4.2 How a program gets built + +Two routes build a `Program`. Quack binds names and literals once into a numeric `Query` and lowers it. The lab writes `MaskOp`s by hand. Neither route evaluates anything. Evaluation is `execute`. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + + DRAFT["Draft: table, where, count or rows"]:::shipped + BINDER["Binder: table, live, field, code"]:::shipped + QUERY["Query: Filter and Agg, numeric only"]:::shipped + LOWER["lower or lower_fused"]:::shipped + PROG["Program"]:::shipped + HAND["lab arm: hand-built MaskOps"]:::lab + RUN["execute_into in mask-risc"]:::shipped + + DRAFT -->|bind| BINDER + BINDER --> QUERY --> LOWER --> PROG + HAND --> PROG + PROG --> RUN +``` + +Reading notes: + +- `Binder::live(table)` returns the plane every row of the table must pass. `Draft::bind` puts it first in the filter as `Filter::plane(...)`. It is the table's validity plane. +- After binding no text remains. A `Query` holds only fixed-width values. +- The Quack crate may build a `Program` and must never evaluate one. That rule is in its module doc. +- `TypedQuery` and `CodeWitness` let a binder vouch that a code space has not been renumbered between bind and use. + +Sources: + +- `crates/lance-graph-quack/src/bind.rs:1-30` (lifecycle: describe, resolve, canonicalize, execute numeric) +- `crates/lance-graph-quack/src/bind.rs:92-105` (`Binder`) +- `crates/lance-graph-quack/src/bind.rs:298-335` (`Draft::bind`, `live` plane first) +- `crates/lance-graph-quack/src/bind.rs:388-465` (`CodeWitness`, `TypedQuery`) +- `crates/lance-graph-quack/src/lib.rs:340-348` (`Filter`, `Filter::Plane`) +- `crates/lance-graph-quack/src/lib.rs:1297-1306` (`Query`) +- `crates/lance-graph-quack/src/lib.rs:1383-1399` (`lower`, `lower_fused`) +- `crates/lance-graph-quack/src/lib.rs:20-24` (the crate builds a `Program` and never evaluates one) +- `crates/foundry-parity-lab/src/arm_b.rs:267-276` (`run`: `Program::new` then `execute`) + +## 4.3 The G2 gap: a decision that is not applied + +The decision chain is complete up to `ScopedDecision`. The read path has a reader `Scope` that is only an id compared between a lane and a request. Nothing derives that scope from a decision. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + EST["ogar-auth establish gives VerifiedLogin"]:::shipped + USER["AuthenticatedUser"]:::shipped + OGR["ogar-rbac authorize_user"]:::shipped + KER["lance-graph-rbac authorize_scoped"]:::shipped + DEC["ScopedDecision: decision, scope, field_mask"]:::shipped + VIS["visible_fields"]:::shipped + MARK["Resolved mask_where over PropertySpec marking"]:::shipped + + LREQ["LaneRequest: scope id and fields mask"]:::shipped + ADMIT["admit_lane: lane scope id equals reader scope id"]:::shipped + + ROWLANE["per-row scope lane on the source"]:::gap + RPLANE["resident scope plane, candidate seam Binder live"]:::gap + DERIVE["scope id derived from the decision"]:::gap + NARROW["visible fields intersect marking mask limits bound fields"]:::gap + + EST --> USER --> OGR --> KER --> DEC + DEC --> VIS + LREQ --> ADMIT + DEC -.-> ROWLANE -.-> RPLANE + DEC -.-> DERIVE -.-> LREQ + VIS -.-> NARROW + MARK -.-> NARROW + NARROW -.-> LREQ +``` + +What is built: + +- `establish` is the only producer of a `VerifiedLogin`. `AuthenticatedUser` is a descriptive envelope. +- `OgarRbac::authorize_user` takes an `AuthenticatedUser` and calls the generic `authorize_scoped` kernel. `authorize_identity` is the membership variant. +- `ScopedDecision` carries the allow or deny verdict, an optional `ScopeSpec` (tenant, predicate key, path, deny) and a `WideFieldMask`. `visible_fields()` returns an empty mask unless access is allowed. +- `Resolved::mask_where` builds a mask from `PropertySpec.marking`. +- `LaneRequest` carries a reader `Scope` and a field mask. `admit_lane` refuses a lane whose scope id differs. + +What is not built: + +- No code in the dt64, quack, mask-risc, xlog, r2il or foundry-parity-lab crates names `authorize_scoped`, `ScopedDecision`, `lance_graph_rbac`, `ogar_rbac` or `ogar_auth`. +- No per-row scope lane exists on a source. A `ScopeSpec` therefore cannot become a plane. +- Where the decision is applied is not decided. `Binder::live` is a candidate seam because it is the plane every row must pass. +- No code derives the `Scope` id from a decision. No code intersects `visible_fields()` with a marking mask to restrict `bind_lanes`. + +Sources: + +- `OGAR/crates/ogar-auth/src/federation.rs:200-212` (`VerifiedLogin`, only producer is `establish`) +- `OGAR/crates/ogar-auth/src/federation.rs:250-266` (`establish`) +- `OGAR/crates/ogar-auth/src/user.rs:298-309` (`AuthenticatedUser`) +- `OGAR/crates/ogar-rbac/src/lib.rs:120-162` (`OgarRbac`, `authorize_user`) +- `OGAR/crates/ogar-rbac/src/lib.rs:322-335` (`authorize_identity`) +- `crates/lance-graph-rbac/src/authorize.rs:168-200` (`ScopedDecision`, `visible_fields`) +- `crates/lance-graph-rbac/src/authorize.rs:210-250` (`authorize_scoped`) +- `crates/lance-graph-rbac/src/authorize.rs:255-282` (`MembershipDecision`) +- `crates/lance-graph-contract/src/rbac.rs:113-132` (`ScopeSpec`) +- `crates/lance-graph-dt64/src/lib.rs:716-728` (`Resolved::mask_where`) +- `crates/lance-graph-contract/src/property.rs:769-783` (`Marking`) +- `crates/lance-graph-dt64/src/lane.rs:90-102` (`LaneRequest`) +- `crates/lance-graph-dt64/src/lane.rs:242-263` (`admit_lane`) +- `crates/lance-graph-dt64/src/dr.rs:518-523` (`Scope`: an opaque id) +- `crates/lance-graph-quack/src/bind.rs:92-97` (`Binder::live`) +- Absence claim: a text search of `crates/{lance-graph-dt64,lance-graph-quack,lance-graph-mask-risc,foundry-parity-lab,lance-graph-xlog,lance-graph-r2il}` (sources and Cargo.toml) for the five names above found no match. That is navigation over six named crates, not proof about the rest of the tree. diff --git a/.mermaid/05-methods.md b/.mermaid/05-methods.md new file mode 100644 index 000000000..9e00b5775 --- /dev/null +++ b/.mermaid/05-methods.md @@ -0,0 +1,228 @@ +# 05 — Methods: ON x WHEN x DO + +A method is a binding stored in DT64. It says which fields trigger it (ON), +which rows it applies to (WHEN), and what it writes (DO). +DT64 keeps the binding and executes nothing. +Other crates run the WHEN part and the DO part, and today they run them +separately. + +Legend: solid edge = in code on main. Dashed edge = gap, not built. +Green = shipped on main. Yellow = lab crate only. Blue = open PR. +Red dashed = gap. + +## 1. Method lifecycle + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + MB["MethodBinding
id, sig_rev, ON mask, WHEN Guard, reads, DO effects"]:::shipped + REG["BindStore::register
checks basis and unique MethodIds
a bound class never changes"]:::shipped + REF["BindStore::method_ref
issues MethodRef + Stamp"]:::shipped + ADM["BindStore::admit
checks revision, guard witness,
binding digest, issuer"]:::shipped + REFUSE["Refusal
UnknownMethod, SignatureRevisionMismatch,
GuardDropped, GuardAdded, GuardMismatch,
BindingMismatch, ForeignStore"]:::shipped + + MB --> REG --> REF --> ADM + ADM -- "fails" --> REFUSE + + ADM -- "WHEN" --> RFS["rel_from_set
SetExpr to Rel"]:::open + RFS --> RWP["relalg run_with_program
normalize, lower"]:::shipped + RWP --> PROG["one mask-risc Program
kept, cached per binding"]:::shipped + PROG --> EXEC["execute_into per question
Out::Mask(keep)"]:::shipped + LAB["G6 lab: PreparedMethod
prepare cold, run hot"]:::open + RFS -.-> LAB + PROG -.-> LAB + + ADM -- "DO" --> PREP["r2il binding::prepare
binds effects to opcodes, cold"]:::shipped + PREP --> NAT["binding_native::prepare_native
keeps compiled programs"]:::shipped + NAT --> RUN["run_native, hot
ExecPermit checked"]:::shipped + RUN --> RCPT["Receipt
plan_id, method, frame, output"]:::shipped + PREP -- "binding has WHEN" --> NOWHEN["GuardNotExecutable
WHEN refused here"]:::shipped + + ADM -- "bag count" --> XL["xlog bag_count_admitted
steps must be GroupCount, GroupCount, MulI64"]:::shipped + + PERMIT["ExecPermit issuer
not decided"]:::gap + ONE["one executor for WHEN and DO"]:::gap + PERMIT -.-> RUN + ONE -.-> EXEC + ONE -.-> RUN +``` + +What the diagram shows: + +- `register` is the only way a binding enters a store. + `method_ref` is the only way a `MethodRef` is issued. + Its `Stamp` has private fields, so a reference built elsewhere cannot carry + a valid stamp. +- `admit` returns the stored binding only after all checks pass. + The order is fixed: identity, signature revision, guard witness, binding + digest, issuer. +- The WHEN path goes through relalg and ends in one mask-risc program. + The code that holds and re-runs that program is the G6 lab + (`PreparedMethod`). + It lives on the open PR #1996 branch, not on main. + `rel_from_set` is also new in that PR. +- The DO path is r2il. + `prepare` runs once and is cold. + `run` and `run_native` are hot and check the frame, focus, scope, route and + an explicit `ExecPermit`. +- r2il refuses a binding that has a WHEN guard. + The lab runs WHEN and r2il runs DO. + Nothing yet runs both for one method. +- `ExecPermit` is documented as a lab permission. + Who issues it is not decided. + +Sources: + +- `crates/lance-graph-dt64/src/lib.rs:1-27` (what DT64 stores, nothing executes) +- `crates/lance-graph-dt64/src/lib.rs:170-196` (`Guard`, `witness`) +- `crates/lance-graph-dt64/src/lib.rs:215-231` (`MethodBinding`: id, sig_rev, on, when, reads, effects) +- `crates/lance-graph-dt64/src/lib.rs:353-374` (`MethodRef`, `Stamp`, private fields) +- `crates/lance-graph-dt64/src/lib.rs:473-575` (`Refusal` arms; 554, 555, 562, 564, 566, 571, 574) +- `crates/lance-graph-dt64/src/lib.rs:814-823` (`register`) +- `crates/lance-graph-dt64/src/lib.rs:1136-1157` (`method_ref`) +- `crates/lance-graph-dt64/src/lib.rs:1161-1167` (`check_issuer`) +- `crates/lance-graph-dt64/src/lib.rs:1180-1211` (`admit`, fixed check order) +- `crates/lance-graph-relalg/src/lower.rs:70-100` (`run_with_program`; `symbolic` answer returns no program) +- `crates/lance-graph-relalg/src/ir.rs:99-119` (`Measure`, `Query`) +- `crates/lance-graph-relalg/src/normalize.rs:82` (`to_set`, on main) +- `crates/lance-graph-relalg/src/normalize.rs:124-130` (`rel_from_set`, PR #1996 branch only) +- `crates/foundry-parity-lab/src/method.rs:200-292` (`PreparedMethod::prepare`, PR #1996) +- `crates/lance-graph-r2il/src/binding.rs:1-73` (module doc: prepare/run, what is checked when) +- `crates/lance-graph-r2il/src/binding.rs:125-131` (`ExecPermit`: "not decided") +- `crates/lance-graph-r2il/src/binding.rs:251-252`, `:336-338` (`GuardNotExecutable`) +- `crates/lance-graph-r2il/src/binding_native.rs:1-42` (native plan, hot run allocates nothing) +- `crates/lance-graph-xlog/src/lib.rs:363-405` (`BoundRefusal`, `bag_count_admitted`) + +## 2. WHEN: prepare once, run per question (lab) + +```mermaid +sequenceDiagram + autonumber + participant C as Caller + participant S as BindStore + participant L as DT64 lanes + participant R as relalg + participant M as mask-risc + + Note over C,M: G6 lab, crates/foundry-parity-lab (PR #1996, open) + C->>S: admit(MethodRef) + S-->>C: MethodBinding (or Refusal) + C->>L: bind_lanes(reads + identity lane), borrow_all + L-->>C: borrowed planes, no copy + C->>C: rename field positions to bound planes + opt reader scope supplied + C->>C: conjoin "id lane != denied store" + end + C->>R: run_with_program(Query{rel_from_set(guard), Keep}) + R->>M: lower and run once + R-->>C: Run + Program + Note over C: refuse unless exactly one program ran + loop each question (hot) + C->>M: execute_into(Program, planes, Out::Mask) + M-->>C: keep mask + C->>C: read identity lane at set bits + end +``` + +Notes: + +- The guard is a `SetExpr` over field positions. + Mask fields are `Sym(field)`. + Value fields are comparisons on a lane. +- The lab keeps the one program relalg returns. + It refuses with `MethodRefusal::Folds` if the guard lowered into more than + one fold, because the last program alone would not be the whole answer. +- The reader scope in the lab is a stand-in written into the program. + It is not the RBAC decision. + See `06-authorization.md`. +- Arm C of the lab refuses a cost question with `CostRefusal::NotBound` when + a demanded cost lane is not bound. + That arm is merged (#1995) and is separate from the method path. + +Sources: + +- `crates/foundry-parity-lab/src/method.rs:47-61` (`can_cook_guard`) (PR #1996) +- `crates/foundry-parity-lab/src/method.rs:77-94` (`can_cook_method`: `when: Some(Guard::new(...))`, `reads`) (PR #1996) +- `crates/foundry-parity-lab/src/method.rs:105-135` (`MethodRefusal`, including `Folds`) (PR #1996) +- `crates/foundry-parity-lab/src/method.rs:211-292` (`prepare`: admit `:217`, `borrow_all` `:229`, rename `:251`, scope `:252-257`, `run_with_program` `:264-272`, one-fold check `:276-280`) (PR #1996) +- `crates/foundry-parity-lab/src/method.rs:295-320` (`run`: `execute_into` `:302-309`, identity read) (PR #1996) +- `crates/lance-graph-relalg/src/lower.rs:70-100` (`run_with_program`) +- `crates/lance-graph-mask-risc/src/exec.rs:1343` (`execute_into`) +- `crates/foundry-parity-lab/src/arm_c.rs:66-92` (`CostRefusal::NotBound`, on main) + +## 3. DO: prepare cold, run hot (r2il) + +```mermaid +sequenceDiagram + autonumber + participant C as Caller + participant P as r2il binding + participant S as BindStore + participant G as Guards + FrameStore + participant F as FoldDialect (mask-risc) + + C->>P: prepare(store, view, lens, Resolution, frames, now) + P->>G: check(frames, now) + P->>S: check_route, resolve(view, class, schema_rev) + P->>S: admit(MethodRef) + Note over P: refuse WHEN guard (GuardNotExecutable)
refuse three-valued truth + Note over P: per effect: lens, label alias,
EXECUTABLE opcode, arity + P-->>C: Prepared (plan_id, steps) + + opt native form + C->>P: prepare_native(plan, ..., planes) + P->>F: run steps once, keep compiled programs + P-->>C: NativePlan (refuses fold results that feed folds) + end + + C->>P: run / run_native(plan, store, frames, now, ExecPermit, ...) + P->>S: check_issuer(plan.method) + P->>G: check(frames, now), check_route + Note over P: permit must list this exact MethodRef,
else NotPermitted + P->>F: execute steps (native: compiled programs) + P-->>C: Receipt (plan_id, method, frame, steps_executed, programs_run, output) +``` + +Notes: + +- `run` never reads the `ClassView` or the vocabulary again. + It rechecks the issuer, guards, route and permit, then runs the bound + opcodes. +- `admit` succeeding is not permission to run. + The permit must list the exact `MethodRef`. +- The `Receipt` is bounded by the number of steps. + It is not a row trace and not the logical provenance of the answer. +- `run_native` also refuses with `PlanesChanged` when the planes have another + shape than the ones the plan was prepared over. +- xlog has its own admitted path. + `bag_count_admitted` accepts a `MethodBinding` only when it is a bag + (`Population::Bag`) and its steps are `GroupCount`, `GroupCount`, `MulI64`. + The intake re-checks its `MapTable` frame on every query. + +Sources: + +- `crates/lance-graph-r2il/src/binding.rs:12-61` (lifecycle, what is checked when) +- `crates/lance-graph-r2il/src/binding.rs:125-131` (`ExecPermit`) +- `crates/lance-graph-r2il/src/binding.rs:165-176` (`Prepared`, `plan_id`) +- `crates/lance-graph-r2il/src/binding.rs:223-231` (`Receipt`) +- `crates/lance-graph-r2il/src/binding.rs:236-317` (`ExecRefusal`) +- `crates/lance-graph-r2il/src/binding.rs:321-355` (`prepare`) +- `crates/lance-graph-r2il/src/binding.rs:377-404` (`run`) +- `crates/lance-graph-r2il/src/binding_native.rs:1-42` (native plan) +- `crates/lance-graph-r2il/src/binding_native.rs:203-216` (`prepare_native`) +- `crates/lance-graph-r2il/src/binding_native.rs:351-417` (`run_native`: permit, `PlanesChanged`) +- `crates/lance-graph-xlog/src/lib.rs:363-405` (`bag_count_admitted`) + +## Open items on this page + +- One executor that runs WHEN and DO for the same method: not built. + The plan lists it as the missing contract for the computed-function row. +- The `ExecPermit` issuer: undecided. + A natural candidate is the `Operation::Act` decision in `lance-graph-rbac`. + That is a candidate only. + See `06-authorization.md`. +- The lab (`PreparedMethod`, `rel_from_set`) is on PR #1996 and is not merged. diff --git a/.mermaid/06-authorization.md b/.mermaid/06-authorization.md new file mode 100644 index 000000000..9aa75fc83 --- /dev/null +++ b/.mermaid/06-authorization.md @@ -0,0 +1,263 @@ +# 06 — Authorization + +A login becomes an authenticated user. +The user and an operation go to an RBAC kernel. +The kernel returns a decision: allow or refuse, a row scope and a set of +visible fields. +That decision exists today. +Nothing yet applies it to a DT64 binding. + +Legend: solid edge = in code on main. Dashed edge = gap, not built. +Green = shipped. Red dashed = gap. +The OGAR crates are in the OGAR repository, not in lance-graph. + +## 1. From login to decision + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + IDP["Identity assertion
federated, local, or kiosk"]:::shipped + EST["ogar-auth establish
verify, trusted issuer,
resolve (issuer, subject)"]:::shipped + VL["VerifiedLogin
only producer is establish"]:::shipped + AU["AuthenticatedUser
user + AuthContext
acting_party()"]:::shipped + OR["ogar-rbac OgarRbac
authorize_user"]:::shipped + ID["ogar-rbac authorize_identity
binding + IdentityActors"]:::shipped + AS["lance-graph-rbac
authorize_scoped"]:::shipped + AM["lance-graph-rbac
authorize_memberships"]:::shipped + SD["ScopedDecision
decision, scope, field_mask"]:::shipped + MD["MembershipDecision
decision, scope set, field_mask"]:::shipped + + IDP --> EST --> VL --> AU + AU --> OR --> AS --> SD + AU --> ID --> AM --> MD +``` + +What the diagram shows: + +- `establish` runs in a fixed order. + The verifier runs first. + The asserted issuer must then be trusted. + The `(issuer, subject)` must then resolve to a user. + There is no fallback to email. +- `VerifiedLogin` has private fields and no public constructor. + `AuthenticatedUser` has public fields. + It is descriptive: building one proves nothing about how the actor logged in. + Refusing unauthenticated actors is a deployment policy, not a grant rule. +- `OgarRbac::authorize_user` takes an `AuthenticatedUser`, not a bare actor + string. + It calls the generic kernel `authorize_scoped` and adds only the identity. +- `authorize_identity` is the second route. + It binds every membership of the user to the user's tenant and calls + `authorize_memberships`. + It returns a `MembershipDecision`, not a `ScopedDecision`. +- A delegated login authorizes as the user whose authority the token carries. + The acting party is recorded and holds no role. + +Sources: + +- `OGAR/crates/ogar-auth/src/federation.rs:206-216` (`VerifiedLogin`, private fields) +- `OGAR/crates/ogar-auth/src/federation.rs:244-283` (`establish`, fixed order, no fallback) +- `OGAR/crates/ogar-auth/src/user.rs:290-335` (`AuthenticatedUser`: descriptive, `actor_context`, `acting_party`) +- `OGAR/crates/ogar-rbac/src/lib.rs:1-36` (crate doc: the chain, where the kernel lives) +- `OGAR/crates/ogar-rbac/src/lib.rs:70-71` (imports `authorize_scoped`, `authorize_memberships`) +- `OGAR/crates/ogar-rbac/src/lib.rs:122-162` (`OgarRbac`, `authorize_user`, kiosk note) +- `OGAR/crates/ogar-rbac/src/lib.rs:225-345` (`IdentityActors`, delegation, `authorize_identity`) +- `crates/lance-graph-rbac/src/authorize.rs:210-253` (`authorize_scoped`) +- `crates/lance-graph-rbac/src/authorize.rs:299-345` (`authorize_memberships`) + +## 2. Inside the kernel + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + + OP["Operation
Read{depth}, Write{predicate}, Act{action}"]:::shipped + S1["Stage 1: authorize
role grant permits the op"]:::shipped + NOT["Deny or Escalate
scope None, field_mask FULL"]:::shipped + S2["Stage 2: fold the granting roles only"]:::shipped + SC["scope: ScopeSpec intersect
None is global and does not narrow"]:::shipped + FM["field_mask: union of role masks
WideFieldMask"]:::shipped + VF["visible_fields()
mask if Allow, else EMPTY"]:::shipped + MS["authorize_memberships
scope: union of ScopeSet
any unscoped grant means unrestricted"]:::shipped + + OP --> S1 + S1 -- "not Allow" --> NOT + S1 -- "Allow" --> S2 + S2 --> SC + S2 --> FM + FM --> VF + S1 -. "memberships route" .-> MS +``` + +Notes: + +- On a refused decision `field_mask` holds the open default (`FULL`). + Use `visible_fields()` for any output stage. + It returns `EMPTY` unless the decision is `Allow`. +- `authorize_scoped` intersects the scopes of the granting roles. + One more role can only narrow what the actor sees. + `authorize_memberships` is the permissive counterpart. + Each granting membership adds its own scope, and the scopes are unioned. +- `Operation::Write` names a predicate. + `Operation::Act` names an action. + `Operation::Read` carries a depth. + The grant gate is a verb bit per class (`OpMask`: READ, WRITE, CREATE, + DELETE, ACT). + A finer check on depth, predicate or action name layers above it. + +Sources: + +- `crates/lance-graph-contract/src/rbac.rs:340-364` (`Operation`: Read, Write, Act) +- `crates/lance-graph-contract/src/rbac.rs:453-466` (`OpMask`, ACT bit) +- `crates/lance-graph-contract/src/rbac.rs:548-550` (`grants_permit`) +- `crates/lance-graph-rbac/src/authorize.rs:164-208` (`ScopedDecision`, `visible_fields`) +- `crates/lance-graph-rbac/src/authorize.rs:210-253` (stage 1 short-circuit, stage 2 fold) +- `crates/lance-graph-rbac/src/authorize.rs:255-345` (`MembershipDecision`, `authorize_memberships`) +- `crates/lance-graph-rbac/src/access.rs:5-13` (`AccessDecision`: Allow, Deny, Escalate) + +## 3. The shape of a decision + +```mermaid +classDiagram + class ScopedDecision { + AccessDecision decision + Option~ScopeSpec~ scope + WideFieldMask field_mask + visible_fields() WideFieldMask + } + class MembershipDecision { + AccessDecision decision + Option~ScopeSet~ scope + WideFieldMask field_mask + visible_fields() WideFieldMask + } + class ScopeSpec { + Option~u64~ tenant + u32 predicate_key + ScopePath path + bool deny + admits(at ScopePath) bool + intersect(other) ScopeSpec + } + class ScopeSet { + members ScopeSpec list + admits(at ScopePath) bool + } + class PropertySpec { + predicate + kind + Marking marking + } + class Marking { + Public + Internal + Pii + Financial + Restricted + } + ScopedDecision --> ScopeSpec + MembershipDecision --> ScopeSet + ScopeSet --> ScopeSpec + PropertySpec --> Marking +``` + +Notes: + +- `ScopeSpec::admits` is address containment. + It is true when the scope is not `deny` and the row's path lies at or below + the scope's path. + The tenant is matched by the store that owns the rows. +- `intersect` never widens. + `deny` absorbs. + Two paths on different branches give `DENY`. + Two different tenants give `DENY`. + A `None` tenant (global) takes the other side's tenant. +- `predicate_key` is reserved. + It is OR-combined and not interpreted. +- `Marking` is a data classification on each `PropertySpec`. + The default is `Internal`. + DT64's `Resolved::mask_where` builds a field mask from meaning, for example + every field with marking at or above `Pii`. + That mask is a way to select fields. + It is not an authorization decision. + +Sources: + +- `crates/lance-graph-contract/src/rbac.rs:114-132` (`ScopeSpec` fields) +- `crates/lance-graph-contract/src/rbac.rs:144-151` (`admits`) +- `crates/lance-graph-contract/src/rbac.rs:153-191` (`intersect`) +- `crates/lance-graph-contract/src/rbac.rs:236-267` (`ScopeSet`, `admits`) +- `crates/lance-graph-contract/src/rbac.rs:302-308` (`Membership`) +- `crates/lance-graph-rbac/src/authorize.rs:164-208`, `:255-278` (decision structs) +- `crates/lance-graph-contract/src/property.rs:41-58` (`PropertySpec.marking`) +- `crates/lance-graph-contract/src/property.rs:768-783` (`Marking`) +- `crates/lance-graph-dt64/src/lib.rs:714-723` (`mask_where`) +- `crates/lance-graph-contract/src/selection.rs:308-360` (`walk_rails_authorized`: applies `visible_fields()`, decides nothing) + +## 4. What is not built + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + SD["ScopedDecision
scope + visible_fields()"]:::shipped + MW["Resolved::mask_where
marking mask"]:::shipped + BL["BindStore::bind_lanes
borrow_all"]:::shipped + QB["Quack Binder::live
validity plane seam"]:::shipped + RAIL["walk_rails_authorized
applies visible_fields()"]:::shipped + PERM["ExecPermit
lab permission"]:::shipped + ACT["Operation::Act decision"]:::shipped + AUDIT["auth_classid, acting_party
identity for the audit"]:::shipped + + G2A["Row scope as a resident plane
per-row scope lane"]:::gap + G2B["visible_fields() limits
the bound fields"]:::gap + ISS["ExecPermit issuer
undecided"]:::gap + JOIN["One record per answer
joins identity, receipt, witness"]:::gap + + SD -.-> G2A + QB -.-> G2A + G2A -.-> BL + SD -.-> G2B + MW -.-> G2B + G2B -.-> BL + ACT -. "candidate only" .-> ISS + ISS -.-> PERM + AUDIT -.-> JOIN + SD --> RAIL +``` + +What is missing: + +- **Applying the decision to a DT64 binding (plan G2).** + The decision exists. + No DT64, r2il, xlog, Quack or mask-risc code uses `lance-graph-rbac`. + A per-row scope lane on the source and a rule that limits bound fields to + `visible_fields()` are not built. + The lab uses a stand-in: one reader denied one store id, written into the + program. + It is not IAM. + Quack's `Binder::live` is named in the plan as the existing seam where a + scope plane could enter. +- **The `ExecPermit` issuer.** + The r2il source says who issues the permit is not decided. + The `Operation::Act` decision is a natural candidate. + That is a candidate only. +- **Provenance.** + `ResolvedIdentity::auth_classid` is documented as the identity for the audit. + Receipts and commit witnesses exist elsewhere. + Nothing joins them into one record per answer. + +Sources: + +- `crates/lance-graph-rbac/Cargo.toml:1-9` (depends on `lance-graph-contract` only) +- `crates/lance-graph-r2il/src/binding.rs:125-131` (`ExecPermit`: "who issues it is not decided") +- `crates/lance-graph-quack/src/bind.rs:91-97` (`Binder::live`) +- `crates/lance-graph-dt64/src/lane.rs:171`, `:379` (`bind_lanes`, `borrow_all`) +- `crates/lance-graph-contract/src/selection.rs:308-360` (`walk_rails_authorized`) +- `crates/lance-graph-rbac/src/auth.rs:376-381` (`auth_classid`) +- `crates/foundry-parity-lab/src/method.rs:14-16`, `:252-257` (reader scope stand-in) (PR #1996) +- `.claude/plans/foundry-parity-v1.md` §0 and §7 G2 on branch `ccr-ca0562e7-hudvlz` (the gap statement; "candidate" wording) +- A search of the DT64, r2il, xlog, Quack and mask-risc crates for `rbac` and `ScopedDecision` finds no use. This is a search, so it shows candidates only; the Cargo.toml dependency lists agree. diff --git a/.mermaid/07-write-path.md b/.mermaid/07-write-path.md new file mode 100644 index 000000000..5c543181d --- /dev/null +++ b/.mermaid/07-write-path.md @@ -0,0 +1,273 @@ +# 07 — Write path + +The pieces of a write exist. +A thought ends in one of four outcomes. +A cycle of thoughts commits as one version. +A receipt can replay a thought after its slot is gone. +A cube can check a batch of edges before it changes. +A durable map table already goes through the cycle writer. +These pieces are not composed into one write-back that is authorized, +preconditioned, committed and witnessed. +That composition is gap G4 in the parity plan. + +Legend: solid edge = in code on main. Dashed edge = gap, not built. +Green = shipped on main. Yellow = lab crate only. Red dashed = gap. +`thought-git-probe` is a lab crate with its own workspace root. +Its `VersionStore` stands in for Lance and has no Lance storage. + +## 1. The Rubicon: how a thought ends + +```mermaid +stateDiagram-v2 + [*] --> Planning + Planning --> CognitiveWork: proceed + Planning --> Prune: Veto (before any work) + CognitiveWork --> Evaluation + Evaluation --> Commit: result lands + Evaluation --> Plan: back to in-memory work, nothing lands + Evaluation --> Prune: intention dropped after evaluation + Commit --> Landed: next_version writes the cell + Commit --> Refused: given cell, different value (Contradiction) + Commit --> Held: same cell, different values in one version (Divergence) + Plan --> [*] + Prune --> [*] + Landed --> [*] + Refused --> [*] + Held --> [*] +``` + +Rules from the oracle (lab): + +1. Only a thought whose outcome is `Commit` lands. + `Plan`, `Prune` and `Veto` leave the version unchanged. +2. A cell with a given value (layer 0) is never overwritten. + The same value is agreement and is not stored. + A different value is refused and recorded as a `Contradiction`. +3. A cell without a given takes the landed value. + If thoughts in one version offer different values, nothing is stored and a + `Divergence` is recorded. +4. A cell nobody wrote reads as absent, never as zero. + +Sources: + +- `crates/thought-git-probe/src/fixtures.rs:30-41` (`Outcome`: Commit, Plan, Prune, Veto) +- `crates/thought-git-probe/src/oracle.rs:1-15` (the four rules) +- `crates/thought-git-probe/src/oracle.rs:50-122` (`next_version`, `Contradiction`, `Divergence`) +- `crates/thought-git-probe/src/slots.rs:1-26` (slot 0 is the givens, 63 live slots, `branch`, `free`) + +## 2. Landing a cycle and keeping a receipt + +```mermaid +sequenceDiagram + autonumber + participant W as Write path (lab) + participant V as VersionStore + participant M as MetadataTable + participant R as replay + + Note over W,R: lab only, crates/thought-git-probe/src/receipt.rs + W->>V: commit_cycle(at, prior, cycle, batch) + alt a thought appears twice + V-->>W: CommitError::DuplicateThought + else at holds another version or witness + V-->>W: CommitError::AlreadySealed + else + V-->>W: CommitWitness{data, cycle, cohort, batch fold} + end + Note over V: data version sealed together with its witness + + W->>M: stage(pool, store, key, receipt) + Note over M: slot still holds the thought
data version exists (data first)
witness lists this thought, output, outcome + W->>M: seal(key) + Note over M: a sealed row is immutable + + R->>M: replay_landed(store, key, config) + M->>M: landed(): seal, witness, contribution checks + M->>R: replay(receipt, store, config) + Note over R: refuses on MissingSeed, ConfigChanged,
UnknownBase, SourceChanged, OutputMismatch +``` + +What this gives and what it does not: + +- `commit_cycle` seals one data version for a whole cycle. + The `CommitWitness` records the cycle and every thought in the cohort with + its output fold and outcome. + Every thought is in the cohort, whatever its outcome. +- A receipt records what is needed to recompute a thought's writes: the sealed + base it read, the program, the seed, the configuration, a fold of the values + read and a fold of the writes. + Replay recomputes both folds and refuses on any difference. + It never reads the old slot. +- The witness is a fold, not a cryptographic proof. + Data and metadata are two writes. + A crash between them leaves a committed thought without a sealed row. + `MetadataTable::incomplete` reports it. + No atomicity across the two is claimed. +- `BaseRef` stands in for a durable reference that has no contract type yet. + `VersionStore::commit_cycle` stands in for the coalesced per-cycle commit + of the real cycle writer. + +Sources: + +- `crates/thought-git-probe/src/receipt.rs:1-45` (module doc, what is a stand-in) +- `crates/thought-git-probe/src/receipt.rs:54-113` (`BaseRef`, `VersionStore`, `Contribution`, `CommitWitness`, `CommitError`) +- `crates/thought-git-probe/src/receipt.rs:153-185` (`commit_cycle`) +- `crates/thought-git-probe/src/receipt.rs:368-407` (`Receipt`, `ReceiptIdentity`) +- `crates/thought-git-probe/src/receipt.rs:426-476` (`Refusal` arms) +- `crates/thought-git-probe/src/receipt.rs:564-600` (`replay`) +- `crates/thought-git-probe/src/receipt.rs:618-692` (`MetaKey`, `MetaRow`, `contribution_check`) +- `crates/thought-git-probe/src/receipt.rs:697-847` (`MetadataTable`: `stage`, `seal`, `landed`, `incomplete`, `replay_landed`) + +## 3. A batch of edges: check, preview, apply (xlog) + +```mermaid +sequenceDiagram + autonumber + participant C as Caller + participant U as Cube (xlog) + participant T as MapTable (DR) + + C->>U: stage(table, grown map?, ops) + Note over U: checks only. Changes nothing.
No version issued. No frame sealed. + U-->>C: StagedEdges (or CubeRefusal) + + opt preview + C->>U: stage_closure(rel, old, staged) + U-->>C: closure as it will be, nothing changed + end + + opt the map grew + C->>T: advance table with exactly the staged map + T-->>C: new Frame (sealed) + end + + C->>U: commit(table, frame, staged) + Note over U: cannot refuse. Logs the ops in order,
adopts size, drops affected cached folds. +``` + +Notes: + +- `stage` takes `&self`. + It checks that a grown map extends the tile's current map. + It checks that every id is placed under the resulting layout. + A map that renumbers is refused with `Renumbered`. +- `commit` takes `&mut self`. + It cannot refuse because every check ran in `stage`. + The frame must be the one sealed after the table advanced with exactly the + staged map. + A batch staged by another cube is a caller bug and is not applied. +- The log is the cube's in-memory delta log. + It is not a Lance version. +- `bag_count_admitted` is a separate xlog entry that runs only from a method + the bind store admitted. + See `05-methods.md`. + +Sources: + +- `crates/lance-graph-xlog/src/cube.rs:60-92` (`EdgeOp`, `StagedEdges`) +- `crates/lance-graph-xlog/src/cube.rs:838-907` (`stage`) +- `crates/lance-graph-xlog/src/cube.rs:909-933` (`stage_closure`) +- `crates/lance-graph-xlog/src/cube.rs:932-975` (`commit`) +- `crates/lance-graph-xlog/src/cube.rs:979-990` (`rebase`, extension check) +- `crates/lance-graph-xlog/src/lib.rs:28-39` (frame re-check) + +## 4. The pieces, and the composition that is missing + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + AUTH["Authorization
Operation::Write / Act decision"]:::shipped + DO["r2il DO effects
prepare cold, run hot, Receipt"]:::shipped + STG["xlog Cube::stage
checks, no version"]:::shipped + CLO["xlog Cube::stage_closure
preview"]:::shipped + CMT["xlog Cube::commit
cannot refuse, frame-bound"]:::shipped + TG["thought-git: VersionStore::commit_cycle
CommitWitness"]:::lab + MT["MetadataTable stage, seal
receipt replay"]:::lab + PC["planner persist_cycle
WalSink::commit_cycle"]:::shipped + OUT["CommitOutcome
NoChange, Committed, Reconciled"]:::shipped + ERR["CommitError
Fenced, HashConflict, Io,
InvalidArtifact, Ambiguous"]:::shipped + CW["LanceCycleWriter
one Lance version per cycle"]:::shipped + ML["dt64-lance MapLog
durable map table"]:::shipped + + G4["G4: one authorized, preconditioned,
committed, witnessed action"]:::gap + + PC --> OUT + PC --> ERR + PC --> CW + ML --> CW + TG --> MT + STG --> CLO + STG --> CMT + + AUTH -.-> G4 + DO -.-> G4 + STG -.-> G4 + CMT -.-> G4 + TG -.-> G4 + CW -.-> G4 +``` + +What is built: + +- **Durable map table (PR #1993, merged).** + `MapLog` makes DR's versioned global-to-local table durable through the + cycle writer. + An advance is one cycle. + The route version is claimed when the advance is built. + It is applied to the table only after the cycle's seal ticket resolves + `Ok`. + A failed seal applies nothing, and its version is a gap in the table and in + the store. + Restore replays every sealed cycle at its own route version. +- **The cycle writer.** + `WalSink::commit_cycle` takes an already ordered and hashed batch and + publishes one new dataset version, all or nothing. + It looks the batch up by `(cycle, batch_hash)` first, so a retry after a + lost acknowledgement returns `Reconciled` and does not append twice. + Zero artifact casts return `NoChange` and never call the sink. +- **A commit that cannot say "nothing landed" after publication.** + There is no such result state. + The case reconciles or surfaces as `Ambiguous`. + +What is not built (G4): + +- Nothing joins an authorization decision, a DO effect, a precondition check, + a commit and a witness for one action. +- `Cube::commit` changes the cube's in-memory log. + It does not write a Lance version. +- The probe's `CommitWitness` is not wired into the real cycle writer. + The probe's own doc names that wiring as an admission decision still to be + made. +- The authorization decision is not applied on this path. + See `06-authorization.md`. +- The plans for the full thought-git API and lifecycle are a proposal and a + contract (docs only). + No production land, rebase or persistence pipeline for thought-git exists. + +Sources: + +- `crates/lance-graph-planner/src/persist_sink.rs:200-300` (`CommitOutcome`, `CommitError`, `LandedSlot`, `FrameMeta`) +- `crates/lance-graph-planner/src/persist_sink.rs:520-640` (`WalSink::commit_cycle`, reconciliation first, `persist_cycle` artifact gate, `NoChange`) +- `crates/lance-graph/src/graph/cycle_sink.rs:170`, `:1047` (`LanceCycleWriter`, `impl WalSink`) +- `crates/lance-graph-dt64-lance/src/lib.rs:1-45` (design: cycle writer, one advance is one cycle, restore) +- `crates/lance-graph-dt64-lance/src/lib.rs:270-325` (`advance`, `settle`) +- `crates/lance-graph-dt64-lance/Cargo.toml:1-25` (own workspace root; excluded from the lance-graph workspace) +- `crates/lance-graph-xlog/src/cube.rs:838-975` (`stage`, `stage_closure`, `commit`) +- `crates/thought-git-probe/src/lib.rs:1-14` (no production land, rebase or persistence pipeline) +- `crates/thought-git-probe/src/receipt.rs:39-45` (stand-ins; wiring is D-TGB-7's admission decision) +- `crates/lance-graph-r2il/src/binding.rs:223-231` (`Receipt`) +- `.claude/plans/foundry-parity-v1.md` §0 on branch `ccr-ca0562e7-hudvlz` (the G4 row: pieces exist, not composed) + +## Open items on this page + +- G4: the composition above. + It needs the authorization decision, the DO effects, a precondition check + and the cycle writer joined in one path. +- Scenario overlay (G3): branches exist (`SlotPool::branch`, `stage_closure`). + An overlay frame that a DT64 binding can read is not built. +- Provenance: receipts, commit witnesses and the authentication identity each + exist. + One record per answer that joins them is not built. diff --git a/.mermaid/08-scenarios.md b/.mermaid/08-scenarios.md new file mode 100644 index 000000000..08a9e6b3c --- /dev/null +++ b/.mermaid/08-scenarios.md @@ -0,0 +1,156 @@ +# 08 — Scenarios: what-if branches + +This page shows what exists for "what if" work, and the one piece that does not. +Branches and previews exist in two lab crates. A DT64 binding cannot read a branch yet. +That join is gap G3 and is drawn dashed. + +Status classes: green = in code on main. Yellow = lab crate only. Blue = open PR. Red dashed = gap. + +## 1. Branch, land, squash (thought-git-probe) + +`thought-git-probe` is a lab probe. It is excluded from the workspace and has its own workspace root. +`VersionStore` is a probe stand-in for Lance, not Lance itself. +The probe has no scenario, overlay or sandbox type. The nearest pieces are the slot pool and the oracle. + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + main["Main: givens (slot 0)"]:::lab + pool["SlotPool: 63 live slots (1..=63)"]:::lab + branch["SlotPool::branch(thought, from)"]:::lab + press["Pressure: Squashed or Evicted when full"]:::lab + stale["Stale handle refused after free"]:::lab + oracle["next_version(givens, prior, landing)"]:::lab + contra["Contradiction: write to a given differs"]:::lab + diverge["Divergence: two thoughts offer different values"]:::lab + gate["squash_gate(lanes)"]:::lab + prot["protected_squash(lanes, readers)"]:::lab + unret["Refusal: Unretainable(reader)"]:::lab + commit["VersionStore::commit_cycle"]:::lab + wit["CommitWitness: data, cycle, cohort, batch"]:::lab + + main --> pool + pool --> branch + branch --> press + branch -. "old handle" .-> stale + branch --> oracle + oracle --> contra + oracle --> diverge + oracle --> gate + gate --> prot + prot --> unret + oracle --> commit + commit --> wit +``` + +What the diagram says: + +- `branch` takes a free slot. It starts from main or from a live thought. +- When all 63 slots are taken, a candidate is squashed into a neighbour or evicted. +- `next_version` computes the next version from scratch. It records a `Contradiction` when a thought writes a different value to a given. It records a `Divergence` when two landing thoughts disagree on an absent cell. +- `squash_gate` refuses on a contradiction or a divergence. `protected_squash` also refuses when a protected reader could not be answered from what the squash keeps. +- `commit_cycle` seals one version per cycle and records a `CommitWitness`. + +Sources: + +- `crates/thought-git-probe/Cargo.toml:9-13` (own workspace root, one dependency on the contract crate) +- `crates/thought-git-probe/src/slots.rs:1-22` (slot 0 is the givens, 63 live slots, squash and evict rules) +- `crates/thought-git-probe/src/slots.rs:28` (`LIVE_SLOTS = 63`) +- `crates/thought-git-probe/src/slots.rs:49-64` (`Pressure::Squashed`, `Pressure::Evicted`) +- `crates/thought-git-probe/src/slots.rs:66-67` (`Stale`) +- `crates/thought-git-probe/src/slots.rs:104-138` (`SlotPool::branch`) +- `crates/thought-git-probe/src/oracle.rs:26-44` (`Contradiction`, `Divergence`) +- `crates/thought-git-probe/src/oracle.rs:66-107` (`next_version`) +- `crates/thought-git-probe/src/squash.rs:99-129` (`squash_gate`, refusals) +- `crates/thought-git-probe/src/squash.rs:182-196` (`Reader`, `Witness`) +- `crates/thought-git-probe/src/squash.rs:289-345` (`protected_squash`) +- `crates/thought-git-probe/src/receipt.rs:40-45` (`VersionStore` stands in for Lance) +- `crates/thought-git-probe/src/receipt.rs:153-185` (`commit_cycle`) + +## 2. Preview without a commit (xlog) + +`lance-graph-xlog` has a two-phase edge write. `stage` only checks. `stage_closure` shows the derived +result as it would be after the commit, and changes nothing. `commit` cannot refuse. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + ops["Edge ops: Add or Retire"]:::shipped + stage["Cube::stage (&self): checks, issues no version"]:::shipped + staged["StagedEdges"]:::shipped + prev["Cube::stage_closure: closure after commit, changes nothing"]:::shipped + com["Cube::commit: cannot refuse, needs the sealed frame"]:::shipped + frame["Frame sealed after MapTable advance"]:::shipped + g3["Overlay frame a DT64 binding can read"]:::gap + + ops --> stage + stage --> staged + staged --> prev + staged --> com + frame --> com + prev -. "G3: not built" .-> g3 +``` + +Sources: + +- `crates/lance-graph-xlog/src/cube.rs:845-907` (`Cube::stage`, doc and body) +- `crates/lance-graph-xlog/src/cube.rs:905-930` (`Cube::stage_closure`) +- `crates/lance-graph-xlog/src/cube.rs:932-945` (`Cube::commit`) +- `.claude/plans/foundry-parity-v1.md (PR #1996):292` (G3 register row) + +## 3. Lance versions as history, and the G3 gap + +Lance itself keeps versions. In the probe, `VersionStore` and `BaseRef` stand in for them. +`BaseRef` stands in for a durable reference that has no contract type yet. +Nothing in `lance-graph-dt64`, `lance-graph-xlog` or the probe source joins a branch to a DT64 +`FrameStore`. A search for "overlay" finds only a test name in the probe +(`tests/signals.rs:30`, `overlay_regressions`). The search is navigation, not proof of absence. + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + lance["Lance dataset versions (external)"] + vs["VersionStore + BaseRef (probe stand-in)"]:::lab + br["Thought branch: SlotPool::branch"]:::lab + prev["xlog stage_closure preview"]:::shipped + fs["DT64 FrameStore: seal, advance, check"]:::shipped + lb["LaneBinding + borrow_all"]:::shipped + ov["Overlay frame: branch joined to FrameStore"]:::gap + q["Store 12 closed on a branch: answer equals oracle, main frame unchanged"]:::gap + orc["Oracle only (today)"]:::lab + + lance -. "stand-in, no wiring" .-> vs + vs --> br + fs --> lb + br -. "G3" .-> ov + prev -. "G3" .-> ov + ov -. "reads" .-> lb + ov -. "smallest test" .-> q + q -. "today" .-> orc +``` + +The smallest falsifier named for G3 is: close store 12 on a branch. The overlay answer must equal the +oracle, and the canonical frame must not change. It has not been written, because the overlay frame +does not exist. + +Sources: + +- `crates/thought-git-probe/src/receipt.rs:37-60` (`BaseRef`, `VersionStore` as stand-ins) +- `crates/lance-graph-dt64/src/dr.rs:379-426` (`FrameStore`, `seal`, `advance`) +- `crates/lance-graph-dt64/src/lane.rs:153,379` (`LaneBinding`, `borrow_all`) +- `crates/thought-git-probe/tests/signals.rs:30` (the only "overlay" name found in the three crates) +- `.claude/plans/foundry-parity-v1.md (PR #1996):133` (matrix row 13, PARTIAL) +- `.claude/plans/foundry-parity-v1.md (PR #1996):292` (G3: needs an overlay frame contract joining a thought-git branch to `FrameStore`) +- `.claude/plans/foundry-parity-v1.md (PR #1996):72` (checklist: P3 scenario overlay frame, open) +- `.claude/plans/foundry-parity-v1.md (PR #1996):328-329` (roadmap P3) diff --git a/.mermaid/09-foundry-parity.md b/.mermaid/09-foundry-parity.md new file mode 100644 index 000000000..17879f3e9 --- /dev/null +++ b/.mermaid/09-foundry-parity.md @@ -0,0 +1,211 @@ +# 09 — Foundry parity: status map + +**This is an evidence map, not a parity claim.** Nothing was compared against a running Foundry. +The plan's own contract grades Foundry parity as a hypothesis, and no matrix row is graded +"parity demonstrated". The Foundry column cites public documentation only. There is no graphical +low-code builder here. + +The source of truth is `.claude/plans/foundry-parity-v1.md (PR #1996)`. That file lives on a branch with +an open PR, so line numbers below refer to that branch. + +Status classes: green = in code on main. Yellow = lab crate only. Blue = open PR. Red dashed = gap. +Gray = not exercised. + +## 1. The report in one table (plan section 0) + +| Question | Path | Status | +|---|---|---| +| `CanCook` | hand-written mask-RISC program (arm B) | lab, merged | +| per-store cost | INT8 packed GEMM (arm C) | lab, merged (#1995) | +| `CanCook` as a registered method (G6) | admit, bind, lowered once by relalg, execute | lab, open PR #1996 | +| per-store maximum | layer-cake sweep over masks | lab, slow: no element-wise min or argmin | +| governed read, scenarios, write-back, joined provenance | pieces in other crates | not composed on this path | + +Sources: + +- `.claude/plans/foundry-parity-v1.md (PR #1996):12-44` (section 0 report) +- `.claude/plans/foundry-parity-v1.md (PR #1996):3-4` (status: level R lab merged, G6 in #1996) +- `crates/foundry-parity-lab/src/` holds `arm_a.rs`, `arm_b.rs`, `arm_c.rs`, `bind.rs`, `fixture.rs`, `oracle.rs`, `source.rs` on main. No `method.rs` there, so G6 is on the PR branch only. + +## 2. Read side: capability to in-repo piece + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + f1["Object types and properties"]:::lab + f4["Object sets, filtering"]:::lab + f5["Computed properties, functions"]:::lab + f6["Versioned method behaviour"]:::shipped + f9["Source ownership"]:::lab + f12["Derived state, invalidation"]:::lab + f14["Aggregations"]:::lab + f3["Link traversal"]:::gap + + p1["ClassView basis + ClassSchema + typed lanes"]:::lab + p4["WHERE as a mask-RISC program"]:::lab + p5a["MethodBinding ON x WHEN x DO"]:::shipped + p5b["CanCook registered method (G6)"]:::open + p6["SigRev, GuardWitness, Stamp checked by admit"]:::shipped + p9["Borrowed lanes, 0 bytes copied"]:::lab + p12["LaneBinding::check on open"]:::lab + p14["Count, MaskedSum, INT8 GEMM"]:::lab + g1["G1: min and argmin across lanes"]:::gap + g5["G5: bound relations, waits on ADMISSION B"]:::gap + + f1 --> p1 + f4 --> p4 + f5 --> p5a + f5 --> p5b + f6 --> p6 + f9 --> p9 + f12 --> p12 + f14 --> p14 + f14 -. "limiting resource" .-> g1 + f3 -. "Gather is banned" .-> g5 +``` + +Sources: + +- `.claude/plans/foundry-parity-v1.md (PR #1996):119-125` (rows 1 to 5) +- `.claude/plans/foundry-parity-v1.md (PR #1996):126` (row 6, identity implemented, execution lab) +- `.claude/plans/foundry-parity-v1.md (PR #1996):129,132,134` (rows 9, 12, 14) +- `.claude/plans/foundry-parity-v1.md (PR #1996):123` (row 3, blocked) +- `.claude/plans/foundry-parity-v1.md (PR #1996):290,294` (G1, G5) +- `crates/lance-graph-dt64/src/lib.rs:215` (`MethodBinding`) +- `crates/lance-graph-dt64/src/lib.rs:772,1180` (`BindStore`, `admit`) +- `crates/lance-graph-dt64/src/lane.rs:153,379` (`LaneBinding`, `borrow_all`) + +## 3. Governance side: capability to in-repo piece + +The decision pieces exist. Applying them on a DT64 path is not built. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + f10["Security, authorization"]:::shipped + f7["Actions, state changes"]:::lab + f13["What-if scenarios"]:::lab + f11["History, replay"]:::lab + f16["Audit, decision provenance"]:::lab + f18["AI-assisted governed decisions"]:::gap + + d10["authorize_scoped: ScopeSpec + WideFieldMask"]:::shipped + g2["G2: apply it to a DT64 binding"]:::gap + d7["Rubicon outcomes, commit_cycle, xlog stage and commit, rbac Write and Act"]:::lab + g4["G4: one authorized, committed, witnessed action"]:::gap + d13["SlotPool::branch, next_version, stage_closure"]:::lab + g3["G3: overlay frame for a binding"]:::gap + d11["FrameStore revisions, receipts, replay"]:::lab + d16["Receipt, CommitWitness, auth_classid"]:::lab + gp["Provenance: one record per answer"]:::gap + g5x["G5: admission B decision (EXO-140)"]:::gap + + f10 --> d10 + d10 -. "G2" .-> g2 + f7 --> d7 + d7 -. "G4" .-> g4 + f13 --> d13 + d13 -. "G3" .-> g3 + f11 --> d11 + f16 --> d16 + d16 -. "not joined" .-> gp + f18 -. "needs 7, 10, 16" .-> g4 + f18 -. "needs 7, 10, 16" .-> gp + g5x ~~~ f18 +``` + +Sources: + +- `.claude/plans/foundry-parity-v1.md (PR #1996):127,130,131,133,136,138` (rows 7, 10, 11, 13, 16, 18) +- `.claude/plans/foundry-parity-v1.md (PR #1996):291-293` (G2, G3, G4) +- `crates/lance-graph-rbac/src/authorize.rs:170-200` (`ScopedDecision`, `visible_fields`) +- `crates/lance-graph-rbac/src/authorize.rs:210` (`authorize_scoped`) +- `OGAR/crates/ogar-rbac/src/lib.rs:155,327` (`authorize_user`, `authorize_identity`) +- `crates/lance-graph-xlog/src/cube.rs:856,909,939` (`stage`, `stage_closure`, `commit`) +- `crates/thought-git-probe/src/receipt.rs:153` (`commit_cycle`) +- `crates/lance-graph-r2il/src/binding.rs:124-131` (`ExecPermit`: "who issues it is not decided") + +## 4. Gap register and hypotheses + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef open fill:#d6e6fa,stroke:#1565c0,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + root["Gap register G1 to G6"]:::lab + G1["G1 element-wise min, argmin: open, needs a decision"]:::gap + G2["G2 governed read: decision exists, read-path wiring missing"]:::gap + G3["G3 scenario overlay frame: not built"]:::gap + G4["G4 governed action: pieces exist, not composed"]:::gap + G5["G5 bound relations: waits on ADMISSION B"]:::gap + G6["G6 registered method: done in lab, PR #1996 open"]:::open + H1["H1 semantic contract suffices for read-only: partly supported"]:::lab + H2["H2 missing algebra: supported by G1 and G5, narrowed by arm C"]:::lab + H3["H3 infrastructure: narrowed, composition is missing"]:::lab + + root --> G1 & G2 & G3 & G4 & G5 & G6 + G1 -.-> H2 + G5 -.-> H2 + G2 -.-> H3 + G3 -.-> H3 + G4 -.-> H3 + G6 -.-> H1 +``` + +No hypothesis is chosen over the others. All three hold in part. +The next step with the most value is G2 in the lab, using `authorize_scoped` instead of the stand-in. + +Sources: + +- `.claude/plans/foundry-parity-v1.md (PR #1996):286-295` (missing-contract register G1 to G6) +- `.claude/plans/foundry-parity-v1.md (PR #1996):297-316` (hypotheses H1 to H3) +- `.claude/plans/foundry-parity-v1.md (PR #1996):41-42` (next step) +- `.claude/plans/foundry-parity-v1.md (PR #1996):318-331` (roadmap P0 to P5) + +## 5. Table version + +Foundry capabilities are as documented in the plan. Status uses the four classes above, or "not exercised". +The verdict column repeats the plan's grading. No row is "parity demonstrated". + +| # | Foundry capability | In-repo piece | Status | Plan verdict | +|---|---|---|---|---| +| 1 | Object types, typed properties | `ClassView` basis, `ClassSchema`, typed lanes | lab | functionally related | +| 2 | Stable identity, provenance | DR route, witnessed coordinate, id lane | lab; per-value provenance gap | partial, provenance missing | +| 3 | Link traversal | relation lanes, `graph_hop` (feature-gated off) | gap | blocked (G5) | +| 4 | Object sets, filtering | WHERE as a mask program | lab | functionally related; lower-level advantage is a hypothesis | +| 5 | Computed properties, functions | `MethodBinding` + `MethodRef`; r2il DO effects | lab (G6 on open PR #1996) | functionally related | +| 6 | Versioned method behaviour | `SigRev`, `GuardWitness`, `Stamp` checked by `admit` | shipped (identity), lab (execution) | functionally related, read-only | +| 7 | Actions, state changes | thought-git write path, xlog stage and commit, rbac Write and Act | lab pieces; composition gap | partial | +| 8 | Multiple sources, sync | `LaneSource` per owner, `FrameStore` per source | lab, one source only | blocked | +| 9 | Source ownership | borrowed lanes, one owner | lab | lower-level advantage is a hypothesis | +| 10 | Security, authorization | `authorize_scoped` decision | shipped (decision); gap (applied on DT64 path, G2) | partial | +| 11 | History, replay | `FrameStore` revisions, receipts, replay | lab | partial | +| 12 | Derived state, invalidation | `LaneBinding::check` | lab | functionally related | +| 13 | What-if scenarios | `SlotPool::branch`, `next_version`, `stage_closure` | lab; overlay frame gap (G3) | partial | +| 14 | Aggregations | `Count`, `MaskedSumI32`, INT8 GEMM | lab; min and argmin gap (G1) | partial | +| 15 | Operational applications | a2ui-rs, consumer objects | not exercised | not comparable here | +| 16 | Audit, decision provenance | `LaneBinding`, `Receipt`, `CommitWitness`, `auth_classid` | lab pieces, not joined | partial | +| 17 | Data connections, egress | source adapters | not exercised | not comparable here | +| 18 | AI-assisted governed decisions | none on this path | gap | missing | + +Notes on the plan's own text: + +- Row 7 of the plan calls PR #1993 "open". It has since merged (`b34d195` on main): durable map-table writes through the cycle writer are now on main. +- `ExecPermit` has no decided issuer. The plan names an `Operation::Act` decision in `lance-graph-rbac` as the natural candidate. That is a candidate, not a decision. + +Sources: + +- `.claude/plans/foundry-parity-v1.md (PR #1996):111-138` (matrix and legend) +- `.claude/plans/foundry-parity-v1.md (PR #1996):113-117` (status and verdict vocabulary) +- `.claude/plans/foundry-parity-v1.md (PR #1996):295` (G6 row: issuer candidate) +- `crates/lance-graph-r2il/src/binding.rs:124-128` (`ExecPermit` doc) diff --git a/.mermaid/10-lgj-crosswalk.md b/.mermaid/10-lgj-crosswalk.md new file mode 100644 index 000000000..0e97cc0a2 --- /dev/null +++ b/.mermaid/10-lgj-crosswalk.md @@ -0,0 +1,175 @@ +# 10 — lance-graph-java crosswalk + +`lance-graph-java` (LGJ) is a separate repo. It puts a Java facade over lance-graph through Panama. +This page shows the layers, the one contract fence, and the two membranes. +Paths for that repo are written as `lance-graph-java/...`. + +Status classes: green = in code. Yellow = lab. Red dashed = not wired. + +## 1. Layers, top to bottom + +```mermaid +flowchart TD + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + facade["Java facade: View, Mask, RowStore"]:::shipped + ffm["Panama FFM membrane: internal/ffm (Abi, Downcalls, Engine, Layouts)"]:::shipped + abi["native/lgj-abi: versioned machine ABI, 30 symbols, generation-checked handles"]:::shipped + contract["lance-graph-contract: ClassView, FieldMask, WideFieldMask"]:::shipped + maskrisc["lance-graph-mask-risc: the one mask evaluator"]:::shipped + nd["ndarray::simd: the only SIMD source"]:::shipped + + facade --> ffm + ffm --> abi + abi --> contract + abi --> maskrisc + maskrisc --> nd + abi --> nd +``` + +What the diagram says: + +- The facade is plain Java. `View.where` returns a new view and does not cross the membrane. A terminal call such as `count` runs the whole chain in one crossing. +- The FFM layer is internal. `ApiSurfaceTest` checks by reflection that no `java.lang.foreign` or `internal` type appears in a public signature. +- `lgj-abi` is a machine ABI, not a C API. It has no header and no JNI. The compiled library describes itself in a manifest, and Java checks every size and alignment against its own layout constants before the first call. +- `lgj-abi` depends on the contract crate, on `lance-graph-mask-risc` and on `ndarray`. It lowers a plan to one `mask_risc::Program` and keeps no second evaluator. +- All SIMD comes from `ndarray::simd`. Java has no compute backend. + +Sources: + +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/View.java:1-42` (lazy view, one crossing per terminal) +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/Mask.java:36` (`Mask`) +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/RowStore.java:43` (`RowStore`) +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/internal/ffm/Downcalls.java:38` (`Linker.nativeLinker()`) +- `lance-graph-java/java/src/test/java/com/adaworldapi/lancegraph/ApiSurfaceTest.java:40,97` (internal prefix check by reflection) +- `lance-graph-java/docs/abi.md:8-58` (no C, manifest instead of a header) +- `lance-graph-java/docs/abi.md:60-84` (the ABI is small and bulk-only; 30 symbols) +- `lance-graph-java/docs/abi.md:420-441` (anti-JNI rule) +- `lance-graph-java/native/lgj-abi/Cargo.toml:30,46,59` (ndarray, contract, mask-risc dependencies) +- `lance-graph-java/native/lgj-abi/src/plan_lower.rs:1-7` (plan lowers to one `mask_risc::Program`) + +## 2. Handles and the G11 fence + +A handle is an opaque `u64`: a generation in the high half and an index in the low half. +The generation is bumped each time a slot is freed. A stale or fabricated handle returns a status code. +The Java caches of lane and mask addresses are weaker. `Mask.words()` re-checks the epoch on use. +`RowStore`'s cached lanes do not. The ABI document records this as an open issue. + +The G11 fence limits what `lgj-abi/src` may import from the contract crate. +Only four modules are allowed: `canonical_node`, `class_view`, `facet` and `ontology`. +The cognitive modules compile unconditionally, so this test is the only barrier. +The test walks `src/` recursively and also rejects aliases and glob imports. +It reads the lists in `CLAUDE.md` and `Cargo.toml` and requires them to equal its own list. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + src["native/lgj-abi/src"]:::shipped + fence{"G11 fence test"}:::shipped + ok["canonical_node, class_view, facet, ontology"]:::shipped + no["kanban, mul, cognition, soa_view, scheduler, and the rest"]:::gap + reg["Registry: handle = generation + index"]:::shipped + st["Stale handle: INVALID_HANDLE status"]:::shipped + + src --> fence + fence --> ok + fence -. "refused" .-> no + src --> reg + reg --> st +``` + +Sources: + +- `lance-graph-java/native/lgj-abi/tests/g11_contract_import_fence.rs:1-40` (what the fence closes) +- `lance-graph-java/native/lgj-abi/tests/g11_contract_import_fence.rs:44` (`ALLOWED`) +- `lance-graph-java/native/lgj-abi/tests/g11_contract_import_fence.rs:224-228` (violation message) +- `lance-graph-java/native/lgj-abi/tests/g11_contract_import_fence.rs:268-273` (documented lists must equal `ALLOWED`) +- `lance-graph-java/native/lgj-abi/Cargo.toml:36-46` (fence comment and contract dependency) +- `lance-graph-java/native/lgj-abi/src/class_view_provider.rs:64-66` (the contract imports in use) +- `lance-graph-java/docs/abi.md:245-272` (handle layout and results) +- `lance-graph-java/docs/abi.md:283` (cached descriptors, `Mask` versus `RowStore`) +- `lance-graph-java/native/lgj-abi/src/registry.rs:1-18` (generation-checked registry) + +## 3. Two membranes: Panama and Valhalla + +Panama carries the verb: operations across the boundary onto lance-graph's bytes. +Valhalla carries the noun: small identity-free values such as `LaneId` and `WideFieldMask`. +Neither owns storage. lance-graph owns the only canonical copy. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef lab fill:#fff3cd,stroke:#a67c00,color:#111 + + java["Java code"]:::shipped + val["Valhalla value records: LaneId, MaskId, Ordinal, FacetId, RowRange, WideFieldMask"]:::shipped + pan["Panama FFM: the computation membrane"]:::shipped + own["lance-graph: owns the canonical bytes"]:::shipped + lab["valhalla-lab: measurement arms, not gates"]:::lab + + java -->|"names and addresses"| val + java -->|"verbs"| pan + pan --> own + val -. "storage membrane: no bytes held in Java" .-> own + lab -. "measures" .-> val +``` + +The repo's own text calls these two orthogonal guarantees, not one layer. +The Valhalla types are declared `public value record`. The repo pins JDK 28 for them. +JEP 401 is a preview feature, so the build sets the preview flag repo-wide. + +Sources: + +- `lance-graph-java/CLAUDE.md:22-47` (JDK 28, two membranes, neither owns storage) +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/LaneId.java:31` (`public value record`) +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/MaskId.java:23` +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/Ordinal.java:19` +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/FacetId.java:21` +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/RowRange.java:19` +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java:34` +- `lance-graph-java/CLAUDE.md:415-440` (the surface is meant to look like `sql()`; mask verbs on a public Java signature are a defect) + +## 4. The contract types + +`WideFieldMask` in Java mirrors `lance_graph_contract::class_view::WideFieldMask`. +It is a one-word, small-tier mirror. The wide promotion is a named seam, not built. + +Sources: + +- `lance-graph-java/java/src/main/java/com/adaworldapi/lancegraph/WideFieldMask.java:1-12,34` (mirror of the contract currency) +- `crates/lance-graph-contract/src/class_view.rs:70` (`FieldMask`) +- `crates/lance-graph-contract/src/class_view.rs:243` (`WideFieldMask`) +- `crates/lance-graph-contract/src/class_view.rs:1057` (`ClassView` trait) + +## 5. DT64 binding and LGJ: not wired + +LGJ reaches the contract crate and `lance-graph-mask-risc`. It has no dependency on `lance-graph-dt64`. +A case-insensitive search of the `lance-graph-java` tree finds "dt64" only in one plan document, +`docs/plans/lgj-holographic-object-facade-v1.md`, where it appears in prose. No Cargo edge or source +line links the two. The search is navigation, not proof of absence. + +```mermaid +flowchart LR + classDef shipped fill:#d8f0d8,stroke:#2e7d32,color:#111 + classDef gap fill:#fadadd,stroke:#c62828,stroke-dasharray:5 3,color:#111 + + lgj["lgj-abi"]:::shipped + contract["lance-graph-contract"]:::shipped + mr["lance-graph-mask-risc"]:::shipped + dt["lance-graph-dt64: BindStore, LaneBinding, FrameStore"]:::shipped + + lgj --> contract + lgj --> mr + dt --> contract + lgj -. "not wired" .-> dt +``` + +Sources: + +- `lance-graph-java/native/lgj-abi/Cargo.toml:30-82` (dependencies: ndarray, contract, mask-risc, optional ogar-class-view, dev-only quack) +- `lance-graph-java/docs/plans/lgj-holographic-object-facade-v1.md:13,18` (the only "DT64" mention found) +- `crates/lance-graph-dt64/Cargo.toml:10` (path dependency on the contract crate) diff --git a/.mermaid/README.md b/.mermaid/README.md new file mode 100644 index 000000000..245438d98 --- /dev/null +++ b/.mermaid/README.md @@ -0,0 +1,51 @@ +# Architecture diagrams + +These pages draw the DR / DT64 / mask-risc stack as it is in the repository. Each diagram has a "Sources" list of `path:line` that the author opened. + +Two limits apply to every page: + +- Nothing here claims Foundry parity. There is no graphical low-code builder. +- A gap is drawn as a gap. If code for a link does not exist, the link is dashed. + +## Status legend + +| mark | meaning | +|---|---| +| solid edge, green box | in code on main | +| yellow box | lab or probe crate only (excluded from the root workspace) | +| blue box | open pull request, not on main | +| red dashed box, dashed edge | gap: not built | +| grey box | another repository | + +All pages use this legend. Pages that need extra classes (grey = another repository) say so at the top. + +## Reading order + +1. `01-stack-overview.md`, for the layers. +2. `02-crate-dependencies.md`, for the real Cargo edges. +3. `03-dr-dt64-binding.md`, for the binding types and refusals. +4. `04-read-path.md`, for one governed read and the G2 gap. +5. `05` to `09`, for methods, authorization, the write path, scenarios and the parity status map. +6. `10-lgj-crosswalk.md`, for the Java side. + +## Files + +| file | what it is | +|---|---| +| `README.md` | This index. | +| `01-stack-overview.md` | The layers: contract, dt64, execution, authorization, write and scenario, lab. One overview flowchart and one table. Shows which layers depend on which and where the open gaps G2, G3 and G4 sit. | +| `02-crate-dependencies.md` | The Cargo dependency edges, read from each `Cargo.toml`. Splits workspace members from excluded crates, and shows the git edges between lance-graph and OGAR in both directions. Ends with a table that lists dev-dependencies. | +| `03-dr-dt64-binding.md` | The `lance-graph-dt64` types. Three class diagrams (DR route log and map table; frames and guards; ClassView, BindStore and lanes) and one sequence diagram of bind then borrow, with every refusal branch. | +| `04-read-path.md` | A governed read from a sealed frame to a result mask, how a `Program` gets built, and the G2 gap: an authorization decision exists but nothing applies it to a binding. | +| `05-methods.md` | The method binding, ON x WHEN x DO. DT64 stores it and executes nothing; other crates run the WHEN part and the DO part separately today. | +| `06-authorization.md` | From login to authenticated user to RBAC kernel to a decision with row scope and visible fields. The decision exists; applying it to a DT64 binding is not built. | +| `07-write-path.md` | The pieces of a write: thought outcomes, cycle commit, receipts, cube staging. They are not composed into one authorized, preconditioned and witnessed write-back (gap G4). | +| `08-scenarios.md` | What-if work: branches, land and squash in `thought-git-probe`, previews in the cube. A DT64 binding cannot read a branch yet (gap G3). | +| `09-foundry-parity.md` | A status map of evidence against the parity plan. It is not a parity claim and nothing was compared against a running Foundry. | +| `10-lgj-crosswalk.md` | The separate `lance-graph-java` repo: its layers, the one contract fence and the two membranes (Panama and Valhalla). | + +## How the diagrams were checked + +- Every Mermaid block was rendered with `mmdc` before the page was handed over. +- Cargo edges were read from the manifests, not from memory. +- Line numbers cite the checkout in this repository at the time of writing. They drift as files change.