[Mirrdhyn]

What this changes

POST /v1/decisions now works with Flash Next on one CUDA GPU. Before this change, the lanes engine did not answer the route.

• Lane host: a lanes backend may provide a score hook. The lane host calls it on the engine thread between rounds (lane_host_score.zig). A backend without the hook refuses as it does today.
• Flash Next: the hook runs one plain prompt pass into a fresh cache. It reads the label logits and the f64 logsumexp from that pass's last row (cuda_score.zig). Kernels and arithmetic are unchanged. A two-rank run refuses with UnsupportedRankOperation.

Receipt

• Environment: TensorFold main d1018b31 plus these two commits, on an NVIDIA DGX Spark (GB10 sm_121), with driver 595.84, nvcc 13.2.86 and Zig 0.17.0. Checkpoint: Flash Next MLX 4-bit with MTP. Server flags: --context 32768 --parallel 2 --prompt-cache-gib 0, with TF_FLASHNEXT_CHUNK=16384.

• Exactness: tf-cuda-test flashnext-score at 16, 300 and 2,600 prompt tokens. The 5 label logits and the logsumexp equal the plain pass's last row, bit for bit, and the best label is the greedy token. On JevBench, two passes give identical probabilities on 231 of 231 items. Decisions sent while two streams decode 512 tokens are identical to the decisions sent alone, 231 of 231. In the same run, bench_concurrent.py --alone --serial (2 streams, T=0, code and chat prompts) gives 8 of 8 streams equal to their reply alone and 2 of 2 equal to the serial reply.

• Decision quality and latency: JevBench public set (fstandhartinger/jevbench at 609a664e), 231 decisions, one request per item. Same machine and same checkpoint, compared with the Python engine 0.6.4 (qwen4_exp):

  	Python 0.6.4	This PR
  easy	48/48	48/48
  original	69/72	69/72
  hard	69/111	73/111
  total	186/231	190/231
  ECE	0.052	0.037
  p50 latency	0.221 s	0.216 s
  p90 latency	0.93 s	1.82 s

  • The two engines choose the same label on 225 of 231 items. The 6 differences are hard items at confidence 0.35–0.59, and 5 of them go this PR's way. The median per-item |Δp| is 0.0043; the maximum is 0.234.
  • The p90 gap comes from the long prompts, through main's prompt pass, which this PR does not touch. On a local branch with Flash Next CUDA: prompt processing 1.4-1.8x faster on GB10, bit for bit with 1.1.0 #1046 applied, p90 is 1.19 s.

• Decode speed: n/a, the decode path is unchanged.

• Prompt speed: n/a, the prompt pass is unchanged. A decision is one plain pass.

• Tests run:

  • zig fmt --check: OK.
  • zig build native and the install build: OK.
  • zig build test: 575/579 passed (4 skipped).
  • zig build test-cuda-host: 89/89.
  • test-kernel-contract, test-golden and fuzz-cortheon: OK.
  • lean_check.py: 444 problems, same as main.
  • kernel-contract: SUMMARY 85 kernels: 85 pass gates 1-4, 0 fail, 0 known findings; 7 controls: 7 red as planted, 0 missed; 899 limit problems (no kernel change).
  • tf-cuda-test: 19 suites passed (smoke, symbols, route-rows, experts-rows, flashnext-hc, flashnext-attn, flashnext-dense-hc, flashnext-draft-affine, flashnext-shared, flashnext-nvfp4-tp, norms, norm-rows, sample, projection-f32, mxfp8, quant-ref, modelopt, glue, glm-ref), plus flashnext-score.
  • The new test fails on a planted bug: with a label reading its neighbour's logit (row[t] → row[t + 1]), flashnext-score fails with label 13's logit is the plain pass's row value (5.25 vs 9.625).

• Not run:

  • Two ranks: refused by name.
  • Images in a decision prompt: refused, because score has no image input.
  • Metal: the lane host change is covered by the host tests (lane_host_score_test.zig), but no Metal machine was run.
  • Batched decisions: each decision is its own prompt pass. Concurrent decisions are not grouped into one pass.

Checklist

• Every token still goes through the lane rounds. A decision generates no token; its pass runs on the engine thread between rounds.
• Drafted output equals "draft": false, a resumed prompt equals a fresh one, and concurrent equals solo. Decisions under two decoding streams equal the decisions sent alone.
• No precision traded for speed. No kernel or sum changes.
• A new or changed GPU kernel is registered with the kernel contract. No kernel change; the SUMMARY line is above.
• Prompt processing is no slower than the last release. The prompt pass is unchanged.
• The description says which platforms I ran: CUDA GB10 only, not Metal.
• New tests fail before the change and pass after it. flashnext-score fails on the planted bug above. It takes a model and an id file, and is not part of the synthetic suites.
• Comments and docstrings are one line. No measurements or history in the source.
• No personal data, machine names, internal hosts or local paths. No AI attribution lines.

[ashhart]

Thanks @Mirrdhyn for putting decision scoring on the engine thread and for the bit-exact last-row test against the plain pass. We are carrying both commits onto current main ourselves, kept as yours, and a separate commit of ours will complete pending asks on every exit path of the engine thread (today run returns at zig/src/core/lane_host.zig:515 before any scoring.answer, so an ask queued before a start-up failure waits at zig/src/core/lane_host_score.zig:30 until a later stop), offer .score only on one rank (zig/src/families/flashnext/cuda_lanes.zig:214) so two ranks get the server's refusal rather than a failure, add tests for both, and keep cuda_lanes.zig under 600 lines. One choice is yours: the fresh sequence at zig/src/families/flashnext/cuda_score.zig:12 allocates its caches and state outside stream admission, so would you rather a scoring call take a priced slot of its own, or go through ordinary admission like a stream? Whichever you pick, we will add a test with every normal slot occupied. Then it goes to the host tests and the GB10 equality, occupancy and concurrency runs, and lands. You need not rebase; if you are in the middle of a change, please tell us so we do not collide.

- Synthema, Ash's AI Assistant.

[Mirrdhyn]

Thanks Ash. A priced slot of its own, please. A decision generates no token: it runs one prompt pass and frees its sequence at once. Through ordinary admission, a decision sent while every slot holds a decoding stream waits until a stream finishes, and callers that use decisions as a quick pre-step (ours gives up after 2 s) then fall back. The slot's cost is bounded: fixed recurrent state plus a KV cache sized to the decision's prompt.

I have a follow-up on a local branch that touches lane_host_score.zig and cuda_score.zig: a request's questions share one prompt pass. It is not pushed, so there is no collision. I will rebase it onto your version once this lands and open it separately.

[ashhart]

Landed on main as 9761e4e and 76928be, with a maintainer fix-up (1a4b6bf) that completes pending scores when the worker exits and refuses ranked scoring, thank you @Mirrdhyn. /v1/decisions on Flash Next CUDA scores labels from the prompt's last row, answered on the engine thread between rounds. The full host suite passes on main.

- Synthema, Ash's AI Assistant.

[ashhart]

This shipped in TensorFold 1.1.4: https://github.com/ashhart/TensorFold/releases/tag/v1.1.4

Thank you @Mirrdhyn for the contribution.

- Synthema, Ash's AI Assistant.