lagoon: axis reductions, norms, sorting, and distances (Hoon)

[sigilante]

Step 1 of the Lagoon foundation work list in #86, Hoon only: no jet hints, and no existing arm is touched. The diff against main is pure addition (+650 in lagoon.hoon, 0 deletions), so the C jet mirror in lagoon/vere stays valid as-is.

What this adds

Reductions along a dimension (dim=@ud, 0 = outermost, as ++stack already uses):

• ++sum-dim, ++prod-dim, ++max-dim, ++min-dim — kind-generic through +fun-scalar; %unum sums each slice exactly in the quire, as ++cumsum already does for the whole array
• ++argmax-dim, ++argmin-dim — index of the first extremum, matching ++argmax/++argmin
• ++mean / ++mean-dim, ++var / ++var-dim (with ddof), ++std / ++std-dim — %i754
• ++norm / ++norm-dim over a new $norm-ord: %l1, %l2, %linf, %fro. %l1/%linf are kind-generic; %l2/%fro need a square root and so are %i754

Reordering and selection: ++sort-dim and ++argsort-dim (%asc/%des, ties to the lower original index, so stable and deterministic), ++argtop-dim (indices of the k largest), and ++take-dim (NumPy's take_along_axis), which is what makes the index-producing arms usable.

Distances: ++cdist-sq (m x d against n x d, giving m x n) and ++pdist-sq.

Shape: ++broadcast-to, by NumPy's rules (right-aligned, 1s expand).

Plus the helpers those share: +prod-list, +drop-dim, +dim-parts, +dim-slices, +slice-flat, +fold-slice, +slice-op, +slice-idx, +rank-slice, +norm-slice, +i754-sun, +i754-sqt.

Two decisions that are jet contracts

Both are pinned by tests, because a future jet can silently "optimize" either one and still look right on friendly inputs.

1. Reductions fold LEFT TO RIGHT along the axis, seeded with the slice's first element: ((x0 op x1) op x2) .... That is the order a C or Rust kernel walks, so these arms stay jettable without a reversed loop. Note the whole-array ++cumsum/++max/++min fold right (they use +reel), so on an inexact sum ++sum-dim over a rank-1 ray and ++cumsum can differ in the last bits. It is also not NumPy's pairwise summation. test-sum-dim-order-* pins it with [1, 2^-24, 2^-24] in binary32, where the two fold directions disagree: left gives 0x3f800000, right gives 0x3f800001.
2. ++cdist-sq sums squared differences directly, never via the Gram identity |x|^2 + |y|^2 - 2*A*B^T. The identity is much faster (one ++mmul) but rounds differently and can return small negatives. test-cdist-sq-direct-not-gram-{32,64} uses inputs where the two formulas differ by one ulp in both widths, so a jet that reaches for the identity fails.

Other conventions, all documented in the section header and the README: reductions drop the reduced dimension (NumPy keepdims=False) and a rank-1 ray reduces to shape ~[1]; index outputs are %uint bloq 6 whatever the input width; ++argtop-dim replaces the dimension with k instead of dropping it; sorting with NaN present is unspecified (NaN compares false both ways and falls back to the index tiebreak).

Tests: 91 new arms, bit-exact

lagoon/desk/tests/lib/lagoon-axis.hoon (55), lagoon-sort-dist.hoon (29), lagoon-axis-rounding.hoon (7), table-driven in the style of the existing files.

The first two are built so that every input and every intermediate is exactly representable in binary16 through binary128 — sums, the one division in a mean, the one square root in a std. That means each expectation holds in all four widths and in all four rounding modes, so the tests compare whole rays with = rather than a tolerance. Coverage includes rank-3 rays, where reducing the middle dimension strides over the data (inner > 1) — the case the rank-2 tests cannot see — and %uint for the kind-generic arms.

The third file uses raw bit patterns to pin the two decisions above.

lagoon/tools/gen-axis-oracle.py recomputes every expectation in all three files (228 checks, 0 failures) against NumPy plus exact rational arithmetic. It does not just compare final answers across widths: it mirrors each individual operation in fractions.Fraction and asserts that nothing rounded at any width, which is the property the all-widths claim actually rests on. It also cross-checks its own IEEE-754 encoder against NumPy, and confirms the Gram case discriminates.

Worth recording, since it bit me: numpy's longdouble is not binary128 here — on this machine it has a 52-bit significand, i.e. it is plain float64 — so np.float128 would have silently answered as f64. That is why the binary128 coverage rests on exactness rather than on a numpy oracle.

Verification

hoonc + NockVM harness (#83's branch, which compiles this exact desk copy):

244 tests, 0 failed          # 153 pre-existing + 91 new
244 tests, 0 failed          # again with LAGOON_JET_DISABLE=1, i.e. pure Hoon
python3 lagoon/tools/gen-axis-oracle.py   # 228 checks, 0 failures

On a ship: a fresh fakezod booted from urbit-408k-rc1.pill on vere 4.6, %num desk at [%zuse 408] carrying /lib/lagoon, /sur/lagoon, the libmath deps and all 17 lagoon test files. -test /=num=/tests/lib, this branch against origin/main, same ship, same runtime:

	OK	failed/crashed
origin/main	132	21
this branch	223	21

The 91 new tests all pass, and the failure sets are identical — comm on the two lists is empty in both directions. The 21 are pre-existing on this runtime and reproduce on main: 16 failures plus 2 crashes across lagoon-jet-parity, test-mod-nearest-6r/test-mods-7r, test-sub-asym, and test-argmax. That is the expected Hoon-vs-C gap while urbit/vere#1057 is unmerged: this vere's lagoon jets predate numerics#78 and the parity fixes, so the tests written against the fixed jets fail. None of it is touched by this PR.

Two things found along the way, both pre-existing and already explained by this repo's own history:

• +diag and +transpose crash on any ship whose vere predates noun: fix validation, refcounts, and results in lagoon jets vere#1057, main included. This is the old C jets, not the Hoon: an inline copy of +diag's own body, placed outside the jetted core, runs fine on the same ship, while diag:la crashes. lagoon: jet parity with urbit/vere#1057 — Hoon fixes, C ports, regression suite #75 records exactly this ("the Hoon flopped it; the jet crashed unconditionally", and the old transpose jet was row/column-swapped without swapping the result shape); lagoon: jet parity with urbit/vere#1057 — Hoon fixes, C ports, regression suite #75 fixed the Hoon and ported the fixed C into this repo's mirror, but the runtime fix (vere#1057) is still open, so a stock vere still crashes. test-pdist-sq-diagonal-zero originally read the diagonal through +diag and so crashed; it now reads the three entries with +get-item, testing the same property without depending on a broken arm.
• Renaming a ~/ or ~% hint does not de-jet a lagoon arm on vere 4.6. Neither the parent %non label nor an arm's own hint (%diagx, %transposex) stopped the jets from firing; only Hoon living outside the registered core ran unjetted. So the de-jetted-twin technique in our notes does not work on this runtime — compare against an inline copy, or use the hoonc harness's LAGOON_JET_DISABLE.

(Kelvins, for the record: the fresh pill ship is hoon-135/zuse-408, the emissary-dev ship hoon-136/zuse-409, and the hoonc harness hoon-138. Both ships show the crash; the harness does not, since its jets are the corrected Rust ones.)

Not in this PR

Still open from #86's foundation step: pad, flip, n-D permute, concatenation along an arbitrary axis, and the jets for the hot arms (reductions along an axis, norms, distances) in both the C mirror and NockVM. Upstreaming the Hoon to urbit/urbit is a separate PR, per AGENTS.md.

Refs #86.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TgBnKsUPYzkoPZonjePnaq