PCA.fit: gemm covariance offers a further 1.8x-3.6x but breaks bit-identical fitted state

[godofecht]

Follow-up from #211 (PR #350). Measured and deliberately NOT shipped there, because #211 scoped itself to allocation/cache/symmetry work and required the fitted state to stay bit-identical.

The measurement

PR #350 got PCA.fit to roughly 2.3x-2.9x by hoisting allocations, computing only the covariance upper triangle, and replacing accessor calls with direct row-major indexing, all while preserving every arithmetic expression term for term.

A prototype that instead forms the covariance with a single cblas_sgemm was built purely to measure it, then reverted:

shape	origin/main	PR #350	gemm prototype
20000x8 -> 4	6.90 ms	2.26 ms	1.24 ms
4000x64 -> 16	110.97 ms	38.00 ms	10.44 ms
400x200 -> 8	196.16 ms	76.33 ms	43.07 ms

So a further 1.8x-3.6x on top of what #350 already banks.

Why it was not taken

The fitted state stops being bit-identical. explained_variance[0] moves from 3176.930419922 to 3176.930908203 — a few f32 ulps from gemm's different accumulation order.

That matters more for PCA than for most estimators. benchmarks/parity_contract.json gives PCA the strictest contract in the repo: explained-variance ratios, singular values, reconstruction error AND sign-aligned components. Eigenvector signs are the real hazard, since a reordering that flips a sign breaks the contract outright rather than drifting within it.

What a follow-up needs to establish

• whether the drift stays inside the declared contract tolerances across all canonical shapes, not just the ones measured here
• whether component signs are stable under the new accumulation order
• a small-input work threshold, since OpenBLAS spends ~0.1 ms on thread dispatch and PCA.fit is called on iris-sized data in the canonical benchmark (see the KMeans experience in perf: KMeans.transform via squared-euclidean expansion and one sgemm #332/perf: KMeans.predict argmin from one sgemm with an exact-tie fallback #341/perf: cut KMeans Lloyd iteration cost without touching its trajectory #345, where the scaled gate caught exactly this)

If the drift turns out to be contract-compatible, this is one of the larger remaining wins in the top 20. If it is not, that is worth recording explicitly so nobody re-derives the prototype a third time.