feat(fastlanes): put per-chunk bit widths behind a preview edition

[mhk197]

Summary

Stacked on #9750. Lets the writer emit the fastlanes.bitpacked_v2 format, gated by editions.

• Declares preview2026.09.0, a draft preview edition whose only member is fastlanes.bitpacked_v2, and makes it the facade's DEFAULT_PREVIEW_EDITION. preview2026.08.0 stays empty.
• Adds Scheme::produced_serialized_ids (default: the produced encodings) and has the writer filter BtrBlocks schemes against the enabled editions' serialized IDs rather than in-memory encodings. Every existing scheme is unaffected, since its encodings have one format each; the new hook is what lets one in-memory encoding with two formats be gated per format.
• Adds BitPackingV2Scheme. It produces ordinary BitPacked arrays with per-chunk widths and declares that its output needs fastlanes.bitpacked_v2, so a writer whose editions do not permit that format drops the scheme and falls back to BitPackingScheme. No cargo feature gates it: in-memory compression uses per-chunk widths everywhere, and only the wire format is edition-gated. While it is present, BitPackingScheme defers to it, so FoR residuals and sampled estimates get per-chunk widths too and the two schemes never compete. The width table child is re-encoded through the cascade's compress_child, like the children of the decimal and temporal schemes. The CUDA-compatible preset excludes it, since there is no per-chunk CUDA kernel yet. The scheme returns the original array when half or more of the values would be patches: a few wide values otherwise pack at width 0 with nearly everything patched, which beats raw storage by a couple of buffer bytes and loses them back in footer metadata.

Tests

• preview_2026_09_adds_bitpacked_v2: the edition is a draft and adds exactly that format ID.
• core_writer_never_emits_bitpacked_v2: a session with the default encodings and only the core edition enabled writes a column whose chunks need 1 to 22 bits, and every array it produces reads back as fastlanes.bitpacked.
• preview_writer_emits_bitpacked_v2 (unstable_encodings, since only then does the facade enable the preview edition): the default session writes the same column as fastlanes.bitpacked_v2 and it reads back equal.
• test_mostly_patched_stays_primitive: six i64 values, five of them wide, stay primitive instead of becoming width 0 with five patches.
• cuda_compatible_excludes_per_chunk_bitpacking: the CUDA preset keeps BitPackingScheme and drops the per-chunk scheme.
• The vortex-btrblocks golden snapshots now show per-chunk widths and a width_table child for bit-packed columns in both the default and unstable configurations; the four default goldens that change all shrink. The tables in the goldens stay primitive, since nothing beats raw storage for 2 to 17 bytes.

Validation

• cargo nextest run -p vortex-compressor -p vortex-file -p vortex-edition -p vortex-btrblocks -p vortex editions, with and without --features unstable_encodings
• cargo clippy --all-targets --all-features on vortex-compressor, vortex-file, vortex-edition, vortex-btrblocks, and vortex with unstable_encodings; cargo +nightly fmt --all

[codspeed-hq]

Merging this PR will regress 2 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 2 regressed benchmarks
✅ 2337 untouched benchmarks
⏩ 206 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	random_i8[0.8]	69.4 µs	98.7 µs	-29.72%
❌	WallTime	arrow_checked_add_u32_avx512[16384]	17.6 µs	21.2 µs	-17%
⚡	WallTime	arrow_checked_add_u32_neon[16384]	20.3 µs	12.8 µs	+58.2%
⚡	Simulation	random_i8[0.5]	90.8 µs	66.9 µs	+35.68%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/bitpacked-v2-editions (01b289f) with mk/bitpacked-v2 (b5b9d6d)²}

Footnotes

1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on mk/bitpacked-v2 (013afc1) during the generation of this report, so 2cf069e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[robert3005]

just a caution that preview means that we are 99% certain that this is the right serialisation format