Tell the compressor which serialized IDs the writer may emit

[mhk197]

Summary

Tells the compressor which serialized IDs the writer may emit, so a scheme whose encoding has more than one wire format can produce the newest form the enabled editions permit. Behaviour-neutral on its own: no scheme consults the set yet, and without a restriction every ID is allowed.

Two format revisions need this and were each carrying their own copy of the plumbing: per-chunk bit widths in fastlanes.bitpacked_v2 (#9754) and 64-bit lower parts in vortex.decimal_byte_parts_v2 (#9759, with the compressor change to follow). Landing it alone lets both build on one reviewed mechanism.

Changes

vortex-compressor. CascadingCompressor gains an optional set of allowed serialized IDs. with_allowed_serialized_ids sets it, intersecting with any earlier restriction, and allows_serialized_id answers the question a scheme asks inside compress. None means unrestricted, which is the newest format of everything. No change to the Scheme trait, since compress already receives the compressor.

vortex-btrblocks. BtrBlocksCompressorBuilder::allow_serialized_ids threads the set through build, next to the existing retain_allowed_encodings filter on in-memory encodings. The CUDA preset restricts itself to each scheme's original formats, since the CUDA kernels decode those; an encoding with newer formats keeps producing the original one under that preset.

vortex-file. new_array_context also returns the serialized IDs of the enabled editions, and the default write path hands them to the builder. An explicitly supplied strategy is still not reconfigured, as before.

Spec. The "Writing with an edition" paragraph now says the writer hands the compressor the allowed serialized IDs so a scheme can pick the newest permitted form, and that the compressor never reads editions directly. The serializer remains the final gate.

API Changes

New public methods: CascadingCompressor::with_allowed_serialized_ids, CascadingCompressor::allows_serialized_id, and BtrBlocksCompressorBuilder::allow_serialized_ids. No behaviour changes for existing callers.

[codspeed-hq]

Merging this PR will regress 2 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 2 regressed benchmarks
✅ 2191 untouched benchmarks
⏩ 206 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	random_i8[0.8]	69.8 µs	99.3 µs	-29.72%
❌	WallTime	mul_i32_nonnull_avx512	7 µs	7.8 µs	-10.34%
⚡	WallTime	arrow_checked_add_u32_neon[16384]	20.5 µs	12.7 µs	+60.72%
⚡	Simulation	random_i8[0.5]	91.3 µs	67.4 µs	+35.4%
⚡	WallTime	arrow_checked_add_u32_avx2[16384]	21.3 µs	17.7 µs	+20.56%
⚡	Simulation	allocate_drop_arrow[0]	456.9 ns	402.7 ns	+13.45%
⚡	WallTime	filtered_sink_i64_neon[NineNullsInTen]	28.4 µs	25.3 µs	+12.14%
⚡	WallTime	filtered_owned_i64_neon[NineNullsInTen]	28.4 µs	25.7 µs	+10.63%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/compressor-serialized-ids (85990c2) with develop (0d9d5b8)}

Footnotes

1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩