feat(fsst): Compute uncompressed size without decoding - #9755
Conversation
Signed-off-by: Will Manning <will@willmanning.io>
Merging this PR will improve performance by 14.57%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | decompress[u64, (4000, 1024)] |
85.7 µs | 70.4 µs | +21.81% |
| ⚡ | Simulation | allocate_drop_bytes[0] |
520.2 ns | 466 ns | +11.62% |
| ⚡ | WallTime | mul_u32_nonnull_avx512 |
6.3 µs | 5.7 µs | +10.61% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing wm/fsst-uncompressed-size (655c3dd) with develop (dab1684)
Footnotes
-
206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Summary
FSST already stores each value’s uncompressed length. This adds an FSST-specific
UncompressedSizeInBytesaggregate that calculates the canonical size without decoding string payloads.Changes
The kernel reads only the lengths child and accounts for canonical views and validity. It rejects negative lengths and arithmetic overflow. Tests compare results with materialized UTF-8 and binary arrays. They cover nullable, all-null, and sliced inputs.
🤖 Generated with Codex