Skip to content

perf(theta): gather and bucket-sort hashes when compacting update sketches - #344

Open
davecromberge wants to merge 1 commit into
apache:mainfrom
davecromberge:perf-theta-compact
Open

davecromberge wants to merge 1 commit into
apache:mainfrom
davecromberge:perf-theta-compact

Conversation

@davecromberge

Copy link
Copy Markdown
Member

Speeds up ThetaSketch::compact by gathering retained hashes without a per-slot branch and bucket-sorting them for ordered output; the compact bytes are unchanged.

  • Hashes are uniform below theta, so a counting pass on their top bits followed by insertion sort replaces sort_unstable; a bucket over 32 entries or fewer than 256 hashes falls back to sort_unstable.
  • Tables at most 1/16 full keep the simple loop, so tiny sketches are not slower.

Speedup vs main from the characterization theta_compact_serde_profile job (lg_k 12; geometric mean over each band, best of 3 interleaved passes):

TrueU compact(true) compact(false)
1–64 1.67× 3.1×
65–4,096 2.1× 5.8×
4,097–1,048,576 1.70× 3.9×

Ordered speedup split between the two parts (separate interleaved run including a gather-only build):

TrueU Gather only Gather + bucket sort
1–64 1.63× 1.59×
65–4,096 1.62× 2.01×
4,097–1,048,576 1.28× 1.67×

Unordered compaction gets its whole gain from the gather.

🤖 Generated with Claude Code

…tches

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant