[leerho]

This adds docs/count-min-format.md, the cross-language Count-Min serialization format, version 2, and links it from the README.

The design was agreed by lazy consensus in #18, which closed on 2026-10-25. The full spec was posted there on 2026-11-02, and no objections were raised by the 2026-11-09 deadline.

Contents

• The byte layout is unchanged from version 1, except for serial version byte 2.
• Row seeds: rowSeed[i] = XXH64(LE64(i), seed = baseSeed).
• Item encoding, defined byte for byte, for integers, UTF-8 strings and byte arrays.
• Item hashing and bucket mapping: bucket = (XXH64(itemBytes, rowSeed[i]) >>> 1) mod numBuckets.
• Operations, and how readers handle version 1 images.
• Test vectors: XXH64, row seeds, item hashes and buckets, and complete images.
• Cross-language test files (section 8): the count_min_v2_<case>_<language>.sk names, the contents of each case, and the checks a reader performs. Version 2 is deterministic, so a case's file must be byte-identical in every language.

Changes from the version posted in #18

• Adds a status line and the ASF license header.

• Splits section 4 (item encoding) into subsections.

• Adds placeholders for the input rules shared by every family, which are under discussion in Cross-language hashing: inconsistent empty-byte and signed-zero handling #20:

  • 4.4: null and empty items;
  • 4.5: floating-point items, covering signed zero and NaN canonicalization;
  • 4.6: other types.

  Each placeholder states what version 2 requires today.

• Notes that Java's historical UTF-16 APIs don't apply to Count-Min.

• Adds section 8, on cross-language test files, which wasn't in the posted version.

Sections 1 to 3 and 5 to 7, including every test vector, match the posted text.

Implementations

• Rust: merged in fix(countmin): use version 2 hashing for new sketches datasketches-rust#299, expected in 0.6.1.
• Go: planned for 0.4.0.
• Java and C++: to follow, against this spec.

🤖 Generated with Claude Code