[venkat1701]

This adds the HStore paper: one document that covers the design of the engine and its evaluation against HyperGraphDB. The PDF is at docs/hstore-paper.pdf (14 pages, ACM sigconf layout).

What is in it

• Data model, then the engine section by section: counted and summarized copy-on-write trees, the tree-of-trees catalog and the reverse incidence index, node images and the page directory, logged keys and checkpoints, recovery, and compaction through the page directory.
• Implementation and testing, including the crash points and the power-loss tests from fix(engine): power-loss crash tests and the five durability bugs they found #98.
• Evaluation against HyperGraphDB at scale 1 (async and sync commits) and scale 4 (data larger than both caches), with throughput, latency percentiles, bytes written, allocation, heap, disk space through deletes and reclamation, and crash recovery.
• Before and after runs for three design choices: logged keys, inline member trees and records decoded on demand. The costs are reported next to the gains.
• Related work, and a conclusion with the limitations we know of.

How the numbers get into the text

Nothing is typed in by hand. paper/R/results.R reads the JSON files that Comparison run writes, and produces the plots, the results table and a \M{...} macro for every number the text quotes. The architecture diagrams are drawn in R as well (paper/R/diagrams.R), in the same fonts as the text. paper/build.sh runs R and tectonic and copies the PDF to docs/. docs/development.md explains how to rebuild it.

Where the results come from

All runs used one Azure Standard_D8ds_v5 VM (Xeon Platinum 8370C, 8 vCPUs, 32 GiB, local SSD, Ubuntu 24.04, GraalVM 25.0.4), built from main at 2eb85c6. The VM was deleted afterwards.

	runs per system
scale 1, async	5
scale 1, sync	5
scale 4, async	3
each ablation pair, HStore only	3

All 44 runs exited cleanly, and every query checksum matched between the two systems. The raw files are under paper/results and paper/ablation; paper/results/environment.txt has the machine details.

Findings worth a look

• In one of five HyperGraphDB runs at scale 1, and in every run at scale 4, JE's cleaner took about 13 minutes to reclaim space after the deletes. The paper reports this rather than dropping the run.
• HStore's median single-commit latency under async durability at scale 1 is higher than HyperGraphDB's, though its tail is much lower.
• At scale 4 the slowest single commit in a run is slower on HStore, and decoding leaf records on demand roughly halved incidence lookups from eight threads. Both are listed as limitations and still need to be traced.
• HStore keeps more heap after ingest than HyperGraphDB.

README

The README now has figures, all drawn by the same R code as the paper and written as SVG to docs/images by paper/build.sh:

• the example hypergraph under the introduction (also Figure 1 of the paper);
• a new "How it works" section with the engine overview, the tree-of-trees layout, recovery and compaction diagrams;
• the benchmark ratio and resource plots in the performance section.

The performance section now quotes the Azure runs instead of the earlier laptop runs, and says where HStore is not ahead. The earlier claim that HyperGraphDB opens a crashed store faster no longer holds and is gone.