[venkat1701]

Closes #56

Where main stands

The issue measured 54.8 KiB per hyperedge for HStore and 21.6 KiB for HyperGraphDB. The published results at
9291e18 (benchmarks/report/results/s1-async) have ingest.edges at 37.2 KB per hyperedge for HStore and
30.0 KB for HyperGraphDB at scale 1, so HStore still allocated more. At scale 4 HStore was already well ahead,
at 104 KB against 892 KB.

An allocation profile of the ingest (JFR jdk.ObjectAllocationSample) showed that about a third of the bytes
were not on the ingesting thread at all:

• Each change feed subscriber (statistics, materialized views, semantic index) calls replay(acknowledged)
  after every commit. replay starts at the nearest index entry, which can be up to 63 frames back. It read
  and checksummed the payload of each of those frames and then dropped it.

On the ingesting thread the cost was spread across many places. The largest were:

• TreeDiff pushed one Element per leaf entry onto its stack. The change log diffs every changed catalog
  and incidence leaf at commit, and Hyperedge.diff diffs every loaded member tree.
• BulkBuilder started with 64-entry key and value arrays. Hyperedge.build makes two builders per edge,
  and for a SET edge one of them stays empty.
• IncidenceCodec.encode built a stream per column to compute its flags. The feed encodes each member
  change on its own.
• Writer.load took a List<Long>. For every member it rechecked write permission, decoded the edge and
  its type, streamed over the roles and the weight, and interned the role set.

Changes

• ChangeFeed.FrameReader takes the generation to skip through. It steps over the payload of an older
  frame after reading its 16-byte header, and it reuses one header buffer. restore() and
  truncateAfter() still read and verify every frame.
• TreeDiff puts a whole leaf on its stack as one Run (the leaf and a position). The deltas and their
  order are unchanged.
• BulkBuilder arrays start empty, grow to 8 entries, then double.
• IncidenceCodec.encode finds the present columns with a loop.
• New Writer.load(long edge, long[] members, MemberSpec spec). It checks the edge and roles once, builds
  the first incidence the way add does, then checks each remaining member and copies the first
  incidence's columns. The List<Long> overload unboxes into an array and calls it. The errors are the
  same as before and come in the same order. Writer.assign checks required properties with a loop.
• The benchmark adapter (HStoreStore) now passes member ids as a long[] through the new public
  overload, the way a user loading from primitive ids would. The other workloads are unchanged.

docs/storage/change-feed.md and the Writer table in docs/database/data-model.md describe the
changes.

Measurements

The numbers come from a small probe, not part of this branch, that runs the
benchmark's ingestNodes and then ingestEdges at scale 1 (100,000 hyperedges, 1,000 per transaction)
through the benchmark's own store adapters. It reports getTotalThreadAllocatedBytes for the whole process,
as the comparison harness does, and the ingesting thread's own bytes. Main is 0c847f7, and every row
uses the same probe.

	all threads, per hyperedge	ingesting thread, per hyperedge	collections
HStore, main (10 runs)	37,711 B (37,003 to 37,964)	24,437 B	3
HStore, this branch (5 runs)	23,650 B (23,475 to 23,823)	16,617 B	3
HyperGraphDB (3 runs)	27,874 B (27,283 to 27,951)	22,638 B	2

The GC count over a 10 s ingest is too coarse to show a
difference.

The effect of each commit is the median of 3 runs per commit, measured all threads:

Commit	all threads	ingesting thread
main	39,210 B	25,939 B
read only the header of a skipped feed frame	32,044 B	25,859 B
walk a leaf by position in TreeDiff	30,628 B	23,357 B
grow bulk builder arrays from 8	29,869 B	22,269 B
incidence column flags without a stream	27,972 B	21,023 B
check the edge once per Writer.load	26,377 B	18,741 B
adapter passes a long[]	24,531 B	17,172 B

Single runs vary by a few KB when the JIT compiles a path differently, so treat the per-commit steps as
approximate. I didn't rerun scale 4.

What is left

These would each need a decision, so they are not in this branch:

• Statistics recomputes its histograms from a full scan of the catalog and the reverse index every 10,000
  changes. A 1,000-edge transaction makes about 5,000 changes, so during a bulk load that is a full scan every
  second transaction, and its cost grows with the square of the load size. Making the threshold a fraction of the store size would fix that, but it changes how stale
  the planner's statistics can get.
• Each of the three feed subscribers decodes every event on its own. Sharing one decoded event, or giving
  Statistics a subscription that only sees the change counts, would change the ChangeFeed API.
• Inline postings copy their value list twice per insert, once into an ArrayList and once in
  List.copyOf, and a written leaf is copied on first write in each transaction.

Tests

./mvnw -q install: engine 185, database 36, server 15, all passing.

• ChangeFeedTest.replayReadsOnlyTheHeaderOfAFrameItSkips: corrupts the payload of frame 2 and checks
  that replay(2) still returns 3, 4 and 5, while replay(0) stops at the bad frame. On main, replay(2)
  returns nothing, so the test fails there.
• TreeTest.diffMatchesAnOracleAcrossRandomEdits (5 seeds): compares TreeDiff with an oracle in both
  directions, against itself, and against an empty tree.
• DatabaseTest.bulkLoadedMembersMatchMembersAddedOneAtATime: members loaded from a long[] or a
  List<Long> match members added one at a time with roles, weight and validity. It also covers an empty
  load, an undeclared role, a missing member in the middle of the array, and a load into a node.

The TreeDiff, BulkBuilder, IncidenceCodec and Writer changes only reduce allocation, so their tests
check that behaviour is unchanged. They pass on main as well.