DDIR: preserve timestamp order across Corgi chunk boundaries - #862
Merged
Conversation
This was referenced Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Corgi chunk merger added in #861 can emit timestamps out of order when an
equal
(key, value)group continues across an input chunk boundary.For identical keys/values (brackets are chunk boundaries):
survey_groupsgroups by key/value, not timestamp. TheBothpath mergestimestamps until one side's range ends, then copies the other range's suffix.
If the exhausted range also ends its whole input chunk, that suffix must wait
for the next chunk: it can contain timestamps beyond the shared merge horizon.
Stop at that boundary and return the unconsumed suffix through the existing
push-back path. The batched key/value survey is unchanged. Eight implementation
lines; no dependency, batch-size, query, or server-protocol changes.
The regression uses the plain ingest
ChunkMerger. A later traceadvancecan re-sort equal-value timestamps and hide the violation, so checking only
that path was insufficient. The test covers the small example and an opt-in
version with a
TARGET-sized first chunk. The unpatched implementation wasobserved to fail the small reproduction.
Validation: both cases pass on this branch with the existing public Corgi pin:
CARGO_PROFILE_RELEASE_DEBUG=0 cargo test --locked --offline --release \ -p interactive --test corgi_chunk_merge -- --include-ignoredThis fixes a sorted/consolidated merge-contract violation. It does not claim
to explain an LDBC performance result or a particular wrong query answer.