Skip to content

Add InsertResult::SuccessUpdate - #1449

Open
Jack Moffitt (metajack) wants to merge 3 commits into
mainfrom
push-zqtpsmqqoxky
Open

Jack Moffitt (metajack) wants to merge 3 commits into
mainfrom
push-zqtpsmqqoxky

Conversation

@metajack

Copy link
Copy Markdown
Contributor

Garnet protocol requires different results for VADD depending on whether it was an insert or an update. This adds an InsertResult::SuccessUpdate variant to flag that case. Also, it turned out updates weren't actually implemented at all, so this also fixes that issue.

Because quantization backfill may interact with updates, a new range mutex is introduced to allow quantization backfill and updates to lock ranges for changes so they don't stomp on each other.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Update detection can corrupt ID mappings after failed deletion and remains unsafe for concurrent inserts of the same external ID.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds vector-update support to the Garnet provider and exposes updates through the FFI result contract.

Changes:

  • Reuses internal IDs for updates and adds SuccessUpdate.
  • Coordinates quantization backfill with vector updates.
  • Adds update/backfill tests, documentation, and a major version bump.
File Description
Cargo.lock Updates the crate version.
diskann-garnet/​Cargo.toml Bumps version to 6.0.0.
diskann-garnet/​diskann-garnet.nuspec Aligns NuGet package version.
diskann-garnet/​docs/​ffi-design.rs Documents update result semantics.
diskann-garnet/​src/​ffi_recall_tests.rs Simplifies unaligned ID decoding.
diskann-garnet/​src/​ffi_tests.rs Tests update results and ID reuse.
diskann-garnet/​src/​fsm.rs Adds guards for existing IDs.
diskann-garnet/​src/​garnet.rs Tracks update state in contexts.
diskann-garnet/​src/​lib.rs Exposes SuccessUpdate through FFI.
diskann-garnet/​src/​provider.rs Implements updates and range synchronization.
diskann-garnet/​src/​test_utils.rs Tracks internal-map reads in tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread diskann-garnet/src/provider.rs
Comment thread diskann-garnet/src/provider.rs
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.01087% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.60%. Comparing base (fcf9053) to head (e5cf957).

Files with missing lines Patch % Lines
diskann-garnet/src/garnet.rs 52.17% 11 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1449      +/-   ##
==========================================
+ Coverage   90.58%   90.60%   +0.01%     
==========================================
  Files         568      568              
  Lines      112680   113023     +343     
==========================================
+ Hits       102069   102401     +332     
- Misses      10611    10622      +11     
Flag Coverage Δ
miri 90.60% <97.01%> (+0.01%) ⬆️
unittests 90.36% <97.01%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-garnet/src/fsm.rs 94.52% <100.00%> (+0.07%) ⬆️
diskann-garnet/src/lib.rs 94.39% <100.00%> (ø)
diskann-garnet/src/provider.rs 82.45% <100.00%> (+3.40%) ⬆️
diskann-garnet/src/test_utils.rs 98.61% <100.00%> (+0.03%) ⬆️
diskann-garnet/src/garnet.rs 93.31% <52.17%> (-2.17%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@harsha-simhadri

Copy link
Copy Markdown
Contributor

IT would be nice to validate this PR on runbooks with replace operations. Thanks.

@metajack

Jack Moffitt (metajack) commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Results from wikipedia-1M using wikipedia-1M_expiration_time_replace_only_runbook.yaml. l_build=300 and maxR=48.

### Recall@100

 Positive differences mean Garnet is higher.

 ┌──────────┬─────────────────────┬───────────────┬──────────────────┬────────────────────────────────────┐
 │ L-search │Baseline             │ Garnet, final │ Final difference │ Mean difference across checkpoints │
 ├──────────┼─────────────────────┼───────────────┼──────────────────┼────────────────────────────────────┤
 │ 100      │ 88.1172%            │ 88.3144%      │ +0.1972 pp       │ +0.2180 pp                         │
 ├──────────┼─────────────────────┼───────────────┼──────────────────┼────────────────────────────────────┤
 │ 192      │ 94.0728%            │ 94.0898%      │ +0.0170 pp       │ +0.0222 pp                         │
 ├──────────┼─────────────────────┼───────────────┼──────────────────┼────────────────────────────────────┤
 │ 300      │ 96.6184%            │ 96.5974%      │ −0.0210 pp       │ −0.0092 pp                         │
 └──────────┴─────────────────────┴───────────────┴──────────────────┴────────────────────────────────────┘

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Distinguish inserts and updates in the insert() FFI call

4 participants