Skip to content

Implement persistent deletion ledger and enhance file sync protocol - #20

Merged
guydols merged 6 commits into
mainfrom
dev
Sep 10, 2026
Merged

guydols merged 6 commits into
mainfrom
dev

Conversation

@guydols

@guydols guydols commented Sep 10, 2026

Copy link
Copy Markdown
Owner

This pull request introduces significant improvements to the deletion and resurrection handling in the file synchronization protocol, ensuring that file deletions are robustly tracked and propagated between client and server. The changes implement a symmetric deletion-ledger exchange, improve tombstone handling to prevent unintended file resurrection, and update message formats and internal methods to support richer metadata for deletes. Some test code was also removed. The most important changes are:

Deletion Ledger Synchronization and Protocol Enhancements:

  • Added a symmetric deletion-ledger exchange step to the sync protocol, where the client and server exchange and merge their deletion ledgers (LedgerExchange), ensuring both sides have a consistent view of deleted files and preventing missed deletes from being overwritten. (crates/filesync/src/client.rs)
  • Updated the Delete message to include deleted_at_ms and deleter metadata, and updated all send/receive paths to use and handle this richer metadata, allowing precise tracking of when and by whom a file was deleted. (crates/filesync/src/client.rs, crates/filesync/src/common.rs) [1] [2] [3]

Tombstone and Resurrection Handling:

  • Implemented logic to filter out files that were deleted (tombstoned) locally but still exist on the peer, ensuring that tombstones take precedence and prevent accidental resurrection of deleted files. Legitimate file recreations clear their tombstones after upload. (crates/filesync/src/client.rs) [1] [2]
  • Added methods to compute and push deletes that the peer missed while offline, preserving the original deletion timestamp and deleter information. (crates/filesync/src/client.rs)

Internal API and Code Cleanups:

  • Added PendingChanges::take_deletes_with_engine to ensure tombstones are recorded before sending deletes, and updated the flush path to use this method. (crates/filesync/src/common.rs, crates/filesync/src/client.rs) [1] [2]
  • Removed in-file test modules and some explanatory comments for clarity and maintainability. (crates/filesync/src/client.rs)

These changes greatly improve the safety and reliability of file deletion propagation, preventing data loss from missed deletes and ensuring proper conflict resolution in the presence of offline edits and file resurrection scenarios.

guydols added 6 commits July 6, 2026 00:03
Offline clients re-uploaded files deleted while they were away because
deletes only lived in memory. The ledger in .bh_filesync/deletion-ledger.json
keeps tombstones across restarts so deletes survive reconnects.
Message::Delete now carries deleted_at_ms and deleter plus LedgerExchange
so peers can compare delete time versus file mtime. Clean break, no compat
shim.
apply_deletes now persists a tombstone even when the file is already
absent, and filter_resurrected blocks re-upload when the remote
tombstone is newer than local mtime. Legitimate recreates with newer
mtime still win and clear the tombstone.
After ManifestExchange both peers swap LedgerExchange, merge with
last-writer-wins, delete locally-resurrected files with the original
stamp, and push deletes the offline peer missed. Live deletes preserve
the sender timestamp on forward.
Covers the offline-delete-stays-deleted repro, recreate-wins, ledger
persist/prune/TTL, corrupt-backup handling, and v8 wire roundtrips.
@guydols
guydols merged commit ad944cb into main Sep 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant