sync: a run's heartbeat never parks a conflict - #270
Merged
Merged
Conversation
The test waited for the file to exist and then read it, but the shell creates
the file before it writes into it, so a loaded machine read it empty
('expected room=design-talk but was <>'). The child now writes beside the
file and renames it into place: existing means complete.
last_activity_at is stamped without a revision, so an owning node's live row runs ahead of what main last heard. When main also holds a stamp the node never recorded as its base (a push main committed whose acknowledgement the node lost), both sides have moved the same field and the round parked a conflict naming last_activity_at alone. A node that had days of failing rounds met 22 of them on its first round after upgrading. A store now declares its latest-wins fields: instant-valued fields that only move forward. When both sides moved one, the detector merges to the later instant instead of reporting a conflict; a real conflict beside it still surfaces. RunStore declares last_activity_at. Adopting an entity's settled state also closes whatever conflict is still open on it. The disagreement no longer exists, and resolving it later would write its detection-time snapshot over the settled row — so a node that upgrades with such conflicts parked heals on its next round.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Field report
After upgrading to v0.44.1, Mady's first
sail syncparked 22 conflicts, every one naming a single field:last_activity_at.Root cause
last_activity_atis a run's heartbeat.RunStore.stampActivitywrites it without a revision (by design — a revision per stamp would flood the change log), so an owning node's live row always runs ahead of what main last heard. That alone is harmless: only local moved → push.It becomes a conflict when main also holds a stamp the node never recorded as its base — a push main committed whose acknowledgement the node lost (a round that died after its push). Base says
T0, main saysT1, the live row saysT2: both sides moved the same field to different values →Conflict[last_activity_at]. Mady's node had days of failing rounds before v0.44; it met 22 such runs on its first good round.Reproduced in
RunSyncTest.aHeartbeatThatMovedOnBothSidesTakesTheLaterStampInsteadOfParkingAConflict(fails on main with exactlyfields=[last_activity_at]).Fix
SyncedStoredeclares instant-valued fields that only move forward (latestWinsFields(), none by default;RunStoredeclareslast_activity_at). When both sides moved one,ConflictDetectormerges to the laterInstant(parsed, so…:41Zvs…:41.5Zcompares correctly). The heartbeat still travels, so presence on main stays fresh. A real conflict beside it (e.g.status) still surfaces.StoreReplica.adoptnow closes whatever conflict is still open on the entity it just settled. This matters beyond tidiness:sail conflicts resolvewrites the detection-time snapshot, so a parked conflict on an entity that has since reconciled cleanly is a loaded gun. A node that upgrades with these 22 parked heals on its next round.Also
SessionVerbsTestfailed once in the full run: it waited for a file to exist and then read it, but the shell creates the file before writing into it (expected room=design-talk but was <>). The child now writes beside the file and renames into place. Pre-existing, unrelated, fixed structurally.Verification
mvn clean verify: 5,266 tests, all coverage gates met.