Fix snapshot recovery for component initializer host calls - #3856
Merged
Merged
Conversation
Amp-Thread-ID: https://ampcode.com/threads/T-01a08615-324b-76fb-883a-9fa3472a5e3a Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108 Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108
Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108 Co-authored-by: Amp <amp@ampcode.com>
✅ Deploy Preview for golemcloud canceled.
|
…nto fix/snapshot-recovery-pr
Amp-Thread-ID: https://ampcode.com/threads/T-01a08ad5-52d3-773d-92fc-a775a25c119f Co-authored-by: Amp <amp@ampcode.com>
…nto fix/snapshot-recovery-pr
…pshot Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108 Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a08ad5-52d3-773d-92fc-a775a25c119f Co-authored-by: Amp <amp@ampcode.com>
…nto fix/snapshot-recovery-pr
Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108 Co-authored-by: Amp <amp@ampcode.com>
noise64
approved these changes
Sep 11, 2026
Amp-Thread-ID: https://ampcode.com/threads/T-01a08a68-f1d0-7572-8122-6ea73577b108 Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix
Core initializers execute before
load-snapshot, after the executor has already skipped the history replaced by the snapshot. An initializer clock read therefore attempts to replay a call whose original record was skipped and fails component instantiation.Apply the existing snapshot durability suppression during primary component instantiation when recovering from a snapshot. End suppression before preparation and replay of the remaining invocation history. Fresh creation and full replay retain their existing behavior.
Regression evidence
The Rust reactor
_initializecallsInstant::nowduring core instantiation (verified in compiled WASM, not a lazy constructor). The test leaves increment 11 outside the last snapshot, asserts the snapshot position did not change, then requires successful snapshot loading and recovery of value 11._initializewithexpected recorded monotonic_clock::now Start ... continuation ... valid only for an incomplete entity(1.254s).Dependency and validation scope
Clock prerequisite #3855 is merged. This branch includes that prerequisite. Its current diff against main contains seven files: the original three-file snapshot fix, three test files corrected for fixture-search expectations and asynchronous snapshot timing, and debugger test-client error diagnostics. No Go code is included. This snapshot PR remains unmerged.
The initial CI run failed 87 tests: 86 reported
cannot block a synchronous task before returning, including the core-start clock fixture, because the clock prerequisite was absent. The remaining HTTP poll-loop timeout occurred when the test attempted interruption in the idle gap between construction andstart_polling; its readiness correction is owned by #3855. The genuine reactor initializer and explicit unsnapshotted increment-11 regression remain unchanged.After including the clock readiness follow-up, the exact combined branch passed all eight poll-loop tests plus eight snapshot/recovery tests: 16 passed, 0 failed (43.522s). Package formatting and diff checks also passed. CI is validating the pushed combination.
That CI run resolved the original failures but exposed three oplog-search assertions expecting two imported calls rather than three (the additional call is the core initializer clock), plus an immediate snapshot-count assertion racing queued snapshot publication. Expectations now include the clock call, and the hot-update test waits with a 30-second bound while retaining exact count and recovered-revision assertions. The worker search, hot-update, and original tail regression passed locally (3/3); hot-update passed five additional runs. PostgreSQL/SQLite variants are validated by CI. Formatting and diff checks passed.
Executor all-targets Clippy with warnings denied passed before the final test-only strengthening. Revised tests, formatting, fixture validation, fixture Clippy, and diff checks passed. Full suites remain for CI. The independent Go SDK fresh-instance restoration correction is preserved separately and is not proposed for main.
CI run 34471956153 passed all worker groups, CLI streaming, and PostgreSQL/SQLite search tests. Its only failure was an intermittent debugger connection closing, reported as
Stream ended unexpectedly. Local investigation passed 41 isolated tests and 90 tests in concurrent debugger runs. The test client now preserves WebSocket close reasons and transport errors; no retry or relaxed assertion was added, and the underlying intermittent cause is not claimed fixed.Final CI: run 34475464292 passed on the latest head. All 66 checks are complete: 53 successful, 10 intentionally skipped, and 3 neutral; the Netlify status also succeeded. The previously failing debugger test passed, and the group-5 report contains 525 passed, 0 failed tests. No checks remain pending or failed.
Investigation and evidence review: https://ampcode.com/threads/T-01a08615-324b-76fb-883a-9fa3472a5e3a