Repository navigation
Add durable remote-step permits and fenced notification recovery - #123
Merged
Merged
Conversation
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| 🔵 In progress View logs |
anchorage-showcase-single-tenant | e8e68de | Oct 16 2026, 05:47 AM |
Use the pinned tool resume schema after exact suspension admission so malformed results cannot advance a durable agent. A fresh-host regression failed before this check and passes afterward.
Persist exact suspension identity and request, serialize permit issue/consume with snapshot and fence state, and expose bounded inventory. Host dispatch and result continuation remain disabled pending permit hardening and live proof.
Refuse pending cancellation, bind effects to the current request and tightened deadline, and keep consumed permits closed until a trusted executor completion contract exists.
Accept the new FlowSafe work category in Fleet scenario evidence and keep gate and inventory tests aligned with the expanded drain surface.
Return opaque authority only to the consume winner, stage bounded exact-byte results after trusted completion proof, and retain unresolved work for drain. Host continuation remains separate.
Store verified results as drain work, remove unproven settlement, and exercise concurrent permit consumption. A later host continuation must prove this result advanced the run.
Recognize the new work category and describe FV2 as seeded-row visibility. Full migration drain still requires an empty inventory across the recovery interval.
Route retained results through the stored principal and exact suspension on the internal thread host. Keep wiring optional; schema upgrade, resource binding, and durable settlement remain release gates.
The resumed proof performs two inventory sweeps. Each now reads the remote-step work category, so the request-count guard expects two additional reads.
Persist the started resource across topology, host, runtime, journal and recovery. Require nonidentity and legacy refusal proof on real workerd before broker dispatch is enabled.
The host forwards the stored principal kind and rejects reserved timeout data before runtime resume. Approval resume keeps its human default; focused and package suites pass.
Document the host reader's referenced types and parse source once in the fence matrix census so the full suite fits its existing timeout under load.
Exercise D1 journal persistence and exact host continuation after recreating the host. The runtime resume itself remains covered by the pinned-core probe.
Upgrade the known prior journal layout transactionally, preserve permit/result records, and refuse unknown or partial schemas before remote-step authority is used.
Require the suspension key constraint and permit default before accepting legacy rows. Recheck the completed schema after concurrent migration races.
Persist the host's start resource and require journal addresses to match that provenance. Ambiguous legacy starts refuse broker authority; host topology support follows separately.
Resolve trusted resource identity for host starts and callbacks. Reject resumed and recovered runs under a different persisted start resource; fail closed for ambiguous legacy mappings. Focused host tests, Biome, and test typecheck pass.
Version the journal, preserve known rows, index active results, and bind witnessed settlement to the exact resume owner. Expired uncertain resumes stay fail-closed across overlapping isolates; durable replay requires a separate effect-fencing protocol.
Keep general Mastra continuation replay blocked; specify a host-owned restricted profile only with durable effect receipts, generation-fenced snapshots, and real cross-isolate proof. The plan records the pinned runtime limitation.
Refuse a retained result when the current trusted resource mapping differs from the journaled start resource. Exercise nonidentity mapping and changed-binding refusal across host rehydration.
Use the existing local spike to prove permit/result retention, resource refusal and operator drain inventory after a fresh process. The fixture uses a synthetic runtime and does not claim Mastra continuation.
The restart proof now fails when the workflow snapshot does not advance, even if the separate fixture status says success.
Bind host-owned notification enqueues to an immutable request digest and a D1 transaction receipt. Replays return the first effect without overwriting the inbox.
Bind issue and winner-only consume to stored principal, resource, deployment and invocation identity. Keep transport and general runtime continuation closed.
Expose a pre-route host helper that reconstructs the persisted principal with the configured approval and service wiring. Reject mismatched invocation identity before dispatch.
Mount the spike remote-step fixture only for its proof generation and check consumed-permit and retained-result replay after a fresh workerd process starts.
Keep Fleet Control removed from Anchorage. Its inventory-count edits move to ASP when the new FlowSafe category is repinned; integrated FlowSafe tests and architecture checks pass.
Persist a versioned recovery cursor and call the authenticated target thread through the singleton maintenance duty. Require an armed duty before selected effect admission and bound local alarm work across wakes.
Move shared type declarations below notification dispatch so the remote-step journal no longer reaches host-kit through a type-only import.
The workerd fixture now checks the persisted request digest and replay response alongside the receipt identity, matching the v5 notification contract.
The documentation reachability check now treats docs/plans as internal design records while still reporting unlinked public guides.
Add an isolated default-off subject with dedicated D1 and thread DO, plus an authenticated loopback service-binding probe. Keep real effect proof gated on a guarded suspension producer.
Keep the isolated Worker variable aligned with the D1 sentinel identity validator before provisioning or deployment.
Use a fixed system-owned agent suspension and no-op dispatch to exercise the selected notification path. Keep the feature off; local workerd proof covers one receipt and continuation.
Require the local staging credential before forwarding bounded fence and maintenance requests with a distinct admin secret over the private Service binding.
Require v5 empty-journal preflight and exact D1 effect receipts. Relock a controller-owned fence on failure and keep live execution closed until the owning deployment lock is available.
Keep bounded exit, signal, process error and stderr detail in the controller cause so a failed staging rollback can be diagnosed without printing credentials.
Run the local controller and detached watchdog with a loopback probe and fake Wrangler, then kill the controller and verify owned fence relock and off deployment.
Share an exclusive Worker lock across controller and watchdog. Seal the fence against delayed unlock, converge lost relock responses, and let the watchdog alone confirm the off deployment.
Name the local node:test files outside Vitest’s test glob so the package suite does not collect files without Vitest suites.
Export the type-only references through their owning subpaths so TypeDoc can resolve the public signatures without changing runtime exports.
Avoid a second D1 getter read while retaining the stored principal, deployment, resource and run-access checks on the same invocation.
Share agent/workflow lifecycle checks and avoid duplicate notification reads while preserving fresh post-write CAS and result checks.
Register one receipt-writing prefix and limit per deployment, preserving ordinary prefixed storage and refusing ambiguous retained histories.
Race two Durable Objects at scope registration and assert one winner with no losing receipt or notification.
Keep the inventory census exact while registering the prefix and limit only on first receipt admission.
Treat a confirmed missing receipt table as an empty initial state while requiring the real table and receipt after the selected effect.
Alias the unused Cloudflare REST constructor to a fail-closed staging stub so binding-backed D1 proof can start within the Worker budget.
Accept lazily absent notification tables before the selected effect, and allow one exact monotonic retry from a verified off 1/1 seal with watchdog rollback.
Validate the enabled Worker, D1, tenant and authenticated probe before opening the fence. Prepare a private off bundle so watchdog rollback survives checkout changes; retain the old sealed resource.
Match the live fence transition in the native proof and pass the version-pinned epoch to start and completion. Attempt exact-version off recovery when the probe fails and confirm the locked fence through D1.
Reduce bundled startup parsing work while retaining Wrangler name preservation. Native epoch-fenced notification proof passes with the minified build.
gcharang
changed the title
gcharang
marked this pull request as ready for review
October 16, 2026 05:48
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a durable remote-step boundary for trusted FlowSafe hosts. A host can register a suspension, issue and consume a one-time dispatch permit, retain the result, and reconcile after a process restart. Principal, stored-start, generation, deadline and execution-fence checks bind authority to the original invocation.
The first automatic recovery profile supports the host-owned notification operation with an atomic idempotency receipt and a configurable retained-receipt cap. General Mastra continuation remains fail-closed when safe replay cannot be established. The versioned D1 journal migration preserves supported records and refuses ambiguous legacy state.
The implementation includes the exported private workflow route, packed consumer coverage, native workerd restart and competing-Durable-Object proofs, and isolated staging controls with version-pinned mutation epochs and watched rollback. ASP still owns customer authentication, signed grants, WFP transport, publication and composed launch proof.
Validation:
A minor changeset prepares FlowSafe 0.29.0. The versioned release will run the maintainer's full local gate before promotion; ASP repin and customer launch remain downstream work.