Skip to content

fix(runtime): persist replay claims and preserve heartbeat cadence - #7969

Merged
bokelley merged 4 commits into
mainfrom
bokelley/fix-runtime-db-reliability
Oct 9, 2026
Merged

bokelley merged 4 commits into
mainfrom
bokelley/fix-runtime-db-reliability

Conversation

@bokelley

@bokelley bokelley commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Production training-agent replay claims can otherwise select an in-memory backend before the database pool initializes, and owner compliance tests can re-arm scheduled heartbeat checks. Select the PostgreSQL replay backend in production before initialization and obtain the pool when a query runs; fail closed if it is unavailable. Publish owner-test verdicts while preserving heartbeat cadence, next scheduled checks, requeues and streaks. New owner-only records have no heartbeat schedule.

This is the existing #7969, refreshed onto main after #7968 merged. Its own four-file correction is preserved; current main supplies the inherited OAuth issuer fields and fixtures. It includes the scheduling fix from #7691, which remains a fallback until this replacement lands. No GCS reporting or 3.3 protocol changes are included.

Validation on 08dd3de94b17fa1bed131dd18559d5359e6f68f1 with current-main SDK15 dependencies:

  • Both real PostgreSQL15.18 integration suites passed all24cases with no skips, including production cold boot, durable replay claims and owner/heartbeat publication.
  • Full normal precommit passed:1106root tests;741server unit files with12396passes and31existing skips; all guards and both typecheck stages. No assertion or timeout budget was weakened.
  • The local current-source signals storyboard tenant passed54clean cases and123steps. The slower sales run was stopped with no observed failures; complete current and stock3.0.28 local advisory matrices were explicitly deferred to authoritative CI under the playbook Local Hooks guidance. The normal push passed version synchronization and scope checks using only the documented advisory-matrix skip.
  • Remote storyboard/build/security checks and fresh review remain merge gates; local partial matrices are not claimed as complete.

Addresses #7899 and #7355 in part. A source merge does not establish fleet deployment, seller recovery or completion of the remaining operator verification. Those issue-level checks remain open.

@bokelley
bokelley force-pushed the bokelley/fix-runtime-db-reliability branch from 6805ccf to 9be693b Compare October 6, 2026 14:22

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean, surgical runtime-reliability fix with no blocking findings.

Checked: the PR touches only server/** (compliance-db.ts, idempotency.ts, and two integration tests); no protocol surface (static/schemas/source/**, docs/reference/**, dist/**, task/compliance assets) is touched, so changeset / schema-drift / oneOf / immutability / 3.1.x-patch gates do not apply. No migrations under server/src/db/migrations/** are added.

  • compliance-db.ts: schedule_next CTE now gates cadence advancement on triggered_by ($7); owner_test runs INSERT next_compliance_check_at=NULL and skip the ON CONFLICT UPDATE, preserving requeued_at/streak/schedule. Parameter numbering consistent; CTE only reached for authoritative runs. NULL reads as due in getAgentsDueForCheck, matching tests.
  • idempotency.ts: lazy {query} wrapper defers getPool() so production fails closed (throws until DB init) rather than downgrading to memory. Verified by the new cold-boot integration test across the claim/save/renew/release/fencing lifecycle.

Gate check: high_risk false, gated_paths false, no no-auto-approve team match, no prior decision. Reviewer reports no critical/high/medium findings. No decision-table trigger (rows 1–8) fires → row 9 approve.

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Oct 6, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean delta with no blocking findings.

This PR adds two server-side fixes, both covered by new integration tests:

  • compliance-db.ts: owner_test runs publish the verdict while leaving the independent heartbeat schedule and pending requeue untouched (#7680), via a CASE on INSERT and a WHERE guard on ON CONFLICT DO UPDATE; NULL next_compliance_check_at correctly reads as due.
  • idempotency.ts: production now builds the fenced PG backend with a lazy getPool() resolver instead of caching a process-local memory backend before initializeDatabase(), failing closed until DB startup.

No published protocol surface (static/schemas/source/**, normative docs, release artifacts) touched, so no changeset required. No dist/ artifact mutation, no schema/docs drift, no oneOf regression. Server-only change.

Gate checks: no critical/high/medium findings (row 1 N/A); gated_paths false and review_decision APPROVED anyway (row 2 N/A); high_risk false (rows 3/5 N/A); no escalation-category mediums (row 4 N/A); prior decision was approve (row 6 N/A); no no-auto-approve team match (row 7 N/A); zero medium findings (row 8 N/A). Falls through to row 9 → approve.

Base automatically changed from bokelley/bootstrap-v9 to main October 9, 2026 09:04
@bokelley
bokelley dismissed aao-secretariat[bot]’s stale review October 9, 2026 09:04

The base branch was changed.

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approval: no reviewable delta since the prior approval; re-submitting after stale-review dismissal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant