Fresh signed connectors stall awaiting initial relay policy in staging

[callumalpass]

Summary

The guarded beta.92 staging gate establishes an authenticated websocket relay for both exact signed beta.92 and beta.90 clients, receives relay_welcome, and then remains at awaiting initial policy until the bounded acceptance timeout. No relay CRUD assertion is credited. Exact cleanup succeeds and staging rolls back to beta.91.

Evidence

• Reproduced against candidate beta.92 commit 617eece41d5eff7d580cfdae70120868df49d953 and the restored beta.91 staging service.
• Candidate deployment passed immutable live identity, migration safety, entitlement reconciliation, health/readiness, OAuth/application registration, R2, and synthetic checks before the signed-client gate.
• Candidate and previous websocket requests remained open for approximately 79s and 83s respectively, ending only when acceptance cleanup shut down the exact leased daemon.
• Client logs contain connected to cloud relay; awaiting initial policy; no policy becomes usable and state never becomes connected.
• Connector, grant, fixture, daemon, browser session, and profile cleanup passed; guarded rollback restored beta.91 commit ea5a1f075a47cbb427cc64a77a85233902efdd5d.

Ruled out

• locked OS keyring
• ambient partial cloud configuration
• installed-daemon loopback port collision
• Unix socket SUN_LEN overflow
• daemon cleanup/logout port reuse
• CLI status response-shape mismatch
• stale candidate service identity

PR #362 bounded the advisory NATS replacement flush because it could block policy startup, but genuine qualification still reproduced after the fix. The next diagnosis should instrument or bound the remaining pre-policy path (session.policy.start() / buildPolicySnapshot() and exact generation checks) without logging connector IDs, grants, record bodies, or credentials, and add a PostgreSQL/NATS regression matching a fresh connector with no pre-existing grant.

[callumalpass]

Instrumented staging reproduction update (privacy-safe):

• Diagnostic PRs Trace delayed connector policy stages #365 (1f502196) and Trace fast connector policy failures #366 (453b5066) passed full PR and merge-group qualification.
• Exact signed beta.92 and beta.90 clients each reproduced the ~80–90 second readiness failure under the candidate deployment.
• No delayed-stage marker and no fast-failure marker occurred for replacement broadcast, initial policy, generation checks, PostgreSQL checkout/transaction/update/inventory/commit, snapshot build, policy delivery/ack, or post-ack generation validation.
• Therefore the instrumented server policy path settled successfully in under two seconds. In particular, policy_delivery_ack did not delay or fail, so the server received a valid exact policy_applied acknowledgement.
• The remaining fault is downstream of that acknowledgement: the signed client does not remain/report top-level state: connected, most likely due to a post-ack disconnect/reconnect or client readiness-state transition/status-observation fault rather than snapshot construction or delivery.
• Both attempts performed exact cleanup and guarded rollback. Staging is healthy again on beta.91 ea5a1f0; production was untouched.

Next diagnosis should instrument only privacy-safe client milestones and websocket close classification around policy-applied -> Connected -> session lifetime, rather than adding further DB/NATS probes.

[callumalpass]

Root cause reproduced with exact signed beta.92 and beta.90 clients: both reached relay readiness, then whoami failed decoding /v1/connectors/control with invalid type: null, expected a string. The dedicated password account stores its address in email_identities while users.email is null; connector control omitted that table and emitted account.user_email: null, incompatible with both beta clients. PR #368 joins the active primary password identity, preserves beta string fields, and adds a permanent password-account snapshot regression. Focused server tests (409 pass), typecheck, architecture, and adversarial review pass; full CI is running.

[callumalpass]

Resolved and released. PR #368 fixed password-account connector snapshots; exact signed beta.92 and beta.90 clients then passed authenticated relay create/read/update, post-revocation read/create denial, and exact cleanup in staging. Staging acceptance was recorded in cloud-ops PR #320, soak run 33374765423 passed, beta.92 was tagged and published, and protected production promotion run 33378502966 plus independent production verification passed. Production is live on commit 48ba492.