Skip to content

test: align eval host startup and cleanup deadlines - #64

Merged
vriesd merged 1 commit into
mainfrom
fix/eval-host-startup-budget
Sep 6, 2026
Merged

test: align eval host startup and cleanup deadlines#64
vriesd merged 1 commit into
mainfrom
fix/eval-host-startup-budget

Conversation

@vriesd

@vriesd vriesd commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Give the real-host integration test a 240-second envelope around the existing 180-second startup contract, retaining the actual native tuple assertion.
  • Route startup timeout through cancellation and preserve environment/scratch cleanup.
  • Cover cancellation during a pending readiness POST with credential-preservation assertions.

Validation

  • Independent Flow final review passed for the bounded repair.
  • Fresh pinned Linux, no credentials and no warm-up: 821 passed, 1 skipped; host test completed in 151 seconds, exceeding its former 30-second limit.
  • Linux replay 13/13, audit and live smoke passed.
  • Focused host/reporting tests: 106 passed.
  • Clean-worktree contribution preflight: 821 passed, 1 skipped; typecheck, lint, metadata and build passed.
  • Canonical grader remains unchanged by this test-only repair. No runtime, package, version or qualification-policy changes.
  • Optional gitleaks unavailable. Historical campaign/canary files and local diagnostic helpers are excluded.

This PR does not claim release qualification or authorize publication.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T00:27:01.777825Z 6d32cad PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@vriesd
vriesd merged commit a617de5 into main Sep 6, 2026
6 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d32cad61f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

files: { "package.json": '{"name":"eval-host-config-test"}\n' },
reviewer: { model: "provider/reviewer", steps: 80 },
// Let startup own cancellation/cleanup before the outer test expires.
signal: AbortSignal.timeout(180_000),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the host's full startup window

On a slow filesystem, this signal can cancel an otherwise valid startup early: its 180-second clock begins when EvalHost.start is invoked, but the harness does not begin its own STARTUP_TIMEOUT_MS deadline until after credential copying, fixture creation, synchronous Git commands, package-cache copying, and process spawning. Consequently, that setup time is subtracted from the host's intended 180-second health/readiness allowance, making this integration test flaky in environments where startup legitimately approaches the internal deadline; give the cancellation signal additional headroom while keeping it below the 240-second outer timeout.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants