Skip to content

fix: job event memory, artifact review gate, e2e port, dev-environment advisories - #326

Merged
dovvnloading merged 4 commits into
mainfrom
fix/small-verified-defects
Oct 17, 2026
Merged

dovvnloading merged 4 commits into
mainfrom
fix/small-verified-defects

Conversation

@dovvnloading

Copy link
Copy Markdown
Owner

Problem

Four small verified defects, one commit each:

  • BE-13 (job event memory).
    • When an attached reader detached, a finished job kept the live-reader headroom: up to 4x the retained-event ceiling, for as long as the job stayed in the registry.
    • While a reader was attached, the byte cap was never consulted. At the defaults that allowed 2048 retained events of up to 1 MiB each.
  • EX-26 (artifact boundary review gate). The gate was weaker than it read:
    • its digest hashed a list of names kept beside the cases, so editing or deleting a case did not change it;
    • a boundary that accepted a hardlink was reported as "blocked" (a host limitation) instead of "failed";
    • --json only changed separators;
    • without --strict, a total failure exited 0.
  • QA-15 (real-API e2e).
    • The real-API Playwright spec used port 8799, the README screenshot server's default, with reuseExistingServer outside CI. Anything left listening there was silently tested as "the real API".
    • Its generation test asserted on the first chat in the list, which depends on earlier runs.
  • QA-24 (dev environment check). scripts/check_dev_environment.py never noticed three setups: system Python with no virtual environment, an interpreter other than the lock target, or a package that shadows a declared one.

Root cause and change

  • BE-13. Eviction ran only inside _append_event, and a terminal job never appends again. The eviction loop is now _trim(record), called after every append and in the finally of events() once the reader's cursor is gone. The live-reader protection now also requires event_bytes <= max_event_bytes * LIVE_READER_EVENT_HEADROOM.
  • EX-26.
    • The digest is now computed from the _CASES tuple that runs: each case's name and source in run order, plus the shared fixture helpers.
    • The status is failed if any case failed, blocked only if a case could not run, and passed only if every case ran and passed. An empty corpus fails.
    • Strict is the default: exit 1 for failed, 2 for blocked. --strict is still accepted, so scripts/check.ps1, the pre-push hook and quality.yml are unchanged.
    • Without --json the tool prints a readable summary naming each failed or blocked case.
    • The twelve-case corpus is unchanged.
  • QA-15.
    • frontend/e2e/e2e-backend.json holds the port (now 8798) and a marker. playwright.config.ts (through e2e/e2eBackend.ts) and scripts/e2e_backend.py both read it.
    • The backend adds an X-Cortex-E2E-Backend header to its liveness route only.
    • real-api.spec.ts checks the marker before each test and refuses a server that lacks it.
    • The generation test sends a prompt unique to the run and asserts on the chat the server accepted it into.
  • QA-24. The check prints advisories for those three cases. Advisories never change the exit code; only pin mismatches fail, as before.

Compatibility and rollback

  • No API or data change.
  • Exit codes. The artifact boundary review now exits 1 instead of 2 on a failed case, and non-zero even without --strict. Every caller passes --strict and checks for non-zero.
  • e2e port. It moves from 8799 to 8798; only the Playwright config and the e2e script use it.
  • Rollback. Each commit reverts on its own.

Checks

Run on Windows, Python 3.14 unless stated.

  • New tests: python -m pytest -q tests/test_job_registry_live_readers.py tests/test_artifact_boundary_review.py tests/test_check_dev_environment.py tests/test_e2e_backend.py gives 20 passed. Same files on Python 3.12 (py -3.12): 20 passed.
  • Against the old code: the same four test files copied onto an export of origin/main (00b76b0) give 16 failed, 4 passed. The 4 that pass on both are guards (for example, a reader that is still attached keeps its unread events).
  • Real run of the dev check: python scripts/check_dev_environment.py on this machine prints three advisories (no venv, 3.14 vs the 3.11 lock, a shadowing py package) and exits 0.
  • Pre-push hook: scripts/check.ps1, quick tier (ruff, actionlint/zizmor, mypy, full backend pytest, artifact boundary review, contract check, tsc, eslint, vitest): all 11 checks passed in 381.2 s.
  • Playwright real-api.spec.ts: not run locally; the CI "Browser tests" job runs it.

Security, concurrency, data loss

  • BE-13 only evicts more eagerly. It still never evicts an attached reader's unread events inside the headroom, and still keeps the newest (terminal) event. Eviction runs under the registry lock, as before.
  • The e2e marker is a fixed string on a loopback test server that holds no real data.
  • The digest includes case source, so it changes when a case changes. Nothing stores or compares it across runs.

Limits

  • Playwright not run locally. The real-API spec relies on CI.
  • Shadowing check scope. It reports top-level names from installed distributions only.

Plan item: BE-13, EX-26, QA-15, QA-24

🤖 Generated with Claude Code

dovvnloading and others added 4 commits October 17, 2026 07:57
…ir bytes

The live-reader headroom lets the event buffer grow past its retention
limits so an attached SSE reader does not lose events it has not read yet.
Two gaps made that headroom outlive its purpose or ignore its budget:

- Eviction ran only inside _append_event. When the reader detached, nothing
  re-trimmed, so a finished job (which never publishes again) kept up to
  max_event_count * LIVE_READER_EVENT_HEADROOM events for as long as it
  stayed in the registry.
- The protection condition checked only the event count, so under a live
  reader event_bytes was never compared with anything; at the defaults that
  is 2048 retained events of up to 1 MiB each.

The eviction loop is now _trim(record). It runs after every append and in
the finally of events() once the reader's cursor is removed, and the
protection also requires event_bytes to stay within
max_event_bytes * LIVE_READER_EVENT_HEADROOM.

Plan item: BE-13

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…il closed

tools/artifact_boundary_review.py had four defects that made its gate
weaker than it read:

- corpus_digest hashed a separate tuple of case names, so deleting or
  rewriting a case left the digest unchanged. It is now computed from the
  _CASES tuple that runs: each case's name and source in run order, plus
  the shared fixture helpers.
- Any non-pass was reported as "blocked", so a boundary that accepted a
  hardlink read like a host limitation. The status is now "failed" when any
  case failed, "blocked" only when a case could not run, and "passed" only
  when every case ran and passed. An empty corpus is a failure.
- --json changed only the separators; both modes printed JSON. Without it
  the tool now prints a readable summary that names each failed or blocked
  case; with it, compact sorted JSON as before.
- Without --strict a total failure exited zero. Strict is now the default:
  1 for failed, 2 for blocked. --strict is still accepted, so check.ps1,
  the pre-push hook and the workflow keep working unchanged.

The twelve-case corpus itself is unchanged.

Plan item: EX-26

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/check_dev_environment.py only compared installed versions with the
declared specifiers. The setups that drift quietly even when every pin
matches went unreported: system Python with no virtual environment (here,
3.14 with a large unrelated site-packages), an interpreter other than the
3.11 the CI locks target, and an import name of a declared dependency that
resolves to code that dependency did not install.

It now prints advisories for all three before the pin result. They never
change the exit code. The lock target is read from the lock header rather
than hard-coded. The shadowing check locates each declared distribution's
top-level import names on the path the tests and the app use (repository
root and backend/ in place of scripts/) and compares the file found with
the distribution's RECORD. Lookup only locates modules; nothing is imported.
On this machine it reports a real case: `import py` loads the separate `py`
distribution, not the shim pytest installs.

Plan item: QA-24

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and its own chat

The real-API Playwright spec started scripts/e2e_backend.py on 8799 with
reuseExistingServer outside CI, and the README screenshot server defaults to
the same port. Anything left answering there was silently reused as "the
real API", and its generation test asserted on the first element of the
chat list, which depends on what earlier runs left behind.

- frontend/e2e/e2e-backend.json holds the port (now 8798) and the marker.
  playwright.config.ts (through e2e/e2eBackend.ts) and e2e_backend.py both
  read it, so the port is written down once.
- e2e_backend.py adds an X-Cortex-E2E-Backend header to its liveness
  route only.
- Playwright's readiness probe only sees a status code, so real-api.spec.ts
  checks the marker before each test, both on the backend port and through
  the dev server's proxy, and refuses with a clear message when it is
  missing.
- The generation test sends a prompt unique to the run, takes the thread id
  from the server's acceptance of that send, and asserts on that chat.

Plan item: QA-15

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dovvnloading
dovvnloading merged commit 2972c7a into main Oct 17, 2026
11 checks passed
@dovvnloading
dovvnloading deleted the fix/small-verified-defects branch October 17, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant