Skip to content

test(ci): isolate semantic probe fixture coverage - #5375

Open
Duang777 wants to merge 2 commits into
mainfrom
codex/fix-coverage-temp-source
Open

Duang777 wants to merge 2 commits into
mainfrom
codex/fix-coverage-temp-source

Conversation

@Duang777

Copy link
Copy Markdown
Collaborator

Summary

Prevent the semantic probe CLI fixture from inheriting pytest and coverage startup variables. The fixture runs from a temporary synthetic checkout; tracing that child records temporary loopx/a.py paths in shard coverage artifacts, which do not exist on the separate aggregation runner.

This keeps the real CLI assertions unchanged and excludes only the synthetic child process from the parent coverage session. It follows the existing CI fixture isolation pattern from dd17cc259. No production code or coverage threshold changes.

Failure evidence

On #5367 exact head f5995f102, all four Python test shards passed, then the required pytest aggregation job failed while combining coverage:

No source for code: /tmp/pytest-of-runner/.../test_cli_inspects_disjoint_ind0/loopx/a.py

Shard 1 and shard 2 reports both contained temporary loopx/a.py entries. The tests were introduced on main by #5317; their earlier CI run never reached coverage combination because other shard failures stopped the aggregate first.

Validation

  • Before the fix, a local cross-runner simulation passed both CLI tests, deleted the pytest temp root, then reproduced coverage report exit 1 with the same missing-source error.
  • After the fix, the same simulation passes after temp-root deletion.
  • Full semantic probe suite: 17 passed.
  • Python CI workflow contract suite: 290 passed.
  • Ruff and git diff --check: passed.
  • loopx canary premerge --from-git-diff: passed, 2 selected canaries, 0 failures, 0 manual holds.
  • Bits Code Guard full-file review: no P0-P2 findings.

Maintainer review and merge are required.

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777

Copy link
Copy Markdown
Collaborator Author

The remaining typescript-core (1/3) failure is confirmed on main run 36770551543 and comes from the duplicate bare SHA-256 matcher introduced by #5364, not from this PR. A minimal main-based fix is now in #5377; this branch remains unchanged at exact head 086f2643355d36b62ccf74cbdac6200ca31a498f.

@Duang777

Copy link
Copy Markdown
Collaborator Author

A second inherited failure appeared in test-shard (2): test_runtime_source_churn_has_a_stable_readiness_diagnostic observed ready instead of package_invalid. The current main fixture deletes later.ts from one parallel read without synchronizing the other read; if later.ts is already prefetched, the expected missing-file retry does not occur. #5367 makes this race deterministic with an Event and its exact-head shard 2 passed. This is independent of the sole coverage-helper change in #5375, so no code change is planned here.

@Duang777

Copy link
Copy Markdown
Collaborator Author

The newly completed test-shard (4) failure is the second assertion covered by #5367: test_runtime_request_source_churn_raises_a_stable_startup_diagnostic observed runtime_exited_before_ready instead of packaged_runtime_source_unstable. #5367 synchronizes this source-churn fixture and its exact-head test-shard (4) passed. This is independent of #5375’s sole coverage-helper change, so this branch remains unchanged.

@Duang777

Copy link
Copy Markdown
Collaborator Author

The final test-shard (3) failure is another inherited main regression: #5366 advanced the usage notice to version 5 but left test_source_first_usage_disclosure_keeps_json_pure_and_does_not_send pinned to 4. I reproduced the exact 5 == 4 assertion on clean origin/main@9c9b90d8e; the one-line test correction is isolated in #5379. No #5375 code change is needed.

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777

Duang777 commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

Synced with main@f49b4a00870604d39fa4318da24d6dd35e72bb6e using signed merge commit eb460e46c206fbca3d6c5ffdb92b39763fdc7e61. The merge was conflict-free, and the PR diff remains limited to tests/architecture/test_semantic_development_probe.py.

Focused validation on the new exact head: tests/architecture/test_semantic_development_probe.py passed all 17 tests. The branch was pushed normally without history rewriting. CI has restarted; no merge action was taken.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant