Review follow-up, 2026-09-12. Rechecked origin at 98f267697b74.
Evidence and impact
Python 3.12.14 editable installation and 121 unit tests passed (89% coverage). CI run 34657207356 passed the unit matrix; its integration job was skipped.
The job is labeled “Integration Tests (full BERT pipeline),” but tests/integration/test_e2e.py patches SentenceTransformer and AutoTokenizer with mocks. Enabling this job alone does not prove weights/tokenizer compatibility or actual embedding inference. Its comment mentions a schedule, while the job condition permits workflow_dispatch or a commit-message marker.
This is a verification/claim gap, not a reproduced real-model crash.
Proposed implementation
- Keep the fast mocked pipeline tests and label them accurately. Separate model-backed checks with a clear opt-in marker/job.
- Choose the shipped supported model and record its identifier/revision, tokenizer revision, Python, torch and sentence-transformers versions. Permit a pre-populated local cache; make downloads explicit and bounded. Use synthetic/public sample text only.
- Run the real model loader and tokenizer without monkeypatches, call actual embedding/extraction paths, and check expected dimensions, finite values, nonempty meaningful output, correct offsets/provenance and repeat-run behavior. Test empty/oversized input according to the existing contract.
- Add failure-path coverage for unavailable/corrupt weights and offline cache misses. Report an explicit failed or unavailable real-model check rather than quietly switching to a mock.
- Keep a small hand-labeled extraction set separate from the installation smoke. Report precision/recall or a similarly defined task metric; record cold/warm latency and peak memory on a named machine. Do not treat successful inference as proof of semantic quality.
- Align workflow comments, trigger conditions and documentation. A manual acceptance job is sufficient initially; do not silently impose model downloads on every contributor run.
Acceptance criteria
Implementation should start by refreshing origin and checking this finding against the new head. Preserve existing safety controls. This issue/comment records findings and proposed acceptance criteria; it does not claim the fixes or unperformed live checks are complete.
Review follow-up, 2026-09-12. Rechecked origin at 98f267697b74.
Evidence and impact
Python 3.12.14 editable installation and 121 unit tests passed (89% coverage). CI run 34657207356 passed the unit matrix; its integration job was skipped.
The job is labeled “Integration Tests (full BERT pipeline),” but tests/integration/test_e2e.py patches SentenceTransformer and AutoTokenizer with mocks. Enabling this job alone does not prove weights/tokenizer compatibility or actual embedding inference. Its comment mentions a schedule, while the job condition permits workflow_dispatch or a commit-message marker.
This is a verification/claim gap, not a reproduced real-model crash.
Proposed implementation
Acceptance criteria
Implementation should start by refreshing origin and checking this finding against the new head. Preserve existing safety controls. This issue/comment records findings and proposed acceptance criteria; it does not claim the fixes or unperformed live checks are complete.