Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/research/e2-wave1/V1V2/CONTRACT.release-remainder.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
DIAGNOSTIC ONLY: test-kernel prerequisite skipped after its separate required-gate failure; this does not make test-release pass.
COMMAND: PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTHONDONTWRITEBYTECODE=1 make -o install -o test-kernel VENV_DIR=/tmp/cc-validation-20261023 test-release
PYTHONPATH=src /tmp/cc-validation-20261023/bin/python examples/minimal/run_bounds.py >/dev/null
PYTHONPATH=src /tmp/cc-validation-20261023/bin/pytest \
tests/integration/test_reproduce_paper.py \
tests/integration/test_verify_paper_artifacts.py -q
.......... [100%]

EXIT_CODE=0
48 changes: 48 additions & 0 deletions docs/research/e2-wave1/V1V2/CONTRACT.test-release.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
COMMAND: PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTHONDONTWRITEBYTECODE=1 make -o install VENV_DIR=/tmp/cc-validation-20261023 test-release
PYTHONPATH=src /tmp/cc-validation-20261023/bin/pytest tests/unit/kernel -q
........................................................................ [ 47%]
........................................................................ [ 95%]
....... [100%]
PYTHONPATH=src /tmp/cc-validation-20261023/bin/pytest tests/unit/kernel/test_sensitivity.py tests/unit/kernel/test_frechet_classes.py -q
.............................................................. [100%]
PYTHONPATH=src /tmp/cc-validation-20261023/bin/mypy src/cc/kernel --strict
src/cc/kernel/frechet_classes.py:441: error: Redundant cast to
"ndarray[tuple[Any, ...], dtype[float64]]" [redundant-cast]
return cast(FloatArray, clipped)
^
src/cc/kernel/frechet_classes.py:586: error: Returning Any from function
declared to return "ndarray[tuple[Any, ...], dtype[float64]]" [no-any-return]
return np.all(states == 1.0, axis=1).astype(float)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cc/kernel/frechet_classes.py:587: error: Returning Any from function
declared to return "ndarray[tuple[Any, ...], dtype[float64]]" [no-any-return]
return np.any(states == 1.0, axis=1).astype(float)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cc/kernel/frechet_classes.py:631: error: Redundant cast to
"ndarray[tuple[Any, ...], dtype[float64]]" [redundant-cast]
return cast(FloatArray, normalized)
^
src/cc/kernel/cliff.py:596: error: Returning Any from function declared to
return "ndarray[tuple[Any, ...], dtype[float64]]" [no-any-return]
return log_c
^~~~~~~~~~~~
src/cc/kernel/cliff.py:703: error: Returning Any from function declared to
return "ndarray[tuple[Any, ...], dtype[float64]]" [no-any-return]
return numerator / denominator * factor
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/cc/kernel/stress.py:611: error: Returning Any from function declared to
return "ndarray[tuple[Any, ...], dtype[float64]]" [no-any-return]
return np.mean(np.abs(states[:, None, :] - states[None, :, :]), ax...
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~...
src/cc/kernel/stress.py:706: error: Redundant cast to
"ndarray[tuple[Any, ...], dtype[float64]]" [redundant-cast]
return cast(FloatArray, pmf / float(np.sum(pmf)))
^
src/cc/kernel/ccf_models.py:347: error: Redundant cast to
"ndarray[tuple[Any, ...], dtype[Any]]" [redundant-cast]
return cast(np.ndarray, np.clip(arr, 0.0, 1.0).astype(float, copy=...
^
Found 9 errors in 4 files (checked 12 source files)
make: *** [test-kernel] Error 1

EXIT_CODE=2
48 changes: 48 additions & 0 deletions docs/research/e2-wave1/V1V2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# V1/V2: structured validator failures and checked timestamps

Base: CONTRACT E1 `d060fee`; applied after SITE E5 `1fe36e6`.

`pre.log`: **exit 1, 24 failed / 19 passed**. The corpus reproduces malformed
replicate/outcome tracebacks, unstructured malformed-file errors, and invalid
timestamps accepted with exit 0. `nonfinite.pre.log`: **exit 1, 3 failed**;
non-JSON NaN/Infinity tokens nested in raw detail were accepted before strict
JSON parsing.

`scripts/validate_e2_observations.py` now reports input/schema violations before
running design checks or audit calculations. Malformed rows never reach the
replicate hash/sort or outcome integer coercion. Input read/decode errors and
empty inputs produce structured violations with exit 1. JSON mode writes one
JSON document to stdout. Valid datasets retain the same audit and warnings
(`audit.log`, including strict JSON decoding of both reference runs).

The schema validator receives a `FormatChecker`. If jsonschema's optional
date-time checker is absent, a registered stdlib fallback checks the timestamp
shape, calendar date, time, and explicit UTC/numeric offset. No validator
dependency was added. Syntax reference:
[RFC 3339 sections 5.6–5.7](https://www.rfc-editor.org/rfc/rfc3339#section-5.6).

`post.log`: **exit 0, 82 passed**, with no skips. This includes the malformed
corpus, E1 interoperability, existing observation-contract tests, and dry-run
tests. Ruff and whitespace checks pass (`audit.log`). No scientific constants,
thresholds, outcomes, sample sizes, or frozen preregistration text changed.

```sh
E2_SITE_ROOT=/Users/pranavbhave/Developer/GitHub/e2-wave1-site \
PYTHONPATH=/tmp/e2-wave1-20261024/test-dependencies:src:. \
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
/tmp/cc-validation-20261023/bin/python -B -m pytest \
-o addopts= -p no:cacheprovider -q -rs \
tests/unit/research/test_e2_validator_malformed.py \
tests/unit/research/test_e2_stratum.py \
tests/unit/research/test_e2_observation_contract.py \
tests/unit/research/test_e2_dryrun.py
```

Repository release checks are recorded separately from the frozen E2 GO
criteria. `CONTRACT.test-release.log` records the required command **exit 2**:
kernel unit tests pass; strict mypy reports nine errors in four unchanged
kernel files. Kernel source equality to base `bf5740c` is checked in `audit.log`.
No unrelated fixes were made. `CONTRACT.release-remainder.log` records a
diagnostic run with that failed prerequisite explicitly skipped: the minimal
example and ten release integration tests pass, **exit 0**. The required
combined release check remains failed.
28 changes: 28 additions & 0 deletions docs/research/e2-wave1/V1V2/audit.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
PASS: date-time registered in FormatChecker=True
COMMAND: /tmp/cc-validation-20261023/bin/python -B scripts/validate_e2_observations.py examples/e2/observations.example.jsonl --json
PASS: clean JSON stdout; exit=0; zero violations; audit and warnings exactly equal pre-V1V2 HEAD; rows=24
COMMAND: /tmp/cc-validation-20261023/bin/python -B scripts/validate_e2_observations.py experiments/e2_dryrun/results/observations.jsonl --json
PASS: clean JSON stdout; exit=0; zero violations; audit and warnings exactly equal pre-V1V2 HEAD; rows=66
COMMAND: /tmp/cc-validation-20261023/bin/python -B scripts/validate_e2_observations.py /tmp/e2-wave1-20261024/does-not-exist.jsonl --json
{
"report": null,
"violations": [
"input violation: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/e2-wave1-20261024/does-not-exist.jsonl'"
],
"warnings": []
}

STDERR:
E2 conformance FAILED with 1 violation(s)

EXIT_CODE=1; PASS structured rejection; audit not computed
COMMAND: /tmp/cc-validation-20261023/bin/python -m ruff check scripts/validate_e2_observations.py tests/unit/research/test_e2_validator_malformed.py
All checks passed!

EXIT_CODE=0
COMMAND: git diff --check

EXIT_CODE=0
COMMAND: git diff --exit-code bf5740c -- src/cc/kernel

EXIT_CODE=0
34 changes: 34 additions & 0 deletions docs/research/e2-wave1/V1V2/nonfinite.pre.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
COMMAND: PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 /tmp/cc-validation-20261023/bin/python -B -m pytest -o addopts= -p no:cacheprovider -q --tb=short tests/unit/research/test_e2_validator_malformed.py -k nonstandard_json_number
FFF [100%]
=================================== FAILURES ===================================
_________ test_nonstandard_json_number_is_a_structured_violation[NaN] __________
tests/unit/research/test_e2_validator_malformed.py:128: in test_nonstandard_json_number_is_a_structured_violation
_reject(_run(_write_rows(tmp_path, rows)))
tests/unit/research/test_e2_validator_malformed.py:46: in _reject
assert result.returncode == 1, (result.returncode, payload, result.stderr)
E AssertionError: (0, {'report': {'rows': 3, 'guardrails': ['gr.alpha', 'gr.beta', 'gr.gamma'], 'items': 1, 'complete_case_items': 1, .....' spans gr.alpha, gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."]}, '')
E assert 0 == 1
E + where 0 = CompletedProcess(args=['/tmp/cc-validation-20261023/bin/python', '-B', '/Users/pranavbhave/Developer/GitHub/e2-wave1-c..., gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."\n ]\n}\n', stderr='').returncode
_______ test_nonstandard_json_number_is_a_structured_violation[Infinity] _______
tests/unit/research/test_e2_validator_malformed.py:128: in test_nonstandard_json_number_is_a_structured_violation
_reject(_run(_write_rows(tmp_path, rows)))
tests/unit/research/test_e2_validator_malformed.py:46: in _reject
assert result.returncode == 1, (result.returncode, payload, result.stderr)
E AssertionError: (0, {'report': {'rows': 3, 'guardrails': ['gr.alpha', 'gr.beta', 'gr.gamma'], 'items': 1, 'complete_case_items': 1, .....' spans gr.alpha, gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."]}, '')
E assert 0 == 1
E + where 0 = CompletedProcess(args=['/tmp/cc-validation-20261023/bin/python', '-B', '/Users/pranavbhave/Developer/GitHub/e2-wave1-c..., gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."\n ]\n}\n', stderr='').returncode
______ test_nonstandard_json_number_is_a_structured_violation[-Infinity] _______
tests/unit/research/test_e2_validator_malformed.py:128: in test_nonstandard_json_number_is_a_structured_violation
_reject(_run(_write_rows(tmp_path, rows)))
tests/unit/research/test_e2_validator_malformed.py:46: in _reject
assert result.returncode == 1, (result.returncode, payload, result.stderr)
E AssertionError: (0, {'report': {'rows': 3, 'guardrails': ['gr.alpha', 'gr.beta', 'gr.gamma'], 'items': 1, 'complete_case_items': 1, .....' spans gr.alpha, gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."]}, '')
E assert 0 == 1
E + where 0 = CompletedProcess(args=['/tmp/cc-validation-20261023/bin/python', '-B', '/Users/pranavbhave/Developer/GitHub/e2-wave1-c..., gr.gamma; observed dependence may be mechanism-induced rather than a property of composition."\n ]\n}\n', stderr='').returncode
=========================== short test summary info ============================
FAILED tests/unit/research/test_e2_validator_malformed.py::test_nonstandard_json_number_is_a_structured_violation[NaN]
FAILED tests/unit/research/test_e2_validator_malformed.py::test_nonstandard_json_number_is_a_structured_violation[Infinity]
FAILED tests/unit/research/test_e2_validator_malformed.py::test_nonstandard_json_number_is_a_structured_violation[-Infinity]
3 failed, 43 deselected in 0.46s

EXIT_CODE=1
6 changes: 6 additions & 0 deletions docs/research/e2-wave1/V1V2/post.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
COMMAND: E2_SITE_ROOT=/Users/pranavbhave/Developer/GitHub/e2-wave1-site PYTHONPATH=/tmp/e2-wave1-20261024/test-dependencies:src:. PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 /tmp/cc-validation-20261023/bin/python -B -m pytest -o addopts= -p no:cacheprovider -q -rs tests/unit/research/test_e2_validator_malformed.py tests/unit/research/test_e2_stratum.py tests/unit/research/test_e2_observation_contract.py tests/unit/research/test_e2_dryrun.py
........................................................................ [ 87%]
.......... [100%]
82 passed in 7.97s

EXIT_CODE=0
Loading
Loading