Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: ci

on:
push:
branches: [main]
pull_request:
branches: [main]

# Cancel in-flight runs for the same branch on a new push.
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

jobs:
pytest:
name: pytest (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# Pin to the lockfile-resolved Python floor (`pyproject.toml`
# `requires-python = ">=3.10"`). The reference stack that
# produces the 1.1.1 fingerprint pins is Python 3.10.
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Sync dependencies (lockfile-resolved)
run: uv sync --extra test --python ${{ matrix.python-version }}

- name: Verify numerical stack
# numpy / scipy / numba versions are pinned in uv.lock; printing
# them in CI logs makes fingerprint-drift debugging trivial.
run: |
uv run python -c "import numpy, scipy, numba; \
print(numpy.__version__, scipy.__version__, numba.__version__)"

- name: Run smoke tests
run: uv run python -m pytest tests/test_smoke.py -q

- name: Run full test suite
# Numba JIT cache is warmed by the smoke run; the rest is fast.
run: uv run python -m pytest tests/ -q

ruff:
name: ruff
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/ruff-action@v1
with:
# No formatter action — the repo doesn't ship a ruff.toml and
# we don't want to introduce formatting churn as part of CI.
args: check
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ results/*.html
results/*.png
!results/.gitkeep

Fernandes2019_BDSIM.pdf
manual.pdf
docs/Fernandes2019_BDSIM.pdf
docs/manual.pdf

# Cursor
.cursor/
Expand Down
20 changes: 12 additions & 8 deletions ADMIN.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,21 +89,25 @@ uv run python -m pytest tests/test_smoke.py -q

## Fingerprint regression

`tests/test_smoke.py` and `tests/test_live_simulator.py` pin SHA-256 fingerprints over the full trajectory. A silent numerical drift in a kernel will fail loud.
`tests/test_live_simulator.py` (and the per-Layer tests) pin SHA-256 fingerprints over the full trajectory. `tests/test_smoke.py` checks shapes, physical ranges, and determinism only — no SHA pins. A silent numerical drift in a kernel will fail loud.

Named profiles (see `docs/00-orientation/Byte-identical-contract.md` and `AGENTS.md`):

| Path | Profile | Meaning | Hash |
|---|---|---|---|
| batch | legacy fingerprint | Tests pass `fouling_dynamic=False` — **not** bare `ProcessFaults()` | `sv=c8807b23b14a9ad1` |
| batch | legacy fingerprint | | `pv=77def506dbfe25c9` |
| batch | legacy fingerprint | | `uv=17e620519474074a` |
| batch | Layer 2.5 fingerprint | Dynamic HEX fouling (`fouling_dynamic=True`) | `sv=696531c4...` |
| batch | Layer 2.6 (active disturbance) | Nonzero disturbance amplitudes | `sv=8865a8c3...` |
| batch | runtime default (1.2.0+) | Bare `ProcessFaults()` enables Layers 2.5 + 2.1 + 2.4 (both) + 2.8a | `sv=691cf51b4c1a0bc2` `pv=baedc29fcfa8f526` `uv=4e4134e40fa0c1ad` |
| batch | legacy fingerprint | Tests pass all Layer masters `False` (21-wide state) | `sv=c8807b23b14a9ad1` `pv=77def506dbfe25c9` `uv=17e620519474074a` |
| batch | Layer 2.5 fingerprint | Dynamic HEX fouling, all other masters off (22-wide state) | `sv=696531c4990c5b1e` |
| batch | Layer 2.5 + Layer 2.1 | `fouling_dynamic=True quality_state=True` (27-wide state) | `sv=d663e17d687b1133` |
| batch | Layer 2.4 pump-only | `pump_wear=True`, no valve_wear / quality_state (23-wide state) | `sv=7ddd7aaa7da4b679` |
| batch | Layer 2.4 valve-only | `valve_wear=True`, no pump_wear / quality_state (23-wide state) | `sv=9425d007ae968ee7` |
| batch | Layer 2.4 both | `pump_wear=True valve_wear=True`, no quality_state (24-wide state) | `sv=c092fe082f4ba6f4` |
| batch | Layer 2.6 (active disturbance) | Nonzero disturbance amplitudes, all other masters off | `sv=8865a8c352cb6b55` |
| live | runtime default (1.2.0+) | Bare `ProcessFaults()` (`tf=14400 dt=10`) | `sv=80f6f04683703382` `pv=9e2b2081f469e473` `uv=c3a05be9a234fe2a` |
| live | legacy fingerprint | Matching legacy knobs | `sv=23c3c885694c3d24` |
| live | Layer 2.6 (active disturbance) | | `sv=bb763a9bde1d3fc9` |

Runtime default is bare `ProcessFaults()` (`fouling_dynamic=True`, `sv` width 22). When a fingerprint updates, that's a "we changed the math **or the numerical stack**" signal. Pins as of **1.1.1** match `uv.lock` (`numpy==2.2.6`, `scipy==1.15.3`, `numba==0.66.0`). Document the why in the commit body and update the pin in the test file. Don't suppress the test.
Runtime default (`ProcessFaults()`) is now the broad-defaults profile — `fouling_dynamic=True`, `quality_state=True`, `pump_wear=True`, `valve_wear=True`, `spectrum_enabled=True` (sv width 30). Tests and demos needing a narrower state must opt out explicitly via the relevant `*_state=False` / `*_wear=False` / `spectrum_enabled=False` overrides. When a fingerprint updates, that's a "we changed the math **or the numerical stack**" signal. Pins as of **1.2.0** match `uv.lock` (`numpy==2.2.6`, `scipy==1.15.3`, `numba==0.66.0`). Document the why in the commit body and update the pin in the test file. Don't suppress the test.

## Health checks

Expand Down Expand Up @@ -145,7 +149,7 @@ If you ever need bdsim to run as a long-lived process for some other consumer, t

- **Original MATLAB:** Natércia C. P. Fernandes, 2019, University of Coimbra (`natercia@eq.uc.pt`). Upstream: `https://github.com/naterciafernandes/BDSIM`.
- **Docs vault:** [`docs/Home.md`](docs/Home.md) (see [`docs/README.md`](docs/README.md)).
- **Optional PDFs:** `Fernandes2019_BDSIM.pdf` and `manual.pdf` are gitignored at the repo root — drop local copies if you have them; not required to run.
- **Optional PDFs:** `Fernandes2019_BDSIM.pdf` and `manual.pdf` are gitignored under `docs/` (see `.gitignore`) — drop local copies there if you have them; not required to run.
- **Citation:** [`CITATION.cff`](CITATION.cff).
- **License:** GPLv3+ (matches upstream); see [`LICENSE`](LICENSE).
- **Maintainer scripts:** [`scripts/`](scripts/) — optional utilities only (see `scripts/README.md`).
Expand Down
29 changes: 17 additions & 12 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Both callers depend on the **byte-identical trajectory contract**: given the sam
- **Don't change the ODE math.** If you think the upstream MATLAB is wrong, write a test that captures the bug, then ask before "fixing" it. See `NOTES.md` for two prior upstream-faithful fixes (Foil noise typo, temperature display bug) — those were agreed, and documented.
- **Don't add a new external dependency** without asking. The current stack (numpy/scipy/numba/torch) is intentional. New deps = new deploy surface.
- **Don't relax the byte-identical fingerprint contract.** If you need different behavior, gate it behind a `ProcessFaults` flag (default off/zero, except documented exceptions — today `fouling_dynamic=True`).
- **Don't touch `pyproject.toml` version without also updating `bdsim/__init__.py:__version__` and the dashboard's minimum-required-version pin in lockstep.** All three are pinned to the same value today (`1.1.1`).
- **Don't touch `pyproject.toml` version without also updating `bdsim/__init__.py:__version__`.** Both are pinned to the same value today (`1.1.1`). The bdsim-dashboard does not currently pin a minimum bdsim version; add the third pin when the dashboard gains an installation contract (see `ADMIN.md` — Versioning).

## File map

Expand All @@ -44,7 +44,7 @@ bdsim/
└── cli.py # `python -m bdsim` entry point

tests/
├── test_smoke.py # 14 smoke tests, fingerprint regression
├── test_smoke.py # 14 smoke tests (shapes, physical ranges, determinism; no SHA pins)
├── test_live_simulator.py # LiveSimulator + byte-identical contract to run_with
├── test_disturbances.py # Layer 2.6 external disturbance track
├── test_layer24_degradation.py # Layer 2.4 pump_health + valve_stiction_pct
Expand All @@ -63,14 +63,19 @@ NOTES.md # historical: upstream-faithful bugs we found and fixed
## Build conventions

- **Numba kernels are sacred.** Don't refactor for readability if it costs a fingerprint pin update. The ODE RHS, reaction kinetics, valve stiction, PID step, ARMAX noise update, sensor measurements, and AE model are all `@njit(cache=True)`. The 72h default sim runs in ~60s on a single core.
- **Default profiles (read carefully).** New Layer flags should default **off / zero**, except documented exceptions. Today `ProcessFaults.fouling_dynamic` defaults to **`True`** (Layer 2.5 on → `sv` width 22). Bare `ProcessFaults()` is **not** the upstream legacy pin path.

| Profile | Construction | Pins (examples) |
|--------|--------------|-----------------|
| Runtime default | `ProcessFaults()` | Layer 2.5 path; not the legacy hash |
| Legacy fingerprint | `fouling_dynamic=False` (+ other masters off/zero) | batch `sv=c8807b23…`, live `sv=23c3c885…` |
| Layer 2.5 fingerprint | `fouling_dynamic=True`, extras off | batch `sv=696531c4…` |
| Layer 2.6 active | disturbance amplitudes on | batch `sv=8865a8c3…`, live `sv=bb763a9b…` |
- **Default profiles (read carefully).** As of 1.2.0, the four major Layer master switches default to **`True`** (Layer 2.5 fouling, 2.1 quality latching, 2.4 pump wear, 2.4 valve wear, 2.8a spectra). Bare `ProcessFaults()` enables all of them → `sv` width 30, hash `691cf51b…`. Tests and demos that need a narrower state vector must pass the relevant `quality_state=False` / `pump_wear=False` / `valve_wear=False` / `spectrum_enabled=False` overrides explicitly. The legacy / Layer-2.5-only profiles below remain canonical — they are the byte-identical contracts to the upstream MATLAB baseline and the 1.0 Layer-2.5 release.

| Profile | Construction | `sv` width | Pins (full SHA-256[:16]) |
|--------|--------------|-------------|----------------------------|
| Runtime default (1.2.0+) | `ProcessFaults()` | **30** | batch `sv=691cf51b…` `pv=baedc29f…` `uv=4e4134e4…`; live `sv=80f6f046…` |
| Legacy fingerprint | `fouling_dynamic=False`, all masters off | 21 | batch `sv=c8807b23…` `pv=77def506…` `uv=17e62051…`; live `sv=23c3c885…` |
| Layer 2.5 fingerprint | `fouling_dynamic=True`, all other masters off | 22 | batch `sv=696531c4…` `pv=3c96ca4f…` `uv=0d9a9673…` |
| Layer 2.5 + Layer 2.1 | `fouling_dynamic=True`, `quality_state=True` (no Layer 2.4, no spectra) | 27 | batch `sv=d663e17d…` (Layer 2.1 review pending — see Open Questions in `Progress.md`) |
| Layer 2.4 pump-only | `pump_wear=True`, `valve_wear=False`, `fouling_dynamic=True`, `quality_state=False` | 23 | batch `sv=7ddd7aaa…` `pv=e0dba881…` `uv=86c2704f…` |
| Layer 2.4 valve-only | `pump_wear=False`, `valve_wear=True`, `fouling_dynamic=True`, `quality_state=False` | 23 | batch `sv=9425d007…` `pv=02296ffa…` `uv=aa146ea3…` |
| Layer 2.4 both | `pump_wear=True`, `valve_wear=True`, `fouling_dynamic=True`, `quality_state=False` | 24 | batch `sv=c092fe08…` `pv=2d26f03f…` `uv=4ef50b9f…` |
| Layer 2.6 active | `fouling_dynamic=True`, `ambient_t_amplitude_k` etc. > 0 (all other masters off) | 22 | batch `sv=8865a8c3…`; live `sv=bb763a9b…` |
| Layer 2.6b live | `LiveSimulator` cw_pump_trip mid-run override | 22 | live `sv` matches Layer 2.6 baseline; trip is a single row-removal event |

Full write-up: `docs/00-orientation/Byte-identical-contract.md`.
- **Tests run from the bdsim root:** `python3 -m pytest tests/ -q`. Full suite: ~6:40.
Expand All @@ -89,8 +94,8 @@ NOTES.md # historical: upstream-faithful bugs we found and fixed
This repo's coverage:

- ✅ Step 1 (live sim driver), Step 2 (MQTT publish is on the dashboard side), Step 4 (live fault injection), Step 5 (sensor failure modes), Step 8 (scenario runner is on the dashboard side)
- ✅ Layer 2.1 (quality latching — `quality_state=True` mode), Layer 2.5 (HEX fouling as continuous state, `fouling_dynamic=True` mode), Layer 2.6 (external disturbances), Layer 2.6b (cw_pump_trip mid-run override), Layer 2.7 (operator-driven disturbance schedule), Layer 2.4 (pump_health + valve_stiction_pct as continuous state — `pump_wear=True` / `valve_wear=True`), Layer 2.8a (NIR/IR virtual spectrum sensor — `spectrum_enabled=True`), **Layer 2.8b** (five-mode fouling stepper — modes 4/5 stochastic ARMAX windowed injection, port of upstream `fouling.m`)
- ⏳ Layer 2.1 `quality_latched` code review (Joel owes) — pending
- ✅ Layer 2.5 (HEX fouling as continuous state, `fouling_dynamic=True` mode), Layer 2.6 (external disturbances), Layer 2.6b (cw_pump_trip mid-run override), Layer 2.7 (operator-driven disturbance schedule), Layer 2.4 (pump_health + valve_stiction_pct as continuous state — `pump_wear=True` / `valve_wear=True`), Layer 2.8a (NIR/IR virtual spectrum sensor — `spectrum_enabled=True`), **Layer 2.8b** (five-mode fouling stepper — modes 4/5 stochastic ARMAX windowed injection, port of upstream `fouling.m`)
- 🟡 Layer 2.1 (quality latching — `quality_state=True` mode): shipped, but `quality_latched` code review (Joel owes) is still pending. Marked 🟡 rather than ✅ until that review closes.

## Coordination with bdsim-dashboard

Expand Down
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,59 @@ Exact pin strings live in `tests/` and are summarized in `ADMIN.md`.
### Changed
### Fixed

## [1.2.0]

### Changed

- **Runtime default expands to four Layer masters ON.** `ProcessFaults()`
now enables Layer 2.5 (`fouling_dynamic`), Layer 2.1 (`quality_state`),
Layer 2.4 pump wear (`pump_wear`), Layer 2.4 valve wear (`valve_wear`),
and Layer 2.8a spectra (`spectrum_enabled`). Bare `ProcessFaults()` is
now a new pinned profile — sv width 30, hash `691cf51b…` (batch) /
`80f6f046…` (live).

- **Layer 2.1 + Layer 2.5 combination fix.** A pre-existing off-by-one in
`bdsim/ode.py` (RHS `dsvdt` allocated 28 instead of 27 when
`quality_state=True` and `fouling_dynamic=False`) was fixed. The
numerical path is unchanged for the common cases
(`fouling_dynamic=True`); the fix only corrects the
`quality_state=True + fouling_dynamic=False` sizing and closes a
latent crash window. Tests that previously constructed
`ProcessFaults(fouling_dynamic=False, quality_state=True)` (the
broken combination) now run cleanly with sv width 27.

- **Tests updated to opt out explicitly.** Every test that targeted the
legacy 21-component profile or the Layer 2.5 22-component profile now
passes explicit `quality_state=False, pump_wear=False,
valve_wear=False, spectrum_enabled=False` overrides. The pinned
hashes for these profiles are unchanged from 1.1.1
(`c8807b23…` / `696531c4…` / `8865a8c3…` / `bb763a9b…` / `23c3c885…`).

### Added

- **New pinned profile: 1.2.0 runtime default.** Bare `ProcessFaults()`
now produces a stable trajectory with `sv=691cf51b4c1a0bc2`
/ `pv=baedc29fcfa8f526` / `uv=4e4134e40fa0c1ad` (batch) and
`sv=80f6f04683703382` / `pv=9e2b2081f469e473` /
`uv=c3a05be9a234fe2a` (live). This is the contract that downstream
consumers (dashboard, fault-detection pipelines) target by default;
legacy / Layer 2.5 only profiles are still available via explicit
overrides.

- **New pinned profile: Layer 2.5 + Layer 2.1.** `fouling_dynamic=True,
quality_state=True` (no Layer 2.4, no spectra) → sv width 27, hash
`sv=d663e17d687b1133`.

### Notes

- AGENTS.md "Default profiles" table and the profile fingerprint tables
in `ADMIN.md`, `docs/30-engine/Fingerprints-and-tests.md`, and
`docs/00-orientation/Byte-identical-contract.md` all updated.
- Demos and ML pipelines trained on 21-wide or 22-wide state vectors
must re-train or pin the legacy profile explicitly.
- Companion dashboard may need its `SimRunner._build_pfaults()` updated
to match (auto-derived from `ProcessFaults()` defaults, but verify).

## [1.1.1]

### Changed
Expand Down
4 changes: 2 additions & 2 deletions NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ pv (display):
DP: 4245.07 - 11562.19 Pa ✓ was −1312 to 19157
```

All 12 smoke tests still pass.
All 14 smoke tests still pass.

### If you need the upstream-faithful noise back

Expand Down Expand Up @@ -134,4 +134,4 @@ fig06: setpoint 50.00 measurement 45.98–50.21 state 46.24–50.00 (TD loop,
fig03: Tmet 46.61–53.40 °C order_lift_oil 39.62–41.93 %
```

All 12 smoke tests still pass.
All 14 smoke tests still pass.
Loading
Loading