Skip to content

Count overlaps in bench and correct documents that had drifted from the code - #24

Merged
schreck61 merged 3 commits into
mainfrom
docs-audit-fixes
Nov 15, 2026
Merged

schreck61 merged 3 commits into
mainfrom
docs-audit-fixes

Conversation

@schreck61

Copy link
Copy Markdown
Owner

What and why

An audit of the documents against the code as released in 0.6.0 gave seven statements to check. Five were wrong and one picture showed the previous version; they are corrected here, each tied to the code that shows it. The seventh agrees with the code and the other documents on main, and is left as it is.

Statement What the code shows Change
1 ACCESSIBILITY.md: "Everything the app measures can be produced from the command line" The app shows the level of service, the maximum delay and CO₂ per vehicle (src/gui/mod.rs:1099-1111), breakdowns by approach, movement and class (src/gui/mod.rs:1188-1228) and charts. bench prints none of those (src/bin/bench.rs:166-169), and of the examples only cooperative prints the maximum delay (examples/cooperative.rs:106). Nothing outside src/gui reads co2_per_veh or calls los The sentence says what the commands print and names what only the app shows. The heading and the introduction made the same claim and change with it
2 docs/assurance-case.md: a controller whose weights file is rejected "starts from a fresh network" Since 75341af the loader skips a copy that doesn't load for the next one and ends with the built-in copy (src/control/weights.rs:51-65). A Hybrid file whose network doesn't fit loads without it and keeps its base tuning (src/control/hybrid.rs:143-156). A fresh network is what Deep RL does only when no copy loads (src/control/rl.rs:107-119). tests/weights_files.rs:152-170 checks the order The bullet describes the fallback and points to section 8.7 of the architecture document
3 ROADMAP.md: keyboard ways to "place sensors" The app's tools are Select, Vehicle and Pedestrian (src/gui/mod.rs:33-37); sensors are set with checkboxes and sliders (src/gui/mod.rs:790-817) "place sensors" is removed
4 docs/architecture.md, section 9.4: the About picture shows 0.5.1 The window prints CARGO_PKG_VERSION (src/gui/mod.rs:1561), which is 0.6.0 (Cargo.toml:4) Retaken with TS_ABOUT=1, as Appendix A lists it, and cut at the same place: 389 pixels differ from the earlier picture, all in the version number. The alt text follows
5 docs/architecture.md, section 10.3, the tests/cooperative.rs row: "Every run must show no monitor violations" safety_failures (tests/cooperative.rs:35-60) also requires no overlap between vehicle bodies, margins and times within a microsecond and at least one crossing measured. Three tests run the crossing without it (tests/cooperative.rs:71, tests/cooperative.rs:351, tests/cooperative.rs:369) The sentence lists what is required, and of which runs
6 README.md: "Zero vehicle overlaps in every run" under the controller table (README.md:271), and the tests/tools.rs row: "without collisions" check_safety is off in a new Sim (src/sim/mod.rs:385). Overlaps are counted only in check_overlaps (src/sim/mod.rs:2217-2261), which runs only when it is on (src/sim/mod.rs:1649-1651). bench never turned it on, so its coll column could only be 0 bench now turns the check on for its strategy tables (src/bin/bench.rs:22-28), reports the overlaps as a total over the seeds, not a mean that could round one away, and has a unit test that fails with the check off (src/bin/bench.rs:226-252). With that, the README sentence and the test row are true as written (see Results), so neither changes
7 The label "Nominal: latency and actuation (no failures)" (src/sim/coop.rs:116) The README and the design notes on main describe the nominal gap the same way: it assumes no vehicle fails None

Because of 6, docs/architecture.md also changes in three places: bench joins the list of where the safety check runs in section 6.1 (the list lacked the green-wave test too), the src/bin/bench.rs row says what the coll column holds, and section 10.3 lists the new unit test.

Results

Traffic behavior doesn't change: the safety check only counts. cargo run --release --bin bench all 20 6 prints the same figures with the check on as it did with it off, in every column of all 32 rows (the run time at the end of each row aside), and coll is 0 in every row: 192 runs of 25 simulated minutes without an overlap. The README's controller table is reproduced exactly.

The check added no run time that could be told from the noise of a busy machine: three alternating pairs of that command took 32 to 40 s without it and 35 to 38 s with it, apart from one run of 118 s during a load spike.

Checklist

  • cargo test --locked --release passes (zero vehicle overlaps)
  • The code is formatted with cargo fmt --all
  • cargo clippy --locked --release --all-targets -- -D warnings is clean (and for wasm32-unknown-unknown if the app changed)
  • New functionality has tests, and a bug fix has a test that fails without it (see CONTRIBUTING.md)
  • Runs stay deterministic for a given seed
  • README, docs/insights.md and in-app text are updated where results or behavior changed
  • New dependencies, if any, are explained and THIRD_PARTY_LICENSES.md is regenerated
  • I've noted anything I couldn't test (for example a platform I don't have)
  • Every commit is signed off (git commit -s) under the Developer Certificate of Origin

Tested on macOS only, with Rust 1.99.0; CI runs Rust 1.100.0, so its Clippy is the first to see the new lines in src/bin/bench.rs. Coverage measured locally is 95.77% of lines, with src/bin/bench.rs at 100%. cargo deny --locked check and packaging/licenses.sh --check pass; no dependency changes.

By submitting this pull request, I agree that my contribution is dual licensed under MIT OR Apache-2.0, as described in CONTRIBUTING.md.

bench prints a coll column, tests/tools.rs requires it to be zero and the
README says "Zero vehicle overlaps in every run" under the controller
table. But bench never turned the safety check on: Sim::check_safety is
off in a new simulation, and collisions is only counted by
check_overlaps, which runs when it is on. The column could only be zero.

* bench now runs every strategy of a table with the check on. The
  sensor study, which reports no overlaps, still runs without it.
* The column is a total over the seeds instead of a mean, so that one
  overlap in one run can't round away to 0.
* A unit test in bench moves one vehicle on top of another and requires
  the run to count it. It fails with the check left off.
* run_scenario's comment says when the count it returns is real.
* docs/architecture.md: bench joins the list of places the check runs,
  which also lacked the green-wave test; the bench row says what the
  column holds; and the test list gains the new unit test.

With the check on, bench all 20 6 prints the same figures as before in
every column and 0 overlaps in all 32 rows (192 runs of 25 simulated
minutes), so the README's table and sentence stand as they are.

Signed-off-by: Jim Schreckengast <schreck@gmail.com>
* ACCESSIBILITY.md said everything the app measures can be produced from
  the command line. The tools print the average and 95th-percentile
  delay, throughput, stops, person delay, pedestrian waits and near
  misses, corridor trips and the safety counts. CO2 per vehicle, the
  level of service, the breakdowns, the charts and a selected vehicle's
  figures are only in the app (src/gui/mod.rs), and only the cooperative
  example prints the maximum delay. The heading and the introduction made
  the same claim and change with it.
* docs/assurance-case.md said a controller whose weights file is rejected
  starts from a fresh network. Since 75341af the loader skips a copy that
  doesn't load for the next one, ending with the built-in copy
  (src/control/weights.rs), and a Hybrid file whose network doesn't fit
  loads without it and keeps its base tuning (src/control/hybrid.rs).
* ROADMAP.md planned a keyboard way to place sensors. The app has no
  such tool: its tools are Select, Vehicle and Pedestrian, and sensors
  are set with checkboxes and sliders (src/gui/mod.rs).
* docs/architecture.md said every run of tests/cooperative.rs must show
  no monitor violations. Three tests run the crossing without that
  check, and the other runs must show more than that: no overlap between
  vehicle bodies, margins and times within a microsecond, and at least
  one crossing measured (safety_failures in tests/cooperative.rs).

Signed-off-by: Jim Schreckengast <schreck@gmail.com>
The picture in section 9.4 of docs/architecture.md still showed 0.5.1,
though the section says that pictures of parts that changed were retaken
with the code released as 0.6.0. Retaken with TS_ABOUT=1, as Appendix A
lists it, and cut at the same place: only the version number differs
from the earlier picture.

Signed-off-by: Jim Schreckengast <schreck@gmail.com>
@schreck61
schreck61 enabled auto-merge (rebase) November 15, 2026 06:04
@schreck61
schreck61 merged commit 7b84409 into main Nov 15, 2026
13 checks passed
@schreck61
schreck61 deleted the docs-audit-fixes branch November 15, 2026 06:11
This was referenced Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant