Skip to content

docs: say how to regenerate the front-end measurements, and that they reproduce (T1-10) - #133

Merged
Telschow merged 1 commit into
mainfrom
docs/t1-10-front-end-provenance
Oct 19, 2026
Merged

Telschow merged 1 commit into
mainfrom
docs/t1-10-front-end-provenance

Conversation

@Telschow

Copy link
Copy Markdown
Owner

What this does

Adds a section "Reproducing these rows" to docs/real_front_end.md. It names the three files, gives the command that regenerates each, and states what was checked, labelled "measured once on one machine, not checked by CI". It says nothing about any earlier mismatch. Two tests pin it. The generated test count is updated.

Evidence: every front-end row reproduces on main

Main at a79b935. Each window was run with the wide study's own call, measure(seq, (1, 4), window=(start, start + 30)). The original window used the original call, measure(seq) with all five gaps, with only its 600 frames on disk. Outputs went to the scratchpad. The raw data was read through symlinks and nothing was written into it.

Window Frames on disk Pairs compared Only in committed / only in new Columns that differ Largest difference
MH_01 original, 60 s (all five gaps) 600 2950 0 / 0 none of 13 0
MH_01, 90 s only the window's 1191 0 / 0 none of 13 0
MH_01, 90 s all of the sequence's 1191 0 / 0 none of 13 0
MH_02, 105 s 600 1192 0 / 0 none of 13 0
MH_03, 80 s 600 1186 0 / 0 none of 13 0
MH_04, 60 s 600 1192 0 / 0 none of 13 0
MH_05, 30 s 600 1192 0 / 0 none of 13 0

The frame-set hypothesis is refuted (MEASUREMENT). measure() filters frames to the window before tracking, and the two MH_01 runs gave identical rows.

Step 2: the two replays

File Command Result
docs/data/vo_wide.csv navkit euroc vo --root <euroc> --from-csv docs/data/vo_errors.csv --wide-replay --wide-csv <scratch> equal, byte for byte
docs/data/vo_clone_check.csv navkit euroc vo --root <euroc> --from-csv docs/data/vo_errors.csv --clone --clone-csv <scratch> equal, byte for byte

The committed CSVs were not touched.

What the page does not claim

The --sequence ... --csv and --wide commands were not run end to end. The front-end rows were checked by calling the measurement function they call, window by window. The page says so.

Audit finding F20 was a mistake

F20 and the premise of #106 came from one mismatch seen in T1-1b, raised without a like-for-like run. The like-for-like runs above do not show a mismatch. The earlier mismatch is unexplained. I did not look for the cause beyond refuting the frame-set hypothesis.

Tests

tests/test_vo_page.py, two new tests, written first and shown failing without the section:

  • the section exists, names vo_errors.csv, vo_clone_check.csv and vo_wide.csv, and carries the exact label;
  • every --flag in the section's navkit euroc vo commands is an option of the parser, and --wide-replay and --clone are among them. This needs no data.

Gates: ruff, format, mypy, check_doc_tables.py, metrics.py --check (run --write: the count went from 1586 to 1588 in four files), check_links.py, the three page and error test files, and the full suite (1557 passed, 29 skipped, 2 xfailed).

Left alone

No ADR notes. No change to src/, the CSVs, the 4x preset, the README or docs/status.md.

Closes #106

🤖 Generated with Claude Code

… reproduce (T1-10)

Add a "Reproducing these rows" section to docs/real_front_end.md with the
commands and the result of one like-for-like check. Add tests that pin the
section, the three file names, the one-off label and the CLI flags. Update the
generated test count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Telschow
Telschow force-pushed the docs/t1-10-front-end-provenance branch from 85671c8 to 5f26dc2 Compare October 19, 2026 16:31
@Telschow
Telschow merged commit f7dd35c into main Oct 19, 2026
20 checks passed
@Telschow
Telschow deleted the docs/t1-10-front-end-provenance branch October 19, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the visual front-end measurement reproducible from the committed code

1 participant