Skip to content

fix(benchmark): start the dead-reckoning control from the true initial state (T2-4) - #136

Merged
Telschow merged 1 commit into
mainfrom
fix/t2-4-dead-reckoning-start
Oct 19, 2026
Merged

Telschow merged 1 commit into
mainfrom
fix/t2-4-dead-reckoning-start

Conversation

@Telschow

Copy link
Copy Markdown
Owner

Why

The synthetic truth starts away from the origin, and DeadReckoning started at the origin. The control's unaligned error therefore mixed a start offset with drift. In the benchmark record the unaligned ATE for this case was far above the start-aligned one.

What changed

  • DeadReckoning.run takes an optional initial (attitude, position, velocity). The default is the same as before: the identity pose at rest at the origin. Biases are ignored, because the integrator has no bias state. The type is a small Protocol, because importing InitialState from eskf made an import cycle that tests/test_no_import_cycles.py catches.
  • run_case passes the true first pose (velocity zero, as the synthetic scene documents) for the DeadReckoning case only. The ESKF cases are untouched.
  • configs/benchmark.yaml: the dead_reckoning description now says it starts from the true state. The stale typed result in the vision comment (the one T1-5 left) is replaced by words.
  • docs/scenarios.md mirrors the new description (a test requires it).
  • tests/test_injection_path.py: the noiseless-IMU test pinned a hand-typed number for the old behaviour. It now asserts that the noiseless control is far below the noisy one.

The failing test (written first)

tests/test_estimators.py. On main the two new tests that pass initial= fail with TypeError: DeadReckoning.run() got an unexpected keyword argument 'initial'. The default-start test passes on main and still passes, so other callers are unaffected.

  • first-epoch position and attitude error are zero with the true start, and the error stays bounded;
  • an initial velocity is carried through;
  • no initial state still starts at the origin.

Golden diff (case-confined)

Compared the old and new tests/golden/benchmark.json, all 15 cases:

  • cases that changed: dead_reckoning only (55 values: ate, drift, headline, rpe_1s, stats, summary, and the description);
  • other 14 cases: identical;
  • top level: config_sha256 only. That is the file hash and moves because the config bytes changed. No case-level config_hash moved.

Before regenerating, the golden test without --update failed in dead_reckoning only (and, once the config was edited, in config_sha256 only).

Run comparison, navkit run --out on main and on this branch with runtime_s, wall_s, realtime_factor and environment stripped: the only differences are the dead_reckoning case and config_sha256.

Table rows, before and after

File Row Before After
README.md, docs/results.md Dead reckoning (no aiding), ATE RMSE (unaligned) 2.359 1.248

Read from the benchmark file. Other alignments of the same case are unchanged:

Alignment Before After
none 2.359 1.248
rigid 0.838 0.838
rigid_start 1.231 1.231
similarity 0.828 0.828

The unaligned figure now sits next to the start-aligned one, as it should. Other tables match (check_doc_tables.py). No figure uses this row.

The control still drifts

test_the_dead_reckoning_control_starts_from_the_true_state_and_still_drifts: the minimum position error is zero (the first epoch), the maximum exceeds the median, and the fitted growth slope and the final drift share are positive. In the record the growth slope was 0.0931 m/s before and 0.1279 m/s after, so the row still shows growing error.

Removed stale comment

In configs/benchmark.yaml: "and measures measurably worse (mean NEES 23 rather than 3)." is now "and measures measurably worse in mean NEES."

Not changed, and one thing to look at

Not changed: the ESKF initialisation, any default, the row (it stays), the 4x preset.

Unexpected, not fixed here: the ESKF cases also call run without an initial, so they also start at the identity pose at the origin with the declared one-metre prior, while the truth starts about two metres away. The comment above the scoring step in benchmark.py says the filter is "initialised in the reference frame". That reads wrong for the current start position. I left it, as asked. It needs its own issue.

Gates

ruff, format, mypy, check_doc_tables.py, metrics.py --check (count updated), check_links.py, the golden test, and the full suite (1562 passed, 29 skipped, 2 xfailed).

Closes #111

🤖 Generated with Claude Code

…l state (T2-4)

DeadReckoning now takes an optional initial state (attitude, position,
velocity) and the benchmark passes the true one for the dead_reckoning case
only. The default start is unchanged. The control's unaligned error was a start
offset plus drift; it is now drift. Regenerate the golden file and the two
tables that quote the row, and drop a stale typed result from a config comment.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@Telschow
Telschow force-pushed the fix/t2-4-dead-reckoning-start branch from c32c40b to e4090e5 Compare October 19, 2026 18:28
@Telschow
Telschow merged commit 21d0711 into main Oct 19, 2026
20 checks passed
@Telschow
Telschow deleted the fix/t2-4-dead-reckoning-start branch October 19, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Start the dead-reckoning control from the true initial state

1 participant