Repository navigation
fix(benchmark): start the dead-reckoning control from the true initial state (T2-4) - #136
Merged
Merged
Conversation
…l state (T2-4) DeadReckoning now takes an optional initial state (attitude, position, velocity) and the benchmark passes the true one for the dead_reckoning case only. The default start is unchanged. The control's unaligned error was a start offset plus drift; it is now drift. Regenerate the golden file and the two tables that quote the row, and drop a stale typed result from a config comment. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Telschow
force-pushed
the
fix/t2-4-dead-reckoning-start
branch
from
October 19, 2026 18:28
c32c40b to
e4090e5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The synthetic truth starts away from the origin, and
DeadReckoningstarted at the origin. The control's unaligned error therefore mixed a start offset with drift. In the benchmark record the unaligned ATE for this case was far above the start-aligned one.What changed
DeadReckoning.runtakes an optionalinitial(attitude, position, velocity). The default is the same as before: the identity pose at rest at the origin. Biases are ignored, because the integrator has no bias state. The type is a smallProtocol, because importingInitialStatefromeskfmade an import cycle thattests/test_no_import_cycles.pycatches.run_casepasses the true first pose (velocity zero, as the synthetic scene documents) for theDeadReckoningcase only. The ESKF cases are untouched.configs/benchmark.yaml: thedead_reckoningdescription now says it starts from the true state. The stale typed result in the vision comment (the one T1-5 left) is replaced by words.docs/scenarios.mdmirrors the new description (a test requires it).tests/test_injection_path.py: the noiseless-IMU test pinned a hand-typed number for the old behaviour. It now asserts that the noiseless control is far below the noisy one.The failing test (written first)
tests/test_estimators.py. On main the two new tests that passinitial=fail withTypeError: DeadReckoning.run() got an unexpected keyword argument 'initial'. The default-start test passes on main and still passes, so other callers are unaffected.Golden diff (case-confined)
Compared the old and new
tests/golden/benchmark.json, all 15 cases:dead_reckoningonly (55 values:ate,drift,headline,rpe_1s,stats,summary, and thedescription);config_sha256only. That is the file hash and moves because the config bytes changed. No case-levelconfig_hashmoved.Before regenerating, the golden test without
--updatefailed indead_reckoningonly (and, once the config was edited, inconfig_sha256only).Run comparison,
navkit run --outon main and on this branch withruntime_s,wall_s,realtime_factorandenvironmentstripped: the only differences are thedead_reckoningcase andconfig_sha256.Table rows, before and after
Read from the benchmark file. Other alignments of the same case are unchanged:
The unaligned figure now sits next to the start-aligned one, as it should. Other tables match (
check_doc_tables.py). No figure uses this row.The control still drifts
test_the_dead_reckoning_control_starts_from_the_true_state_and_still_drifts: the minimum position error is zero (the first epoch), the maximum exceeds the median, and the fitted growth slope and the final drift share are positive. In the record the growth slope was 0.0931 m/s before and 0.1279 m/s after, so the row still shows growing error.Removed stale comment
In
configs/benchmark.yaml: "and measures measurably worse (mean NEES 23 rather than 3)." is now "and measures measurably worse in mean NEES."Not changed, and one thing to look at
Not changed: the ESKF initialisation, any default, the row (it stays), the 4x preset.
Unexpected, not fixed here: the ESKF cases also call
runwithout aninitial, so they also start at the identity pose at the origin with the declared one-metre prior, while the truth starts about two metres away. The comment above the scoring step inbenchmark.pysays the filter is "initialised in the reference frame". That reads wrong for the current start position. I left it, as asked. It needs its own issue.Gates
ruff, format, mypy,
check_doc_tables.py,metrics.py --check(count updated),check_links.py, the golden test, and the full suite (1562 passed, 29 skipped, 2 xfailed).Closes #111
🤖 Generated with Claude Code