Skip to content

Pedestrian data format docs, faster heading/speed derivation, log window fix, per-session log files - #24

Merged
sir306 merged 6 commits into
mainfrom
dev
Oct 22, 2026
Merged

sir306 merged 6 commits into
mainfrom
dev

Conversation

@sir306

@sir306 sir306 commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Promotes four commits from dev to main.

What changes

  • Pedestrian data format docs (465510ea). New docs/PEDESTRIAN-DATA-FORMAT.md, docs/HDF5-EXPORT-GUIDE.md and docs/EXODUS-EXPORT.md, plus docs/examples/pedestrian-hdf5/. The examples are a C writer for the bundled HDF5 library, a Python writer and CSV converter, and a validator that mirrors the importer's checks. The heading convention and agent direction were checked in the app with a -game capture.
    Also fixes doc links still pointing at the old Hdf5DataPlugin name.
    Known gap: a WKT outline is mirrored against Mobius-format agents. It is documented, not fixed.
  • Heading/speed derivation in one pass (53ba807a). Files without rotation/speed (every Jülich file, and positions-only Mobius HDF5) were derived in roughly agents × samples. Results are bit-identical on the Jülich sample, and a 1.9 M-sample file goes from 12.0 s to 0.043 s.
  • Log window and FrameGrabber (a3a08bad).
    • The log window rebuilt and re-laid out its whole 1000-line backlog for every new line. It is now a virtualized list: one line per frame goes from 30.2 ms to about 0.3 ms, and a 3,600-line burst from 97.7 ms to about 3 ms.
    • FrameGrabber no longer logs a failure on every frame while the viewport is 0x0. In a headless run that drops from 1,805 lines to 1.
  • One custom log file per session (70b558d0). MobiusCustomLog.txt took every session's lines forever (115 MB on one dev machine) and mixed processes running side by side. Each session now writes MobiusCustomLog_<yyyy.mm.dd-hh.mm.ss>.txt, starting with a === Mobius session started <time> (pid N) === line. The newest 20 are kept, and the old shared file is left alone. The settings window names the current file, and the macOS beta readme uses the new name.

Testing

  • The Game target compiles and links; the Editor target builds.
  • ProjectMobius.SimData. passes 13/13, including the new DerivedMotionParity.
  • New tests: Mobius.RenderedTiming.LogWindow.* (needs a renderer) and in-game tests for the FrameGrabber spam and the log cache order.
  • ProjectMobius.Logger.SessionLogFilePerProcess checks the per-session file name, its start line, and that a file holds one session. By hand: two runs gave two files, each with its own start line, and 25 seeded old session files were trimmed to the newest 20.
  • The doc examples were run end to end: the C example built against standalone HDF5 2.0.0, and every output passes the validator.

Not checked: the log window in the dark theme or while resizing, and the Linux/macOS HDF5 build commands.

AI-assisted with Claude Code; all changes reviewed, tested, and verified by a human maintainer in line with this repository's AI-tooling policy.

🤖 Generated with Claude Code

sir306 and others added 6 commits October 22, 2026 12:21
Files without rotation/speed members (every Juelich dataset, and Mobius HDF5
written positions-only) had heading and speed derived by scanning every
timestep's sample list once per agent, then again to write the result back:
roughly agents x samples. A 1.9 M-sample positions-only file took 12.0 s to
derive.

Walk the timesteps once instead, keeping a previous-sample pointer per agent.
The rules are unchanged (look-ahead heading, hold while the step is under
0.1 cm, last frame carries the previous value, fewer than 2 samples skipped),
and the derived values are bit-identical on the Juelich 030_c_56_h0 sample.
The progress bar now advances by timesteps rather than agents, and
FMobiusImportTimings gains DeriveSeconds.

Timing, old -> new: 360 k samples 0.300 s -> 0.0064 s; 719 k samples
2.022 s -> 0.0121 s; 1.90 M samples 12.02 s -> 0.043 s.

Tests: ProjectMobius.SimData.DerivedMotionParity compares against a verbatim
copy of the old loops (real Juelich import plus edge cases);
Mobius.Timing.DerivedMotionScaling is opt-in.

AI-assisted with Claude Code; all changes reviewed, tested, and verified by a human maintainer in line with this repository's AI-tooling policy.
With the log window open, every incoming line rebuilt the whole backlog (up to
1000 lines) into one string and re-laid out a single word-wrapped text block,
and a flush delivered its lines one at a time. Once the backlog was full, each
frame that received a line cost ~30 ms; a 3,600-line heatmap burst ~98 ms. The
log file itself is only ever appended to, never read back.

- Log window: a virtualized SListView with one row per line, so only visible
  rows are laid out and appending is O(1); old lines are trimmed in batches
  (1000 stays the cap). It follows the newest line unless the user scrolls up.
  Close/destroy ordering is unchanged.
- Feedback log cache: a ring buffer instead of RemoveAt(0);
  GetCachedLogLines() still returns oldest-first.
- FrameGrabber: log a failure only when its reason changes and retry every
  0.5 s instead of every frame. A 0x0 viewport (headless runs) had produced
  over a million identical log lines. Screenshot capture still initialises
  immediately and the resize re-init is unchanged.

Measured with the window open and a full backlog: one line per frame
30.2 ms -> ~0.3 ms; 250-line flush 34.7 ms -> ~2 ms; 3,600-line flush
97.7 ms -> ~3 ms. FrameGrabber failure lines in a headless run: 1,805 -> 1.

Tests: Mobius.RenderedTiming.LogWindow.* (needs a renderer, kept out of
Mobius.Timing.) and in-game tests for the FrameGrabber spam and cache order.

AI-assisted with Claude Code; all changes reviewed, tested, and verified by a human maintainer in line with this repository's AI-tooling policy.
External evacuation and crowd modellers need to export trajectories Mobius
can load, and the HDF5 reader decodes most fields by position with no type
conversion, so a wrong dtype loads garbage instead of failing.

- docs/PEDESTRIAN-DATA-FORMAT.md: field-by-field Mobius HDF5 and JSON layout,
  units, axes, time model, heading convention (degrees CCW from +x, checked in
  the app), the rules the importer relies on, and a checklist. Notes that WKT
  geometry is mirrored against Mobius-format agents.
- docs/HDF5-EXPORT-GUIDE.md: get the files (sparse clone), write with Python
  (h5py) or with the bundled HDF5 C library built standalone, validate, open
  in Mobius, troubleshoot.
- docs/EXODUS-EXPORT.md: short buildingEXODUS route via a long-format CSV.
- docs/examples/pedestrian-hdf5/: C writer + CMake, mobius_h5_writer.py
  (library, from-csv with linear/hold resampling and floor heights, demo) and
  validate_mobius_h5.py, which mirrors the importer's checks.
- Fix links to the renamed plugin (Hdf5DataPlugin -> MobiusDataImporter),
  correct the ThirdParty README's claim that the HDF5 install tree is tracked,
  and mirror the format page into packaged builds.

AI-assisted with Claude Code; all changes reviewed, tested, and verified by a human maintainer in line with this repository's AI-tooling policy.
MobiusCustomLog.txt took every session's lines forever (115 MB on one dev
machine) and interleaved processes running side by side, such as the editor
and a headless test run.

Each session now writes MobiusCustomLog_<yyyy.mm.dd-hh.mm.ss>.txt, with a _2
suffix if two start in the same second. Its first line is
"=== Mobius session started <time> (pid N) ===", so a file is easy to match
to a run. The newest 20 session files are kept; the old MobiusCustomLog.txt
is left alone. The settings window now names the file this session writes,
and the macOS beta readme points at the new name.

Tests: ProjectMobius.Logger.SessionLogFilePerProcess checks the file name,
the start line and that a file holds one session. Checked by hand: two runs
gave two files, each with its own start line, and 25 seeded old session
files were trimmed to the newest 20.

AI-assisted with Claude Code; all changes reviewed, tested, and verified by a human maintainer in line with this repository's AI-tooling policy.
@sir306 sir306 changed the title Pedestrian data format docs, faster heading/speed derivation, log window fix Pedestrian data format docs, faster heading/speed derivation, log window fix, per-session log files Oct 21, 2026
@sir306
sir306 merged commit 0891fb9 into main Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant