Skip to content

lib: preserve MxDSSelectAction list extension through Read/Write - #48

Merged
foxtacles merged 3 commits into
masterfrom
selectaction-roundtrip
Jul 25, 2026
Merged

foxtacles merged 3 commits into
masterfrom
selectaction-roundtrip

Conversation

@foxtacles

Copy link
Copy Markdown
Member

MxDSSelectAction child lists ('Act_'/'RAND') replace the regular list count with a null-terminated selector string (e.g. "RANDOM_3"), the actual count, and one null-terminated value string per child. Read previously consumed this extension only for the description stream (assuming single-character values) and Write always emitted a bare child count, corrupting select actions in rewritten files.

This retains the raw extension bytes on the object (Object::list_extension_) and re-emits them verbatim on write. CalculateMaximumDiskSize accounts for the extra bytes.

Verified byte-exact roundtrip of the extension on the four retail SI files that use it (COPTER.SI, DUNECAR.SI, JETSKI.SI, RACECAR.SI), and full Read→Write→Read object-tree equality on all 26 retail 1.1 English SI files.

MxDSSelectAction child lists ('Act_'/'RAND') replace the regular list
count with a null-terminated selector string (e.g. "RANDOM_3"), the
actual count, and one null-terminated value string per child. Read
previously consumed this extension only for the description stream
(assuming single-character values) and Write always emitted a bare
child count, corrupting select actions in rewritten files.

Retain the raw extension bytes on the object and re-emit them verbatim
on write. Verified byte-exact roundtrip of the extension on the four
retail SI files that use it (COPTER.SI, DUNECAR.SI, JETSKI.SI,
RACECAR.SI).
Three defects conspired to corrupt rewritten SI files once object sizes
changed from the source file (any replacement shifts every downstream
chunk):

- Object::CalculateMaximumDiskSize did not account for RIFF odd-size
  padding of nested chunks, so the pre-MxSt padding projection could
  undershoot the actual object tree by one byte per odd-sized chunk.
- The projection did not include the MxDa LIST header that follows the
  object tree, so that header could land within the last bytes of a
  buffer, where both the game and Interleaf::Read skip it (chunks must
  not straddle buffer boundaries), desyncing the entire parse.
- WritePadding silently no-ops for gaps smaller than a pad_ chunk;
  WritePaddingIfNecessary now fills such gaps with zeroes instead.

Also preserve the MxOf object count field through Read/Write instead of
rewriting it with the top-level slot count (retail files store the total
object count there; e.g. ISLE.SI says 3708 while having 1186 slots).
The runner image's newer CMake resolves the relative root against the
app/ subdirectory when find_package(FFMPEG) runs, so configuration
fails with the include paths reported under app/. Resolve the path in
the workflow instead of relying on version-specific behavior.
@foxtacles

Copy link
Copy Markdown
Member Author

Regression verification done against retail 1.1 English data and the isle-portable asset pipeline:

  • From-scratch writer unchanged: isle-portable's asset_generator (widescreen/hdmusic/badend/rabbits packs) produces byte-identical output built against master's lib vs this branch.
  • Retail rewrite fidelity: Read→Write→Read over all 26 retail SI files compares equal on every object field and every data chunk (including the select-action extension, which now survives verbatim — byte-checked in the four Build SIs that use RAND lists). MxOf count round-trips retail-exact (e.g. ISLE.SI 3708; the engine reads this field to size and fill its offset table — mxdsfile.cpp reads m_lengthInDWords dwords — so preserving it reproduces retail behavior exactly; the previous slot-count value was the deviation).
  • Byte-consumption parity: for retail RAND lists (single-character values) the new null-terminated-string parse consumes exactly the same bytes as the old u16 model, so parse alignment is unchanged; for multi-character values or Act_ lists the old model desynced and the new one follows the engine's deserializer layout.
  • Boundary safety: a structural audit (headers must not straddle buffer boundaries, non-split MxCh must not cross) reports zero issues over retail files rewritten with size-changed objects — the scenario that corrupted files before the padding fixes. CalculateMaximumDiskSize only grew, so padding can only become more conservative.

One caveat worth knowing: if an object is added to or removed from a file, the preserved MxOf count becomes stale by that delta (previously it was rewritten with the top-level slot count, which was also not the retail semantic). Fresh files with no count read still fall back to the child count.

@foxtacles
foxtacles merged commit abb6d47 into master Jul 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant