Repository navigation
fix(trace): write bundle files with LF regardless of platform - #37
Open
how2how2how2-arch wants to merge 1 commit into
Open
how2how2how2-arch wants to merge 1 commit into
how2how2how2-arch wants to merge 1 commit into
Conversation
Bundle files are hashed as bytes (trace/integrity.py::_hash_file, export/receipt.py::bundle_digest), but the three writers in storage.py used text mode without newline="", so on Windows "\n" became "\r\n" and the same trace produced different bytes and a different digest per platform. Pass newline="" in _write_manifest, _write_events and _write_redactions, and add a .gitattributes rule so a Windows checkout cannot rewrite the committed bundles either (git's default core.autocrlf would corrupt the recorded digests before any test runs). The rule is narrow: **/*.sftrace/** matches nested bundle files, while *.sftrace/* and *.sftrace/**/* do not, and no committed file changes in the working tree as a result. Adds three tests, documented for what each one can and cannot show: the no-CRLF assertion fails on Windows if newline="" is dropped but is a no-op on POSIX, the digest-sensitivity test describes the stake rather than the fix, and the gitattributes test fails on every platform when the rule is removed.
Owner
Verified
Why it matters
The new test Small changes
Also please rebase onto |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #36
Summary
Makes bundle files byte-identical on every platform, so the digests recorded in
integrity.jsonmean the same thing everywhere.Two parts, because there are two ways the bytes can change:
_write_manifest,_write_eventsand_write_redactionsintrace/storage.pynow passnewline="". Without it, text mode on Windows turned\ninto\r\nfor the same trace..gitattributesrule,**/*.sftrace/** text eol=lf. Without it, git's defaultcore.autocrlf=trueon Windows rewrites the 35 committed bundle files before any test runs, so the recorded digests are already wrong on disk.+106/-1across three files.Why this matters
Bundle digests are SHA-256 over file bytes —
trace/integrity.py::_hash_file,export/receipt.py::bundle_digest/file_sha256. So line endings are not abstracted away; they are part of what is hashed. Measured on the committedlegacy_v01.sftraceby rewriting its files as CRLF in a temp copy:And the user-visible version, on a bundle that carries an
integrity.json(8 of the 35 do):Note for anyone reading this as "it still works":
events.jsonlis JSONL, so a CRLF file still parses. The breakage is in the digests, not in parsing — verified--verify-integrityis opt-in, which is exactly why Linux CI never noticed.On the
.gitattributespattern**/*.sftrace/**is the form that works;*.sftrace/*and*.sftrace/**/*do not match nested files. I checked rather than assumed:The rule is deliberately narrow.
git statusshows no bundle file modified after adding it, so there is no renormalisation commit hiding in here, and non-bundle files keep git's default behaviour.Tests
Three tests in
tests/unit/test_storage.py, and I would rather state what each one can and cannot prove than let the count imply more:test_saved_bundle_files_contain_no_crlfnewline=""is dropped. On POSIX, newline translation is a no-op, so it passes before and after here — a guard for the platform CI can reach but this machine cannottest_bundle_digest_changes_when_a_file_gains_crlftest_gitattributes_pins_bundle_files_to_lf.gitattributesand watching it go red, then restoring it and watching it passI deliberately did not fake a stronger writer guard. Simulating Windows by patching
os.linesepdoes not reproduce the translation (CPython decides it elsewhere), so an assertion built that way would pass regardless of the defect — the failure mode where a test certifies something it cannot see.Testing
Committed bundles still verify after the writer change, which is the check that this did not disturb existing data:
Checklist
newline=""is commented at each writer, and.gitattributescarries the same explanation; no user-facing docs change