Skip to content

rsync 3.5.1: read the protocol 33 block stat, and stop truncating byte totals - #45

Merged
ralyodio merged 1 commit into
mainfrom
worktree-rsync-351-support
Sep 22, 2026
Merged

ralyodio merged 1 commit into
mainfrom
worktree-rsync-351-support

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

What

rsync 3.5.1 shipped 2026-09-21 with protocol 33, which adds one line to --stats:

Number of 4 KiB logical blocks touched: 857

printed between Literal data and Matched data, and only when both ends negotiate 33.

DiskPush already ran fine on 3.5.1 — every capability check is a lower bound, and the parser passes lines it does not recognise through to the log — but that number was being dropped on the floor. This reads it, and covers both the version distros ship today (3.4.x) and the latest.

  • RsyncStats.logicalBlocksTouched, parsed from the real label, verbatim from rsync's main.c.
  • A blockStats capability taken from the negotiated protocol rather than the version string, falling back to the version only for a banner that prints no protocol line.
  • Intersected like every other capability, so a mixed 3.5/3.4 pair reports null rather than a number only one end can produce.
  • The CLI prints Blocks touched: 856 x 4 KiB when it is available and stays silent when it is not. stats is now in the --json payload.

The bug this turned up

Verifying against a real 3.5.1 build exposed something unrelated to 3.5.1 and older than it. rsync prints byte totals with human_num, so under --human-readable — which DiskPush passes by default — the output is Total bytes sent: 3.50M. The stats patterns only read [\d,]+, so that parsed as 3.

Every byte field was affected on every rsync version:

field was now
totalBytesSent 3 3500000
literalBytes 3 3500000
speedup null 1

Byte fields now read the unit suffix back through the same conversion the progress line already used. Counts keep their exact comma_num parsing, so the new block stat is unaffected either way.

I fixed this rather than only reporting it because this PR puts stats into the --json output, and publishing knowingly wrong numbers in a new contract seemed worse than the extra diff. Happy to split it out if you would rather.

Verification

Not just unit tests — built rsync 3.5.1 from source and ran both binaries for real.

3.5.1: protocol 33 | blockStats true  | mkpath true | secludedByDefault true
3.4.1: protocol 32 | blockStats false | mkpath true | secludedByDefault true
mixed ->  protocol 32 | blockStats false

Parsed from genuine --stats output:

3.5.1: {"filesTotal":5,"filesTransferred":3,"totalBytesSent":3500000,"literalBytes":3500000,"logicalBlocksTouched":857,"speedup":1}
3.4.1: {"filesTotal":5,"filesTransferred":3,"totalBytesSent":3500000,"literalBytes":3500000,"logicalBlocksTouched":null,"speedup":1}
  • 723 tests pass on stock 3.4.1.
  • The full live suite (12 tests) passes against real 3.5.1, including the resume test — which exercises exactly the partial-directory validation 3.5.1 tightened. --partial-dir resume is unaffected.
  • CLI end to end on both: the line appears on 3.5.1, is absent on 3.4.1.
  • The new live test is version-aware, so it asserts the right branch on whatever rsync the machine has.

Note for a follow-up

3.5.1 also changed --files-from paths to be treated as operator-supplied rather than transfer-root-relative. DiskPush passes --files-from straight through, so the semantic shift is upstream's, but it could change path resolution for anyone using that flag on 3.5.1. Not addressed here.

🤖 Generated with Claude Code

…e totals

rsync 3.5.1 bumped the protocol to 33 and added a line to --stats:

    Number of 4 KiB logical blocks touched: 857

printed between "Literal data" and "Matched data", and only when both ends
negotiate 33. Nothing gated 3.5.1 out before this -- every capability check is
a lower bound and the parser drops lines it does not recognise -- but the new
number fell through to the log instead of being read.

Parse it into RsyncStats.logicalBlocksTouched, gate it on a new blockStats
capability taken from the negotiated protocol rather than the version string,
and intersect it like the rest so a mixed 3.5/3.4 pair reports null instead of
a number only one end can produce. The CLI prints the line when it is there and
stays silent when it is not.

Verifying that against a real 3.5.1 build turned up a separate bug. rsync
prints byte totals with human_num, so under --human-readable -- which DiskPush
passes by default -- "Total bytes sent: 3.50M" was being read by a [\d,]+
pattern as 3. Literal data, matched data, both byte totals and speedup were all
wrong on every version, not just 3.5.1. Byte fields now read the unit suffix
back through the same conversion the progress line already used; counts keep
their exact comma_num parsing.

Checked against real binaries on both sides: stock 3.4.1 and a 3.5.1 built from
source. The full live suite passes against each, including the resume test that
exercises the partial-dir validation 3.5.1 tightened.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ralyodio
ralyodio merged commit 3d92cc2 into main Sep 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant