Skip to content

Fix GH-24170: read concatenated bzip2 streams - #24690

Open
kairosci wants to merge 3 commits into
php:masterfrom
kairosci:fix/gh-24170-bzip2-concatenated-streams
Open

kairosci wants to merge 3 commits into
php:masterfrom
kairosci:fix/gh-24170-bzip2-concatenated-streams

Conversation

@kairosci

Copy link
Copy Markdown
Contributor

Fixes #24170 by reading concatenated bzip2 members in the stream wrapper and bzopen(), while preserving the exported API and trailing-data behavior.

Adds regression coverage; bzip2 tests pass in debug ZTS and ASan/UBSan builds with applicable skips.

@kairosci

Copy link
Copy Markdown
Contributor Author

The macOS job fails during compilation because OpenSSL 4.1 deprecates ASN1_STRING_set() and ASN1_STRING_length(), and deprecation warnings are treated as errors. Tests never start. This is unrelated to the bzip2 changes and is already addressed separately by #24686.

Failing job: https://github.com/php/php-src/actions/runs/40205742173/job/121206203893

@anun333

anun333 commented Oct 30, 2026

Copy link
Copy Markdown

Quoted below: written with Claude, an LLM, as CONTRIBUTING.md asks us to mark. The builds and test runs behind it are on my machine; the commands to repeat them are under the fold.

Tested on today's master (1a72e5f2542) with this PR cherry-picked, debug builds with libbz2 1.0.8. Files made with bzip2 1.0.8, every output compared with bzcat's by length and sha256. "Stream readers" is file_get_contents('compress.bzip2://…'), bzopen($path) + bzread(), bzopen($fp) + bzread() and fread($h, 7) on the wrapper; they agreed in every row:

File bzcat stream readers, master stream readers, this PR bzdecompress(), both
one member 143,379 bytes match match match
two members 215,128 the first member only match the first member only
fifty members, as a parallel compressor writes them 143,379 2,868 match 2,868
an empty member between two 215,128 the first member only match the first member only
trailing trailing junk\n 143,379 match match match
trailing BZh9 and random bytes the first member, exit 2 the first member the first member the first member
the second member cut in half the first member, exit 2 the first member the first member the first member

ext/bz2/tests passes on both (31; 34 with the three new tests), and the three new tests fail on master without the fix.

Two things I noticed:

  • bzdecompress() still stops after the first member, so on these files the string API and the stream API now give different results. The UPGRADING entry names only the wrapper and bzopen(); it could say which applies to bzdecompress().
  • When a later member is damaged, as in the last row, bzcat exits with "Compressed file ends unexpectedly". With this PR, bzerror() afterwards reports UNEXPECTED_EOF (master says OK), but bzread() and fread() never return false and file_get_contents() returns the first member's 143,379 bytes without a warning. A single truncated member does make bzread() return false, so a cut-off multi-member file reads as complete unless the caller checks bzerror().
How this was tested

Linux x86_64, gcc 13.3, debug builds of origin/master and of it with this PR's commit cherry-picked:

./buildconf --force
./configure --enable-debug --disable-cgi --disable-phpdbg --disable-fileinfo --enable-pcntl --enable-zend-test --enable-fpm --with-curl --enable-mbstring --disable-mbregex --with-mysqli --with-pdo-mysql --with-openssl --with-bz2
make -j4

Two texts of 143,379 and 71,749 bytes; bzip2 -c a.txt; bzip2 -c b.txt into one file, and so on for each row (the fifty-member file is split -n 50 of the first text, each part compressed); the truncated row is the first member plus the first half of the second. Each reader loops until EOF (bzread($bz, 8192), or fread($h, 7)), stopping on false. Suites: TEST_PHP_EXECUTABLE=$PWD/sapi/cli/php sapi/cli/php run-tests.php -n -q -j4 ext/bz2/tests.

@kairosci

Copy link
Copy Markdown
Contributor Author

@ndossche As soon as you can, could you take a look at it ? Thanks

@ndossche
ndossche self-requested a review October 31, 2026 23:21
@kairosci
kairosci force-pushed the fix/gh-24170-bzip2-concatenated-streams branch from 0e63671 to ec720cd Compare November 1, 2026 13:27
@kairosci
kairosci requested a review from derickr as a code owner November 1, 2026 13:27
@kairosci
kairosci force-pushed the fix/gh-24170-bzip2-concatenated-streams branch from ec720cd to 353ac64 Compare November 1, 2026 13:28
@kairosci
kairosci force-pushed the fix/gh-24170-bzip2-concatenated-streams branch from 353ac64 to e8dba59 Compare November 2, 2026 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bzip2.decompress stream wrapper truncates valid bz2 files from pbzip2

2 participants