Skip to content

Fix read_to_iter crash with buffer-protocol input - #355

Open
CAOShurong wants to merge 1 commit into
indygreg:mainfrom
CAOShurong:codex/fix-compression-iterator-buffer
Open

CAOShurong wants to merge 1 commit into
indygreg:mainfrom
CAOShurong:codex/fix-compression-iterator-buffer

Conversation

@CAOShurong

Copy link
Copy Markdown

Fixes #303.

Buffer-protocol input leaves readResult null because the iterator holds the
input through its Py_buffer. If a small output buffer makes compression resume
on the next iteration, consuming the remaining input calls Py_DECREF on that
null pointer. Use Py_XDECREF, matching the initial-feed and deallocation paths.

The regression uses 131,073 bytes of deterministic incompressible input with
small output chunks. It covers bytes, bytearray, memoryview and BytesIO, and
checks the complete frame, decompressed data, chunk limits and repeated EOF.

Validated on Windows x64 / CPython 3.13.1 using non-editable source and
sdist-built wheels, with repeated focused/full tests on C and CFFI backends:
C 249 passed / 44 skipped; CFFI 225 passed / 68 skipped. All 292 prior test
outcomes per backend are unchanged. The original native crash, mmap input,
buffer lifetime and error paths were also checked; related iterator fuzzing,
Ruff, mypy, builds and package checks passed. Other platforms and Rust were not
tested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Segfault in ZstdCompressorIterator_iternext with buffer-protocol data

1 participant