Skip to content

2.7.18.16 Release: 17 security fixes, bundled Expat 2.9.0 - #113

Merged
icanhasmath merged 43 commits into
2.7from
2.7.18.16
Oct 28, 2026
Merged

icanhasmath merged 43 commits into
2.7from
2.7.18.16

Conversation

@icanhasmath

Copy link
Copy Markdown

Release PR for 2.7.18.16. It merges the 2.7.18.16 integration branch into 2.7. Tracking: Linear PROD-1588.

Every change below was reviewed and merged into 2.7.18.16 as its own PR. The only new commit here is the release commit (60c3e21), which sets PY_VERSION to 2.7.18.16 and adds Misc/NEWS.d/2.7.18.16.rst. The branch also merged 2.7 after #91, so it contains 2.7.18.15 and the PR merges cleanly.

Security fixes

PR Fix CVE
#92 tarfile streaming-mode seek loops at EOF CVE-2026-11972
#93 csv.Sniffer quadratic regex on quoted fields CVE-2026-18503
#94 ftplib.ftpcp() trusts the PASV host CVE-2026-8328
#95 httplib Content-Length / chunk-size validation CVE-2026-103995
#96 httplib limits after 100 Continue, plus the interim-response and trailer limits CVE-2021-3737, GHSA-w4q2-g22w-6fr4
#97 SimpleHTTPServer // open redirect CVE-2021-28861
#98 mimetypes reads fixed paths on Windows CVE-2024-3220
#99 pyexpat stack overflow on deeply nested content models CVE-2026-4224
#100 bz2 decompressor and BZ2File reused after an error CVE-2026-9669
#101 ElementPath index predicates are O(n²) CVE-2026-6879
#102 urllib2 HTTPPasswordMgr sends https credentials over http CVE-2026-15806
#103 HTMLParser incremental feed is quadratic CVE-2026-15308
#104 bundled Expat 2.6.4 → 2.9.0 30 Expat CVEs
#105 16-byte random Expat hash salt CVE-2026-7210
#106 ssl SNI callback use-after-free CVE-2026-19445
#107 unicodedata.normalize() quadratic canonical ordering CVE-2026-3276
#108 os.mkdir(path, 0700) / tempfile.mkdtemp ACL on Windows CVE-2024-4030
#111 review follow-ups: _socket debug-build link, SNI None reference, sddl.h, test_ssl without mock —

Testing

Linux x86_64, built against the production OpenSSL 1.1.1w. I ran the full suite (regrtest -j6 -w -uall,-largefile,-cpu) on the combined branch after #92–#108 merged:

  • Passing: 363 tests.
  • Failures, all matching 2.7.18.15 or my test setup:
    • test_ftplib fails identically on 2.7.18.15;
    • test_ssl's only failure, test_context_setget, needs the network;
    • test_socket fails on 2.7.18.15 too, because the module never imports fcntl;
    • test_import passes when run without my PYTHONDONTWRITEBYTECODE setting.
  • After [2.7.18.16] Review follow-ups: _socket debug link, SNI None ref, sddl.h #111 and the version bump: test_ssl, test_os, test_tempfile, test_posix, test_sys, test_platform and test_sysconfig were re-run. sys.version reports 2.7.18.16.

Windows x64: not built here. Several fixes are Windows-only (#98, #104's _elementtree project change, #108), so this needs a Windows build before tagging.

Not in this release

  • stringprep (CVE-2026-17084): needs a regenerated Unicode database for an IDNA 2003 edge case.
  • Morsel.js_output (CVE-2026-6019): skipped on 3.9/3.10 too.
  • CVE-2023-36632 and CVE-2026-3479: disputed.
  • Windows errno and distutils compiler detection: carried forward from 2.7.18.10 dev #60 in PROD-1626.
  • test_socket: never imports fcntl or multiprocessing, so it doesn't run; a separate test-only fix.

After merge: tag 2.7.18.16 on the merge commit and publish the release. Then add the platform inventory version and the CVE-scanner override rows.

icanhasmath and others added 30 commits October 28, 2026 00:50
Backport of python/cpython pythonGH-151982 (3.13 3f031d4).

In streaming mode (mode="r|"), _Stream.seek() skipped forward by reading
the declared number of blocks even after the stream hit EOF.  A member
header that declares a huge size (e.g. 1 << 64 via a GNU/PAX header)
made getmembers() spin for an effectively unbounded time.  seek() now
stops reading once read() returns no data, and the following header read
raises ReadError("unexpected end of data").

2.7.18.x: xrange() loop; the test loops over GNU_FORMAT and PAX_FORMAT
instead of using support.subTests, and lives in StreamReadTest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…VE-2026-18503)

Backport of python/cpython pythongh-98820 (3.12 063d455).

Sniffer._guess_quote_and_delimiter() matched a quoted field body with a
lazy ".*?", which scans to the end of the sample from every start
position: quadratic in the sample size.  An attacker-controlled sample
passed to Sniffer.sniff() could burn significant CPU.  The body of a
quoted field now ends at the first quote that is not doubled, as it does
for a csv reader.

2.7.18.x: Python 2.7's re module has no possessive quantifiers (*+ is
3.11+), so the body is matched atomically with the equivalent
lookahead + backreference idiom (?=(?P<body>...*))(?P=body).  A
differential fuzz of 20000 random samples under Python 3.12 found the
2.7 pattern and upstream's possessive pattern produce identical matches.
assertRaisesRegexp instead of assertRaisesRegex in the test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…026-8328)

Backport of python/cpython pythongh-87451 (3.15 5dadc64).

2.7.18.x already carries the CVE-2021-4189 fix in FTP.makepasv(): it
ignores the IPv4 address a server advertises in its PASV reply and uses
the control connection's peer address unless
trust_server_pasv_ipv4_address is set.  ftpcp() still called parse227()
directly and passed the server-supplied host to target.sendport(), so a
malicious source server could point the target's data connection at an
arbitrary host:port (SSRF).  ftpcp() now applies the same rule.

2.7.18.x: no unittest.mock in 2.7, so TestFtpcpSecurity uses small fake
FTP/socket classes; % formatting instead of f-strings; the class is
added to test_main().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…2026-103995)

Backport of python/cpython pythonGH-150752 and pythonGH-159156 (pythongh-150751; the
3.10.22.1 ports are 833a27a and dbefda3).

HTTPResponse parsed Content-Length with int() and chunk sizes with
int(_, 16).  Both accept input the RFC 9112 grammar forbids: a sign
("+5", "-5"), surrounding whitespace, and for base 16 an "0x" prefix.
A peer that disagrees with httplib about message framing can use this
for response splitting / desynchronisation.  Values are now matched
against Content-Length = 1*DIGIT and chunk-size = 1*HEXDIG first; an
invalid Content-Length is ignored (the body is read to close) and an
invalid chunk size closes the connection with IncompleteRead.  Optional
whitespace (SP/HTAB) around the Content-Length value is still allowed,
per RFC 9112 section 5.1.

2.7.18.x: re has no fullmatch, so the patterns are anchored with \Z and
used with match().  2.7's int() never accepted underscores, but the
grammar check covers them anyway.  Tests use plain loops instead of
subTest and also cover "0x5" as a Content-Length.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Backport of python/cpython bpo-44022 (pythonGH-25933, 078b146) to 2.7.18.x.

HTTPResponse.begin() skips the header block of each 100 Continue
response.  That loop limited the length of each line (_MAXLINE) but not
the number of lines, so a malicious server could keep a client reading
header lines forever (an infinite loop / DoS).  The skipped lines are now
counted and HTTPException("got more than 100 headers") is raised past
_MAXHEADERS, the same limit already applied to a final response's headers.

2.7.18.x: upstream refactored the loop into _read_headers(); 2.7 parses
headers with HTTPMessage.readheaders(), so the counter is added to the
existing skip loop instead.  The upstream test would pass on unpatched
2.7 (EOF after the 'r' lines ends in BadStatusLine, itself an
HTTPException), so the 2.7 test appends a valid 200 response and checks
the message; a second test checks a normal 100 Continue is still skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CVE-2021-28861)

Backport of python/cpython pythongh-87389 (pythonGH-93879, 4dc2cae) to 2.7.18.x.

A request path starting with "//" that resolves to a directory made
SimpleHTTPRequestHandler.send_head() answer with a 301 whose Location
header also started with "//".  Browsers treat "//host/path" as a
scheme-relative URL, so a crafted link to the server redirected the user
to an attacker-chosen host.  BaseHTTPRequestHandler.parse_request() now
reduces a leading run of slashes in the path to a single "/".

Verified on 2.7.18.15: the "//python.org/..." request answers
"Location: //python.org/..."; with this change it is "/python.org/...".

2.7.18.x: the fix lands in BaseHTTPServer.parse_request(); the test uses
string concatenation and the literal 301 status instead of f-strings and
HTTPStatus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CVE-2024-3220)

Backport of python/cpython pythongh-159428 (pythonGH-159445; the 3.10.22.1 port is
f3c1882) to 2.7.18.x.

mimetypes.init() reads the registry on Windows and then every file in
knownfiles.  Those are Unix paths ("/etc/mime.types", ...), which on
Windows resolve to the root of the current drive (C:\etc\mime.types).
Unprivileged users can usually create such files, and so could
register arbitrary MIME types for every Python process on the machine,
e.g. to change what SimpleHTTPServer serves a file as.  knownfiles is now
empty on Windows; only the registry is used by default.  Callers can
still pass files to init() or add to knownfiles themselves.

2.7.18.x: test_mimetypes empties mimetypes.knownfiles at import for
isolation, so the test keeps a reference to the default list taken
before that line; the non-Windows check lives in a new
KnownfilesTestCase registered in test_main().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…model (CVE-2026-4224)

Backport of python/cpython pythongh-145986 (eb0e8be) to 2.7.18.x.

When a parser has an ElementDeclHandler, pyexpat converts each element
declaration's content model to nested tuples with the recursive
conv_content_model().  An inline DTD with a deeply nested content model
(e.g. 500000 levels of "(a, ") overflowed the C stack: on 2.7.18.15 the
reproducer segfaults.  conv_content_model() now goes through
Py_EnterRecursiveCall()/Py_LeaveRecursiveCall(), so deep nesting raises
RuntimeError("maximum recursion depth exceeded in conv_content_model")
instead.  As upstream, models nested deeper than the recursion limit
(1000 by default) now raise rather than being converted.

2.7.18.x: the public Py_EnterRecursiveCall() replaces the internal
_Py_EnterRecursiveCall(); C89 declarations; 2.7 raises RuntimeError, not
RecursionError.  The tests go into a new ElementDeclHandlerTest class
registered in test_main(), with a second test that a normal nested model
still converts.  The crash is in pyexpat itself; it reproduces and is
fixed with the bundled expat 2.6.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…6-9669)

Backport of python/cpython pythongh-150599 (3.14 157a5df) to 2.7.18.x.

After BZ2_bzDecompress() reported an error, BZ2Decompressor.decompress()
raised but left the object usable.  An application that caught the error
and called decompress() again resumed libbz2 from an invalid internal
state; with crafted input that can write out of bounds of a stack
buffer.  On 2.7.18.15 the upstream reproducer's second call goes back
into libbz2 and returns ''.  The decompressor now records the error and
raises ValueError("Decompressor is unusable after a previous error") on
any later call.

2.7.18.x: 2.7 has its own BZ2Decompressor implementation (no needs_input
or eof attributes), so the guard is a new bzerror field checked next to
the existing "end of stream was already found" check.  2.7 also has a
second path upstream's 3.x code doesn't: when growing the output buffer
fails, it calls BZ2_bzDecompressEnd() but left the object usable; that
path now marks the decompressor unusable too.  The test expects IOError
("invalid data stream"), as 2.7 raises, and uses binascii.unhexlify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ates (CVE-2026-6879)

Backport of python/cpython pythongh-152674 (2ffab08) to 2.7.18.x.

For an index predicate ([1], [last()], [last()-N]) ElementPath called
parent.findall(elem.tag) once per candidate element, so findall() and a
fully consumed iterfind() over n same-tag siblings did O(n^2) work.  A
crafted document plus a query like ".//item[last()]" could burn CPU.
select() now caches the selected sibling per (parent, tag) pair.

2.7.18.x: cElementTree's find()/findall() also go through this Python
ElementPath module, so the fix covers both implementations.  The test
counts findall() calls through an Element subclass; cElementTree's
Element is a factory function in 2.7 and cannot be subclassed, so the
test is skipped there.  super(Class, self) and %-formatting replace the
3.x spellings.  Without the fix, findall() calls grow as 1025/2049/4097
for 1024/2048/4096 siblings; with it they are constant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-2026-15806)

Backport of python/cpython pythongh-155694 (3.14 a0d023f) to 2.7.18.x.

HTTPPasswordMgr (and HTTPPasswordMgrWithDefaultRealm) reduced URIs to
(authority, path) and dropped the scheme, so credentials added for
https://example.com/ were also returned for http://example.com/.  An
attacker who can redirect or downgrade a client to plain HTTP could then
capture Basic/Digest credentials meant only for TLS.  Stored and
requested URIs now keep their scheme, and a stored URI with a scheme
only matches requests with the same scheme.  A stored bare authority
("example.com") or network-path reference ("//example.com/") still
matches any scheme, as before; reduce_uri() and is_suburi() keep their
public behavior.

2.7.18.x: 2.7 has no HTTPPasswordMgrWithPriorAuth, so only
add_password()/find_user_password() change.  (scheme,) + tuple replaces
the 3.x star-unpacking.  2.7's password-manager tests are doctests, so
the new tests go in a PasswordManagerSchemeTests class registered in
test_main(), with an extra HTTPPasswordMgrWithDefaultRealm case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eeding (CVE-2026-15308)

Backport of python/cpython pythongh-153030 (3.14 07efb08) to 2.7.18.x.

HTMLParser.feed() concatenated each chunk onto rawdata and rescanned the
whole unparsed buffer.  While a construct stays unterminated (a comment,
processing instruction, doctype, CDATA section, start tag or RAWTEXT
element such as <script>), every feed() rescans everything buffered so
far, so feeding attacker-controlled data in small chunks takes quadratic
time.  On 2.7.18.15, 10000 64-byte chunks inside <script> take 22s
(5.6s for 5000).  feed() now collects chunks in a list and only joins
and parses once the pending data reaches a threshold: 1 byte after a
call that made progress, otherwise the current buffer size, so the
buffer must double before it is rescanned.  close() flushes pending data.
200000 chunks now take 0.1-0.3s per construct.

As upstream: when a feed() makes no progress, the events for data in
the next small feeds may be reported later (once the buffer doubles, or
at close()); no data is lost.

2.7.18.x: Lib/HTMLParser.py; del list[:] instead of list.clear().  The
cpu-gated upstream test goes in a new IncrementalFeedTestCase, plus a
test that char-by-char feeding still yields the same events and that
close() delivers held-back data.  sgmllib/htmllib keep their own feed()
and are not changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Update the vendored Expat in Modules/expat from 2.6.4 to 2.9.0.  The
files are taken from 3.10.22.x (95dadd7, the 3.10 backport of
pythongh-158876), which fixes these Expat CVEs:

CVE-2024-8176 CVE-2025-59375 CVE-2026-24515 CVE-2026-25210
CVE-2026-32776 CVE-2026-32777 CVE-2026-32778 CVE-2026-41080
CVE-2026-45186 CVE-2026-50219 CVE-2026-56131 CVE-2026-56132
CVE-2026-56403 CVE-2026-56404 CVE-2026-56405 CVE-2026-56406
CVE-2026-56407 CVE-2026-56408 CVE-2026-56409 CVE-2026-56410
CVE-2026-56411 CVE-2026-56412 CVE-2026-66046 CVE-2026-72522
CVE-2026-76641 CVE-2026-76956 CVE-2026-76957 CVE-2026-77214
CVE-2026-93990 CVE-2026-102633

Expat 2.9.0 adds xcs.c (and the new headers fallthrough.h, hash_table.h,
memory_sanitizer.h and xcs.h).  xcs.c is added to setup.py's
expat_sources, to Modules/Setup.dist and to PCbuild/_elementtree.vcxproj,
which on Windows compiles the vendored sources.  The stale macconfig.h
entry in that project is replaced, as upstream did.

2.7.18.x keeps the two local changes it already had on top of 2.6.4:
- expat_external.h does not include pyexpatns.h.  On Windows,
  pyexpat.vcxproj compiles pyexpat.c against these headers but links the
  ActiveState libexpat.lib, so the symbols must keep their XML_* names.
- xmlparse.c's PREFIX type is renamed to PPREFIX, because PC/pyconfig.h
  defines a PREFIX macro.

The Windows pyexpat therefore builds against 2.9.0 headers.  The external
expat it links must be 2.9.0 too.

pyexpat.c is unchanged.  Expat 2.9.0 deprecates the XML_Size position
getters, XML_GetInputContext and XML_SetHashSalt, so gcc now warns about
them.  They still work, and switching pyexpat.c to the *64 replacements
would stop it from linking against an Expat older than 2.9.0.

test_pyexpat gains test_deeply_nested_entities, which uses a chain of
200000 nested general entities (CVE-2024-8176).  It segfaults with the
bundled 2.6.4 and passes with 2.9.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontext (CVE-2026-19445)

Backport of python/cpython pythongh-156293 (87665c9; 3.10: ec44b5a) to
2.7.18.x.

set_servername_callback() registered the SSLContext as a borrowed
pointer in the SSL_CTX's servername arg, and _servername_callback() used
that pointer to find the Python callback.  When the callback switches the
connection to another context (sslsock.context = other), the original
SSLContext can be deallocated while its SSL_CTX lives on as the
connection's session_ctx.  The callback then reads freed memory: right
after the Python callback returns (if it dropped the last reference and
raised), or on a later ClientHello such as one after a HelloRetryRequest.
On 2.7.18.15, a callback that switches context, drops it and raises
segfaults the server.  Under captured stderr it reports
"in <NULL> ignored", read from the freed object.

_servername_callback() now finds the context through the SSL object's
app data (the PySSLSocket and its current, live context), and holds
strong references to that context and its callback while they are used.
set_servername_callback() no longer registers the servername arg, and
context_dealloc() unregisters the servername callback before freeing the
SSL_CTX.  As upstream, after the callback switches contexts, any further
ClientHello on that connection goes to the new context's callback.

2.7.18.x: the legacy set_servername_callback()/set_hostname is the only
API (there is no sni_callback property); no Py_NewRef; context_dealloc's
unregister is guarded by HAVE_SNI like the rest of the SNI code.  2.7 has
no MemoryBIO, so the tests run the handshake over a socketpair with the
client in a thread.  The "callback raises" test also checks that the
unraisable error names the callback, which is what fails on 2.7.18.15
("<NULL>").  The HelloRetryRequest test passes on 2.7.18.15 in this
release build; it is kept, as upstream, as a regression guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ze() (CVE-2026-3276)

Backport of python/cpython pythongh-149079 (991224b; 3.10: db744c0) to
2.7.18.x.

nfd_nfkd(), which every normalization form goes through, put combining
characters into canonical order with an in-place insertion sort.  A long
run of combining marks in alternating canonical combining class order
made it quadratic.  On 2.7.18.15, NFC of u"a" + u"\u0300\u0327" * n
takes 0.22 s for n=20000, 1.6 s for n=50000 and 19.9 s for n=200000.

Each run of non-starters is now sorted on its own: insertion sort for
runs shorter than 20, otherwise a stable counting sort on the combining
class through a reusable buffer, and runs already in order are skipped.
The same input now takes 0.001 s, 0.004 s and 0.015 s.

2.7.18.x: the sort works on the Py_UNICODE result buffer instead of
PyUnicode_READ/WRITE.  On narrow (UCS-2) builds it therefore sees UTF-16
code units, as the old sort did; surrogates have combining class 0, so
they end runs as before.  C89 declarations.  The buffer grows through
PyMem_Realloc rather than upstream's PyMem_Resize, which in 2.7 overwrites
its pointer argument and would leak the old buffer on failure.

Tests: upstream's test_long_combining_mark_run and
test_combining_mark_run_fast_paths, plus test_combining_mark_run_is_linear
(a 100000-mark run through the counting sort).  On 20000 random strings
for each of three seeds, mostly combining marks with runs of 1-200,
NFC/NFD/NFKC/NFKD from both unicodedata and ucd_3_2_0 hash identically
to 2.7.18.15.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backport of python/cpython pythongh-118486 (eb29e2f; 3.10: c8f868d) to
2.7.18.x.

On Windows os.mkdir() ignored its mode, so tempfile.mkdtemp(), which
calls os.mkdir(name, 0700), created directories that inherit the
parent's ACL.  Under a shared parent other local users can read the
"private" temporary directory.

posix_mkdir() now handles mode 0700 on Windows by creating the directory
with a security descriptor whose protected discretionary ACL only grants
full control (inherited by files and subdirectories) to SYSTEM,
Administrators and the owner: "D:P(A;OICI;FA;;;SY)(A;OICI;FA;;;BA)
(A;OICI;FA;;;OW)".  Other modes are ignored, as before.  The same mode
check on POSIX is unchanged.

2.7.18.x: 2.7's posix_mkdir() has a wide (CreateDirectoryW) and a narrow
(CreateDirectoryA) path, and mkdtemp() normally takes the narrow one for
str names, so the descriptor is built by a shared helper and applied on
both.  C89 declarations.  ConvertStringSecurityDescriptorToSecurityDescriptorW
is in advapi32, which pythoncore already links (_winreg).  The docstring
and Doc/library/os.rst describe the 0700 handling as upstream does.

Tests (Windows only, ported from upstream): test_os.MakedirTests.
test_win32_mkdir_700 checks the SDDL with cacls.exe, and
test_tempfile.test_mkdtemp.test_mode_win32 checks with icacls.exe that
BUILTIN\Users has no access.  Not built or run on Windows yet: the
Windows code was syntax-checked with gcc -std=c89 -pedantic against
stub declarations, and the Linux build and test_os/test_tempfile pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Expat 2.9.0 moved the rand_s() entropy code out of xmlparse.c into
random_rand_s.c/.h, which xmlparse.c includes under
"_WIN32 && ! XML_POOR_ENTROPY".  CPython does not vendor that file;
it builds Expat with XML_POOR_ENTROPY instead, as setup.py already does
on POSIX and as 3.10.22.x's PCbuild/_elementtree.vcxproj does.
_elementtree.vcxproj compiles the vendored xmlparse.c, so without the
define the Windows build fails with "random_rand_s.h: No such file or
directory".  Python sets the hash salt itself, so Expat's own entropy
source is not used.

pyexpat.vcxproj does not compile the vendored sources (it links the AS
libexpat), so it needs no change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…026-7210)

Backport of python/cpython pythongh-149018 (24b8f12; 3.10: ea70712) to
2.7.18.x.

pyexpat and cElementTree seed every Expat parser's hash-flooding
protection with XML_SetHashSalt(parser, _Py_HashSecret.prefix).  On 2.7
_Py_HashSecret is all zeros unless hash randomization (-R) is on.  A salt
of 0 makes Expat generate its own, and our build defines
XML_POOR_ENTROPY, so it comes from the time and pid.
EXPAT_ENTROPY_DEBUG=1 shows "fallback(8)" salts whose upper bits are the
same on every run.  Even with -R the salt is at most one long.

With Expat >= 2.8.0, pyexpat and _elementtree now read 16 bytes from
_PyOS_URandom() once at import, and pass them to every parser through
XML_SetHashSalt16Bytes().  As upstream, the function is added to the end
of the PyExpat_CAPI struct, which is NULL for Expat < 2.8.0.  If the OS
random source fails, or with an older Expat, the old XML_SetHashSalt()
path is used.

2.7.18.x: upstream reads the salt from a new 16-byte expat member of
_Py_HashSecret.  2.7's _Py_HashSecret_t is two longs, and changing it
would alter that exported struct, so each module keeps its own random
salt instead.  _elementtree uses the 16-byte path only through the
pyexpat CAPI (USE_PYEXPAT_CAPI, which setup.py and the Windows project
both define).

Needs Expat >= 2.8.0 (the bundled copy is 2.9.0).  On Windows, pyexpat
links the external AS libexpat, which must then also be >= 2.8.0 (the
current platform build uses 2.8.4).

The new test_pyexpat.HashSaltTest runs pyexpat and cElementTree under
EXPAT_ENTROPY_DEBUG=1 and requires every parser's salt to be
"explicit(16)".  With bundled Expat 2.9.0 and no fix, both tests fail
with fallback(8).  With the fix, both pass.  Both are skipped for
Expat < 2.8.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Review follow-up.  In 2.7, BZ2File is C code over BZ2_bzRead(), which
calls BZ2_bzDecompress() on the same stream on every call.  After a read
fails, the file stayed in MODE_READ, so the next read, readline,
readlines, iteration or forward seek re-entered the errored stream.
That is the out-of-bounds write this change closes for BZ2Decompressor.
In 3.x, BZ2File is built on BZ2Decompressor, so upstream's fix already
covers it.

BZ2FileObject now records the libbz2 error wherever a read path gets one
(Util_CatchBZ2ReadError).  read(), readline(), readlines() and iteration
then raise ValueError("BZ2File is unusable after a previous error").
seek() reopens the stream first when an error was recorded, and every
backwards seek reopens it anyway, so seeking reads from a fresh stream
and clears the error.  The rewind code moves into Util_BZ2FileRewind(),
shared by both cases.

test_bz2.BZ2FileTest.test_read_after_data_error writes the crafted
stream plus 4000 trailing bytes.  It checks that every read method
raises ValueError after the first IOError, and that seek(0), seek(10)
and seek(0, 2) start over and hit the same IOError instead of reusing
the stream.  Without this change the second read re-enters libbz2 and
fails with IOError again.  The crafted stream is now a module constant
shared with the decompressor test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tplib

Backport of python/cpython pythongh-150743 (41cc78a; 3.10: 1790e58),
GHSA-w4q2-g22w-6fr4, to 2.7.18.x.  Review follow-up to the CVE-2021-3737
fix in this PR, which bounded the header lines inside one interim
response but not these two sibling loops:

- HTTPResponse.begin() skipped any number of "100 Continue" responses.
  It now stops after _MAXINTERIMRESPONSES (100) and raises
  HTTPException("got more than 100 interim responses").
- _read_chunked() discarded any number of trailer lines after the last
  chunk.  It now raises HTTPException("got more than 100 trailers") past
  _MAXHEADERS.

A server streaming either one forever kept the client reading
indefinitely, and a socket timeout cannot catch that because data keeps
arriving.

2.7.18.x: % formatting instead of f-strings, and assertRaisesRegexp.  2.7
has no chunked fixtures in test_httplib, so the tests build their own
bodies.  test_chunked_trailers is added to show that a few trailers are
still accepted.  test_too_many_interim_responses and
test_chunked_too_many_trailers fail on the previous head of this branch
and pass here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[2.7.18.16] Update bundled Expat to 2.9.0 (30 expat CVEs)
[2.7.18.16] Give Expat parsers a 16-byte random hash salt (CVE-2026-7210)
2.7.18.16: tarfile streaming-mode seek loops at EOF (CVE-2026-11972)
2.7.18.16: csv.Sniffer quadratic regex on quoted fields (CVE-2026-18503)
2.7.18.16: ftplib.ftpcp() trusts the PASV host (CVE-2026-8328)
2.7.18.16: httplib Content-Length / chunk-size validation (CVE-2026-103995)
2.7.18.16: httplib unbounded headers after 100 Continue (CVE-2021-3737)
…irect

2.7.18.16: SimpleHTTPServer // open redirect (CVE-2021-28861)
2.7.18.16: mimetypes reads C:\etc\mime.types etc. on Windows (CVE-2024-3220)
Resolve the test_pyexpat.py conflict with #105: both add a test class
after ForeignDTDTests and register it in test_main(); keep both
ElementDeclHandlerTest and HashSaltTest.  Modules/pyexpat.c merged
cleanly.
icanhasmath and others added 13 commits October 28, 2026 12:07
2.7.18.16: pyexpat stack overflow on deeply nested content model (CVE-2026-4224)
2.7.18.16: bz2 decompressor reused after an error (CVE-2026-9669)
2.7.18.16: ElementPath index predicates are O(n^2) (CVE-2026-6879)
…r-scheme

2.7.18.16: urllib2 HTTPPasswordMgr sends https credentials over http (CVE-2026-15806)
…ntal

2.7.18.16: HTMLParser incremental feed is quadratic (CVE-2026-15308)
[2.7.18.16] ssl: do not call the SNI callback through a released SSLContext (CVE-2026-19445)
[2.7.18.16] Fix O(n^2) canonical ordering in unicodedata.normalize() (CVE-2026-3276)
[2.7.18.16] Support mkdir(mode=0700) on Windows (CVE-2024-4030)
Brings in the last two #91 commits: the corrected certifi note in the
2.7.18.15 release notes (28706dd) and the bundled pip 20.3.4.post4
(ff19c1d).
…dl.h

Three small fixes raised in the reviews of #106 and #108:

- socketmodule.c: remove socket_parse_timeout().  It came in with the
  CVE-2023-40217 getblocking() backport (4737406, first released in
  2.7.18.10), has no callers, and calls _PyTime_FromSecondsObject(),
  which 2.7 never builds (Python/pytime.c is not compiled).  Optimized
  builds drop the unused static function, but an -O0 or --with-pydebug
  build leaves an undefined symbol in _socket, so _socket (and ssl)
  fail to import.

- _ssl.c: _servername_callback() set ssl_socket to Py_None without a
  reference when the SSL object has no Python socket, and the error path
  then did Py_DECREF(ssl_socket).  Take the reference.  This predates
  #106; it is not reachable through ssl.SSLSocket, which always sets
  ssl_sock.

- posixmodule.c: include <sddl.h> under MS_WINDOWS, as upstream does,
  instead of inside the _MSC_VER block, so it matches the MS_WINDOWS
  helpers that call ConvertStringSecurityDescriptorToSecurityDescriptorW
  (#108).

Correction to the #108 commit message (386508c): it cites upstream's
pythongh-118486 commit as eb29e2f.  The original main commit is
81939da (pythonGH-118488, 2024-05-02), followed by 8ed5466 (OWNER
RIGHTS) and 8af84b5 (pythongh-118773, the SDDL string the port uses).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-up (#111).  test_ssl.py imported "from mock import Mock"
(added with an earlier ssl backport, 37df9f7, and in 2.7.18.14), but
mock is not part of the 2.7 standard library, so on a plain build the
whole module failed with "ImportError: No module named mock" and none
of test_ssl ran.  Mock itself was unused; the only mock use,
mock.patch.dict(os.environ) in test_keylog_env, referred to a name the
file never imported.  Use support.EnvironmentVarGuard there instead and
remove the import.  (test_keylog_env is skipped on 2.7, whose _ssl has
no keylog callback.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[2.7.18.16] Review follow-ups: _socket debug link, SNI None ref, sddl.h
Bump PY_VERSION to 2.7.18.16 and add release notes for the security
fixes merged into 2.7.18.16 (#92-#108, #111): tarfile (CVE-2026-11972),
csv.Sniffer (CVE-2026-18503), ftplib.ftpcp (CVE-2026-8328), httplib
(CVE-2026-103995, CVE-2021-3737 and GHSA-w4q2-g22w-6fr4),
SimpleHTTPServer (CVE-2021-28861), mimetypes on Windows (CVE-2024-3220),
pyexpat (CVE-2026-4224), bz2 (CVE-2026-9669), ElementPath
(CVE-2026-6879), urllib2 (CVE-2026-15806), HTMLParser (CVE-2026-15308),
bundled Expat 2.9.0 (30 CVEs), the Expat hash salt (CVE-2026-7210), the
ssl SNI callback (CVE-2026-19445), unicodedata (CVE-2026-3276) and
os.mkdir on Windows (CVE-2024-4030), plus the review follow-up build
fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@ezequielp-activestate ezequielp-activestate left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blockers. I checked that this release contains exactly what was reviewed:

  • Commits: every non-merge commit in 2.7..2.7.18.16 is part of an approved head of #91 or #92–#108, #111, apart from the release commit 60c3e21.
  • Merges: I re-ran each of the 21 merge commits with git merge-tree and compared the result with the committed tree. 20 are identical to the automatic merge. The one exception is 7d13ea2c1b (2.7.18.16 merged into #99's branch after it was approved). It resolved a test-only conflict in Lib/test/test_pyexpat.py, where #99's ElementDeclHandlerTest and #105's HashSaltTest were added in the same place. The resolution keeps both classes and lists both in test_main, which is correct.
  • Release commit: it changes only PY_VERSION (2.7.18.15 → 2.7.18.16) and adds Misc/NEWS.d/2.7.18.16.rst. The notes name every CVE in the table above. Their Expat section lists 22 IDs plus the range CVE-2026-56403 to 56412, which matches the 30 in #104.
  • Head: this PR's tree is the current 2.7.18.16 branch.

As the description says, the Windows-only changes (#98, #104's _elementtree project define, #108) haven't been built yet, so a Windows build should come before tagging.

@icanhasmath
icanhasmath merged commit 194c0c3 into 2.7 Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants