gh-160185: Fix CJK StreamWriter.reset() not returning to ASCII by gpshead · Pull Request #160186 · python/cpython

gpshead · 2026-10-22T04:30:16Z

MultibyteStreamWriter.reset() now passes an empty string to the encoder when nothing is pending instead of returning early, so the codec's reset hook writes the shift-back sequence again, as it did before gh-67436. It also drops the leaked stream.write() result.

Since reset() can now write bytes, StreamWriter.seek(), StreamReaderWriter.seek() and StreamRecoder.seek() reset the writer before moving the stream; otherwise seek(0) would write ESC ( B over offset 0. Other codecs' StreamWriter.reset() writes nothing, so the reorder doesn't change them.

Fixes #160185

ANT-2026-4AU4SRS4

Issue: CJK StreamWriter.reset() does not write the shift-back sequence #160185

reset() returned early when nothing was pending, so the shift-back sequence of hz and iso2022_* was never written (regression from pythongh-67436). Also fix a reference leak of the stream.write() result.

corona10

lgtm!

# Conflicts: # Lib/test/test_multibytecodec.py

miss-islington-app · 2026-11-01T06:44:07Z

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

bedevere-app · 2026-11-01T06:44:21Z

GH-161067 is a backport of this pull request to the 3.15 branch.

bedevere-app · 2026-11-01T06:44:28Z

GH-161068 is a backport of this pull request to the 3.14 branch.

bedevere-bot · 2026-11-01T07:02:33Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL9 3.x (tier-3) has failed when building commit aa4f541.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1565/builds/5992) and take a look at the build logs.
Check if the failure is related to this commit (aa4f541) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1565/builds/5992

Failed tests:

test_zipimport

Failed subtests:

testGetCompiledSource - test.test_zipimport.DeflateCompressedZipImportTestCase.testGetCompiledSource
testBoth - test.test_zipimport.UncompressedZipImportTestCase.testBoth
testGetCompiledSource - test.test_zipimport.UncompressedZipImportTestCase.testGetCompiledSource
testGetCompiledSource - test.test_zipimport.ZStdCompressedZipImportTestCase.testGetCompiledSource
test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.UncompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc
test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.ZStdCompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc
testBoth - test.test_zipimport.DeflateCompressedZipImportTestCase.testBoth
testBoth - test.test_zipimport.ZStdCompressedZipImportTestCase.testBoth
test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.DeflateCompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (test.test_zipimport.UncompressedZipImportTestCase.testTraceback) ... ok


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 877, in testGetCompiledSource
    self.doTest(pyc_ext, files, TESTMOD, call=self.assertModuleSource)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[77 chars]d/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[77 chars]d/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                               +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 315, in testBoth
    self.doTest(pyc_ext, files, TESTMOD)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[77 chars]d/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[77 chars]d/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                               +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 877, in testGetCompiledSource
    self.doTest(pyc_ext, files, TESTMOD, call=self.assertModuleSource)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[77 chars]d/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[77 chars]d/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                               +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 315, in testBoth
    self.doTest(pyc_ext, files, TESTMOD)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[77 chars]d/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[77 chars]d/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                               +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 286, in test_syntax_error_in_source_with_valid_pyc
    self.assertEqual(spec.origin, expected_path)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[69 chars]ild/build/test_python_2697239æ/junk95142.zip/ziptestmodule.py' != '/var[69 chars]ild/build/test_python_2697239æ/junk95142.zip/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2697239æ/junk95142.zip/ziptestmodule.pyc
?                                                                                                                                       +


Traceback (test.test_zipimport.ZStdCompressedZipImportTestCase.testTraceback) ... ok


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/Lib/test/test_zipimport.py", line 286, in test_syntax_error_in_source_with_valid_pyc
    self.assertEqual(spec.origin, expected_path)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[69 chars]ild/build/test_python_2691828æ/junk95142.zip/ziptestmodule.py' != '/var[69 chars]ild/build/test_python_2691828æ/junk95142.zip/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-rhel9-s390x/3.x.cstratak-rhel9-s390x/build/build/test_python_2691828æ/junk95142.zip/ziptestmodule.pyc
?                                                                                                                                       +


Traceback (test.test_zipimport.DeflateCompressedZipImportTestCase.testTraceback) ... ok

…GH-160186) (#161067) gh-160185: Fix CJK StreamWriter.reset() not returning to ASCII (GH-160186) reset() returned early when nothing was pending, so the shift-back sequence of hz and iso2022_* was never written (regression from gh-67436). Also fix a reference leak of the stream.write() result. (cherry picked from commit aa4f541) Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>

…GH-160186) (#161068) gh-160185: Fix CJK StreamWriter.reset() not returning to ASCII (GH-160186) reset() returned early when nothing was pending, so the shift-back sequence of hz and iso2022_* was never written (regression from gh-67436). Also fix a reference leak of the stream.write() result. (cherry picked from commit aa4f541) Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>

bedevere-bot · 2026-11-01T19:47:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Refleaks 3.15 (tier-3) has failed when building commit 6d18fc9.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/2056/builds/471) and take a look at the build logs.
Check if the failure is related to this commit (6d18fc9) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/2056/builds/471

Failed tests:

test_imaplib

Failed subtests:

test_imaplib_timeout_functionality_test - test.test_imaplib.NewIMAPTests.test_imaplib_timeout_functionality_test
test_timerfd_ns_TFD_TIMER_ABSTIME - test.test_os.test_os.TimerfdTests.test_timerfd_ns_TFD_TIMER_ABSTIME

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/test/test_os/test_os.py", line 4437, in test_timerfd_ns_TFD_TIMER_ABSTIME
    self.assertLess(abs(next_expiration_ns - offset_ns),  limit_error)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 1137723 not less than 1000000


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/test/test_imaplib.py", line 745, in test_imaplib_timeout_functionality_test
    client = self.imap_class("localhost", addr, timeout=0.001)
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/imaplib.py", line 217, in __init__
    self.open(host, port, timeout)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/imaplib.py", line 331, in open
    self.sock = self._create_socket(timeout)
                ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/imaplib.py", line 320, in _create_socket
    return socket.create_connection(address, timeout)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/socket.py", line 887, in create_connection
    raise exceptions[0]
  File "/home/buildbot/buildarea/3.15.opsec-fbsd14.refleak/build/Lib/socket.py", line 872, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 61] Connection refused

gpshead added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 22, 2026

gpshead self-assigned this Oct 22, 2026

bedevere-app Bot mentioned this pull request Oct 22, 2026

CJK StreamWriter.reset() does not write the shift-back sequence #160185

Closed

pythongh-160185: Fix CJK StreamWriter.reset() not returning to ASCII

d86d637

reset() returned early when nothing was pending, so the shift-back sequence of hz and iso2022_* was never written (regression from pythongh-67436). Also fix a reference leak of the stream.write() result.

gpshead force-pushed the cjk-streamwriter-reset branch from 345ec10 to d86d637 Compare October 22, 2026 04:44

gpshead marked this pull request as ready for review October 22, 2026 05:30

gpshead requested a review from corona10 as a code owner October 22, 2026 05:30

bedevere-app Bot added the awaiting core review label Oct 22, 2026

gpshead added the sprint label Oct 22, 2026

github-project-automation Bot added this to Sprint Oct 22, 2026

github-project-automation Bot moved this to Todo in Sprint Oct 22, 2026

corona10 approved these changes Oct 22, 2026

View reviewed changes

bedevere-app Bot removed the awaiting core review label Oct 22, 2026

github-project-automation Bot moved this from Todo to In Progress in Sprint Oct 22, 2026

bedevere-app Bot added the awaiting merge label Oct 22, 2026

gpshead enabled auto-merge (squash) November 1, 2026 05:56

gpshead disabled auto-merge November 1, 2026 05:56

Merge remote-tracking branch 'origin/main' into cjk-streamwriter-reset

dc3c5b6

# Conflicts: # Lib/test/test_multibytecodec.py

gpshead enabled auto-merge (squash) November 1, 2026 06:08

gpshead merged commit aa4f541 into python:main Nov 1, 2026
57 checks passed

github-project-automation Bot moved this from In Progress to Done in Sprint Nov 1, 2026

bedevere-app Bot removed the awaiting merge label Nov 1, 2026

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Nov 1, 2026

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 1, 2026

Navigation Menu

Repository navigation

gh-160185: Fix CJK StreamWriter.reset() not returning to ASCII - #160186

gh-160185: Fix CJK StreamWriter.reset() not returning to ASCII#160186
gpshead merged 2 commits into
python:mainfrom
gpshead:cjk-streamwriter-reset

gpshead commented Oct 22, 2026 •

edited by bedevere-app Bot

Loading

Uh oh!

corona10 left a comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Nov 1, 2026

Uh oh!

bedevere-app Bot commented Nov 1, 2026

Uh oh!

bedevere-app Bot commented Nov 1, 2026

Uh oh!

bedevere-bot commented Nov 1, 2026

Uh oh!

bedevere-bot commented Nov 1, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

gpshead commented Oct 22, 2026 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Nov 1, 2026

Uh oh!

bedevere-app Bot commented Nov 1, 2026

Uh oh!

bedevere-app Bot commented Nov 1, 2026

Uh oh!

bedevere-bot commented Nov 1, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Nov 1, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gpshead commented Oct 22, 2026 •

edited by bedevere-app Bot

Loading