[cmaloney]

_Py_strerror decodes strerror_r with the locale codec on every call, a large part of the cost of raising OSError. Cache the result for codes 0-255 in a per-interpreter dict; 256 just to keep things from growing unbounded. Error strings are locale specific so _locale.setlocale clears it.

PGO+LTO, main vs this change:

Benchmark	main	strerror cache
os.strerror(2)	130 ns	21.9 ns: 5.93x faster
stat_missing()	935 ns	724 ns: 1.29x faster
open_missing()	1.35 us	1.15 us: 1.17x faster
raise_oserror()	105 ns	107 ns: 1.02x slower

Benchmark:

import os
import pyperf

def stat_missing():
    try:
        os.stat('/nonexistent')
    except OSError:
        pass

def open_missing():
    try:
        open('/nonexistent', 'rb')
    except OSError:
        pass

def raise_oserror():
    try:
        raise OSError(2, 'No such file or directory')
    except OSError:
        pass

runner = pyperf.Runner()
runner.bench_func("stat_missing()", stat_missing)
runner.bench_func("open_missing()", open_missing)
runner.bench_func("raise_oserror()", raise_oserror)
runner.bench_func("os.strerror(2)", os.strerror, 2)

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

• Issue: Improve performance of exceptions #161242

[vstinner]

Error strings are locale specific so _locale.setlocale clears it.

A C library can change the LC_CTYPE locale. Or when Python is embedded in an application, the application can change the LC_CTYPE locale (without telling Python).

Such optimization would (only?) be safe if it checks if the LC_CTYPE locale changed. Either check setlocale(LC_CTYPE, NULL) string or check nl_langinfo(CODESET) string. I suggest checking nl_langinfo(CODESET).

[cmaloney]

This is looking like it needs two cache keys (CODESET + LC_MESSAGES) if we need to invalidate when things outside of python change locale but don't tell Python about it directly... which gets to significant added complexity.

This is safe in terms of "python won't crash". It's not great in "locale may not match exactly". Open to suggestions that keep the complexity from significantly expanding (cutting 20% off each FileNotFoundError when we start python and it looks for .pyc would be really nice)

[vstinner]

I don't know if it's feasible, but I always wanted to defer when OSError message are formatted. In short, store components of the error message (like error number, optional path), and only render the message string if it's requested. In the majority of cases, the message is never requested.

One pain point is OSError.args which contains the message, so it should be a getter function. But BaseException constructor stores its positional arguments as the self->args attribute. I'm not sure how to deal with that.

If that would be possible, it should be faster than cached os.strerror().

[cmaloney]

I'll put this in draft and experiment a bit :). Planning to make the BaseException Argument Clinic Vectorcall shortly which moves performance relative to 3.15 a bit and changes the tuple args handling some.

As part of the posix -> _os I was looking at ntpath vs posixpath and other than some constants the key performance difference is ntpath uses Windows APIs to return "exists" and the like without requiring the exception while posixpath does not. A new primitive could potentially help there as well.... That said "exception on open" is the only safe/secure way to do a lot of things (check than open runs into timing attack issues)

[vstinner]

Another different approach: optimize PyUnicode_DecodeLocale() for UTF-8 (most common locale encoding): vstinner#21. My hack makes the function 3.2x faster and makes os.strerror() 2.0x faster.

[cmaloney]

Have a PR that helps with that #161147 :). Although mainly focused on "We load large mainly-ascii text files a lot". Definitely very aligned here, just a matter of finding what is going to be shippable simplifications. For me generally aiming for do less work -> it runs faster. Eliminate a whole stack of things most direct, optimize them longer path but also very doable.