Skip to content

Cache failed signature lookups in curry - #651

Open
ice-pring wants to merge 1 commit into
pytoolz:masterfrom
ice-pring:perf/cache-curry-signature-failures
Open

ice-pring wants to merge 1 commit into
pytoolz:masterfrom
ice-pring:perf/cache-curry-signature-failures

Conversation

@ice-pring

Copy link
Copy Markdown

Repeated partial applications of builtins such as curry(map)(func) and curry(filter)(func) are slow on Python versions where inspect.signature cannot inspect those builtins. curry stores the failed lookup as None, which also means "not checked yet". The argument-check helpers then try introspection again before falling back to the builtin-signature registry.

Cache a traceback-free ValueError or TypeError instance instead. The existing argument-check helper already understands those results, so subsequent calls use the same fallback without repeating signature inspection. Keep the distinction between unavailable signatures (ValueError) and invalid signatures (TypeError), and avoid retaining exception traceback frames.

Addresses #583. Two new regression cases exercise repeated calls through objects with unavailable or invalid signatures. Both fail before the fix (15 signature lookups rather than 1), then pass while preserving partial application, successful full calls, and TypeError propagation.

Measured on Windows / CPython 3.12.14, reusing one curried wrapper and evaluating list(wrapped(identity)([1, 2, 3])); each timing is the median of five runs of 10,000 operations:

Builtin Before After Speedup inspect.signature calls per 100 warmed operations
map 0.2205 s 0.0564 s 3.9x 300 to 0
filter 0.2768 s 0.0652 s 4.2x 400 to 0

CPython 3.14 already exposes signatures for these builtins; its timings stay approximately unchanged (map 0.058/0.057 s, filter 0.075/0.075 s).

Validation:

  • CPython 3.12.14: coverage run -m pytest --doctest-modules toolz/: 270 passed, 2 expected skips.
  • CPython 3.14.5: same command: 271 passed, 1 expected doctest skip.
  • pytest bench/: 19 passed on each interpreter, using the original Gutenberg Shakespeare fixture. Used UTF-8 mode for this text fixture on Windows.
  • Repository source pycodestyle --ignore="E731,W503,W504,E402" check and git diff --check passed.

Other operating systems and the remaining interpreter matrix were not run locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant