Skip to content

Stable code identifier pickling - #40711

Draft
kristynsmith wants to merge 14 commits into
apache:masterfrom
kristynsmith:stable-code-identifier-pickling
Draft

kristynsmith wants to merge 14 commits into
apache:masterfrom
kristynsmith:stable-code-identifier-pickling

Conversation

@kristynsmith

Copy link
Copy Markdown
Contributor

Fixes several edge cases in code_object_pickler when generating and resolving stable code object identifiers:

  • Addressed the case where multiple nested def functions have the same name (e.g. defined across if/else branches) by applying the same signature and bytecode-hash disambiguation in _search_locals (formerly _search_lambda) to any duplicate code object name in co_consts, and updating _OBJECT_WITH_ARGS_PATTERN and _OBJECT_WITH_HASH_PATTERN to match named functions in addition to <lambda>.
  • Addressed the case where one of multiple lambdas/nested functions takes no arguments by updating _OBJECT_WITH_ARGS_PATTERN to match empty argument tuples ().
  • Addressed the case where two lambdas or nested functions have identical co_code bytecode instructions (e.g. referencing different closure variables or constants) by including co_consts, co_names, co_varnames, co_freevars, and co_cellvars in _create_bytecode_hash.
  • Addressed the case where a parent package re-exports a symbol with the same name as a submodule (e.g. from pkg.submod import submod in pkg/__init__.py) by resolving the longest matching module prefix in get_code_from_identifier before traversing attributes.
  • Addressed the case where an outer and inner function share the same decorator wrapper by returning None (falling back to default pickling) instead of raising ValueError('Qual name parts too long') when a branch does not match the full __qualname__.

Replace config assignment with dataclasses.replace for stability.
Refactor code object handling and improve normalization for hashing.
Added new test cases for lambda functions and decorators in code_object_pickler.
Refactor code object search logic to handle local code objects and update lambda handling. Replace lambda-specific functions with more general object handling functions.
Added tests for handling same-named nested functions with different and same arguments in code object pickler.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant