Repository navigation
Import circuits, experimental, and two scipy submodules only when first used - #907
Merged
Merged
Conversation
…st used import qldpc no longer imports the subpackages circuits and experimental, which no other subpackage imports at import time. A module-level __getattr__ in qldpc/__init__.py imports them on first access, __dir__ lists them, and __all__ still names them, so that qldpc.circuits, from qldpc import circuits, and star imports work as before; type checkers see them through a TYPE_CHECKING import. The functions that use scipy.special and scipy.sparse.csgraph in math, abstract.rings, objects.complexes, and codes.monte_carlo import them in their bodies. The fresh-import test now checks that none of these modules is loaded by import qldpc, and that star imports serve the deferred subpackages. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ONOPTIMIZE from the child The fresh-import test now checks dir(qldpc) and unknown names before anything loads the deferred subpackages, and checks again for optional backends after the star import loads them, so that other test modules in the same process cannot satisfy it. The in-process test is gone. The lazy __getattr__ falls back to _deprecations.get_deprecated_attribute, as the other package stubs do, and its comments state the rule that the eagerly imported subpackages keep and why it sits under TYPE_CHECKING's else branch. run_python no longer passes PYTHONOPTIMIZE to the child, which stripped the assertions in the child's code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…hon runs assertions The fresh-import test now reaches circuits and experimental by attribute access before the star import, which would succeed even without the lazy __getattr__. A new test checks that run_python's child runs assertions while PYTHONOPTIMIZE is set. Comments and docstrings name the list of deferred modules and state the import rule for every module that import qldpc loads. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rtion error Final-round fixes from the review: the docstrings of import_test.py and run_python say what the tests check, the fresh-import program reaches experimental by plain attribute access, and the PYTHONOPTIMIZE test requires the child's traceback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
import qldpctakes about a second, and much of qLDPC's use is interactive: notebooks, the REPL, and one-off scripts. Part of that time loads modules that many sessions never use. The subpackagescircuitsandexperimentalhold 45 of qLDPC's 149 modules, and none of the modules thatimport qldpcloads imports them at import time.scipy.specialandscipy.sparse.csgraphare used by only a few functions.Changes
qldpc/__init__.pyimportscircuitsandexperimentalon first access, through a module-level__getattr__with a matching__dir__. Like the other package stubs,__getattr__falls back to_deprecations.get_deprecated_attribute. Both names stay in__all__, soqldpc.circuits,import qldpc.circuits,from qldpc import circuits, star imports, and deprecated names served by these subpackages work as before. Type checkers see them through an import underTYPE_CHECKING, so they still reject unknown names.scipy.special(math.log_choose,ErrorRateFunc._get_tail_probability) andscipy.sparse.csgraph(inabstract.ringsandobjects.complexes) import them at the top of their bodies.import_test.pychecks, in a fresh interpreter, thatimport qldpcloads none of these four modules, thatdir(qldpc)lists the deferred subpackages, that attribute access imports them and unknown names raiseAttributeError, and that a star import serves both subpackages without loading any optional decoder backend.run_pythoninconftest.pyno longer passesPYTHONOPTIMIZEto the child, which stripped the assertions that subprocess tests rely on. A new test checks this.Validation
import qldpctimed in fresh processes, 20 interleaved rounds againstmainwith a shared warm numba cache: median 1.104 s onmainand 1.015 s on this branch, a paired saving of 93 ms (quartiles 61 and 141 ms; faster in 18 of 20 rounds). The Sphinx docs build ofmainand of this branch produce identical API pages.Notes
The first access to
qldpc.circuitsorqldpc.experimentalin a session now pays their import cost, instead ofimport qldpcpaying it.