Skip to content

Import circuits, experimental, and two scipy submodules only when first used - #907

Merged
perlinm merged 6 commits into
mainfrom
perf/defer-unused-imports
Nov 16, 2026
Merged

perlinm merged 6 commits into
mainfrom
perf/defer-unused-imports

Conversation

@perlinm

@perlinm perlinm commented Nov 16, 2026

Copy link
Copy Markdown
Collaborator

Motivation

import qldpc takes about a second, and much of qLDPC's use is interactive: notebooks, the REPL, and one-off scripts. Part of that time loads modules that many sessions never use. The subpackages circuits and experimental hold 45 of qLDPC's 149 modules, and none of the modules that import qldpc loads imports them at import time. scipy.special and scipy.sparse.csgraph are used by only a few functions.

Changes

  • qldpc/__init__.py imports circuits and experimental on first access, through a module-level __getattr__ with a matching __dir__. Like the other package stubs, __getattr__ falls back to _deprecations.get_deprecated_attribute. Both names stay in __all__, so qldpc.circuits, import qldpc.circuits, from qldpc import circuits, star imports, and deprecated names served by these subpackages work as before. Type checkers see them through an import under TYPE_CHECKING, so they still reject unknown names.
  • The functions that use scipy.special (math.log_choose, ErrorRateFunc._get_tail_probability) and scipy.sparse.csgraph (in abstract.rings and objects.complexes) import them at the top of their bodies.
  • The fresh-import test in import_test.py checks, in a fresh interpreter, that import qldpc loads none of these four modules, that dir(qldpc) lists the deferred subpackages, that attribute access imports them and unknown names raise AttributeError, and that a star import serves both subpackages without loading any optional decoder backend.
  • run_python in conftest.py no longer passes PYTHONOPTIMIZE to the child, which stripped the assertions that subprocess tests rely on. A new test checks this.

Validation

import qldpc timed in fresh processes, 20 interleaved rounds against main with a shared warm numba cache: median 1.104 s on main and 1.015 s on this branch, a paired saving of 93 ms (quartiles 61 and 141 ms; faster in 18 of 20 rounds). The Sphinx docs build of main and of this branch produce identical API pages.

Notes

The first access to qldpc.circuits or qldpc.experimental in a session now pays their import cost, instead of import qldpc paying it.

perlinm and others added 6 commits November 16, 2026 13:59
…st used

import qldpc no longer imports the subpackages circuits and experimental, which no other subpackage
imports at import time.  A module-level __getattr__ in qldpc/__init__.py imports them on first
access, __dir__ lists them, and __all__ still names them, so that qldpc.circuits, from qldpc import
circuits, and star imports work as before; type checkers see them through a TYPE_CHECKING import.
The functions that use scipy.special and scipy.sparse.csgraph in math, abstract.rings,
objects.complexes, and codes.monte_carlo import them in their bodies.  The fresh-import test now
checks that none of these modules is loaded by import qldpc, and that star imports serve the
deferred subpackages.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ONOPTIMIZE from the child

The fresh-import test now checks dir(qldpc) and unknown names before anything loads the deferred
subpackages, and checks again for optional backends after the star import loads them, so that
other test modules in the same process cannot satisfy it.  The in-process test is gone.  The lazy
__getattr__ falls back to _deprecations.get_deprecated_attribute, as the other package stubs do, and
its comments state the rule that the eagerly imported subpackages keep and why it sits under
TYPE_CHECKING's else branch.  run_python no longer passes PYTHONOPTIMIZE to the child, which
stripped the assertions in the child's code.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…hon runs assertions

The fresh-import test now reaches circuits and experimental by attribute access before the star
import, which would succeed even without the lazy __getattr__.  A new test checks that run_python's
child runs assertions while PYTHONOPTIMIZE is set.  Comments and docstrings name the list of
deferred modules and state the import rule for every module that import qldpc loads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rtion error

Final-round fixes from the review: the docstrings of import_test.py and run_python say what the
tests check, the fresh-import program reaches experimental by plain attribute access, and the
PYTHONOPTIMIZE test requires the child's traceback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@perlinm
perlinm merged commit dc7df2b into main Nov 16, 2026
9 checks passed
@perlinm
perlinm deleted the perf/defer-unused-imports branch November 16, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant