diff --git a/README.md b/README.md index c79d11a3d..07ccd32fd 100644 --- a/README.md +++ b/README.md @@ -22,8 +22,8 @@ Circuit tools currently only support qubit codes. - **Code families.** Construct surface, toric, Hamming, and Reed-Muller codes, along with hypergraph-product, lifted-product, balanced-product, bivariate-bicycle, quantum Tanner, SHYPS, GALA, and other families. - **Distance calculations.** - Compute exact distances of binary codes with an [optimized Brouwer-Zimmermann search](src/qldpc/codes/distance/brouwer_zimmermann.md). - Codes with sparse checks can instead use a [connected-cluster search](src/qldpc/codes/distance/connected_cluster.md). + Compute exact distances of binary codes with an [optimized Brouwer-Zimmermann search](src/qldpc/codes/distance/brouwer_zimmermann.md) or a [connected-cluster search](src/qldpc/codes/distance/connected_cluster.md), which can be much faster on quantum codes with sparse checks. + By default, each search chooses its method, and enumerates every word if that is cheap. Bound code distance using [QDistRnd](https://arxiv.org/abs/2308.15140) or [`sqetch`](https://github.com/a7b/yarn/tree/main/sqetch). - **Code structure and analysis.** Work with check matrices, Tanner graphs, canonical logical operators, subsystem gauge operators, and code-capacity error estimates. diff --git a/docs/source/exact_search_tuning.rst b/docs/source/exact_search_tuning.rst index 3483176ff..6ce83188e 100644 --- a/docs/source/exact_search_tuning.rst +++ b/docs/source/exact_search_tuning.rst @@ -18,8 +18,15 @@ Start from a symptom - What to try first, then measure * - A distance search of a classical code of low dimension, whose distance is a large fraction of its length, is slow - Pass ``method="brute_force"`` to enumerate every word instead. - * - A distance search of a quantum LDPC code is slow - - Pass ``method="connected_cluster"``, which is often much faster on such codes, though not on all of them (`Section 2 `__ of the connected-cluster guide). + * - A distance search of a quantum LDPC code by ``method="brouwer_zimmermann"`` is slow + - Pass ``method="connected_cluster"``, which is often much faster on such codes, though not on all of them (`Section 2 `__ of the connected-cluster guide), or ``method="auto"``, the default, which starts with that search and may switch to the Brouwer-Zimmermann method. + * - A distance search by ``method="auto"``, the default, is slow + - Time the search with ``method="brouwer_zimmermann"`` and with ``method="connected_cluster"``, and pass the faster method. + With a large ``cutoff``, ``"auto"`` cannot switch to the Brouwer-Zimmermann method before it has searched past the cutoff, and with one near the distance it may never switch, so pass ``method="brouwer_zimmermann"``. + To make ``"auto"`` stay longer with a connected-cluster search, lower :attr:`~SearchTuning.connected_cluster_state_evaluations`, and to make it switch to the Brouwer-Zimmermann method sooner, raise that field. + Raise :attr:`~SearchTuning.auto_free_work` to decide later, with better estimates of growth, or lower it to decide sooner (`Choosing a method `__ in the guide to the distance package). + Raise :attr:`~SearchTuning.auto_probe_work` to let the short Brouwer-Zimmermann search that ``"auto"`` runs before it decides, its probe, finish more searches, or find a lighter word for the connected-cluster search to stop at, at a higher fixed cost. + Lower :attr:`~SearchTuning.auto_brute_force_max_words` if ``"auto"`` enumerates every word for too long. * - A hard distance search spends its time in its last, costliest layers - Raise :attr:`~SearchTuning.completion_bound_max_cost_percent` or lower :attr:`~SearchTuning.completion_bound_minimum_layer_work` to use completion tables in more layers. For a basis too large for its tables, raise :attr:`~SearchTuning.completion_bound_max_table_bytes` or lower :attr:`~SearchTuning.completion_bound_block_bits`. @@ -42,6 +49,7 @@ Start from a symptom * - A keyboard interrupt takes long to take effect - Lower :attr:`~SearchTuning.enumeration_batch_exponent`, :attr:`~SearchTuning.information_set_batch_exponent`, :attr:`~SearchTuning.scan_chunk_size`, :attr:`~SearchTuning.scan_chunks_per_call`, or :attr:`~SearchTuning.macwilliams_chunk_size`. A connected-cluster search takes an interrupt only between its weight limits, or between coset queries, and no value changes that. + ``method="auto"``, the default, runs one for the distances of quantum codes and the fault distances of circuits; pass ``method="brouwer_zimmermann"`` for prompt interrupts. * - A cheap connected-cluster search spends its time finding automorphisms - Raise :attr:`~SearchTuning.automorphism_minimum_states`. Searches ask for them before a first limit that ``cutoff`` sets, unless the field is ``MAX_VALUE``. @@ -67,6 +75,10 @@ Routes and the names that control them - :attr:`SearchTuning.enumeration_batch_exponent` - - `Section 3.6 `__ + * - Choosing a method, which ``method="auto"`` does for each distance search + - :attr:`SearchTuning.auto_brute_force_max_words`, :attr:`SearchTuning.auto_free_work`, :attr:`SearchTuning.auto_probe_work`, :attr:`SearchTuning.connected_cluster_state_evaluations` + - + - `Choosing a method `__ in the guide to the distance package * - Estimating the work of searches of information sets, which chooses between their routes - - :data:`BINOMIAL_CACHE_SIZE` diff --git a/docs/source/library_map.rst b/docs/source/library_map.rst index 16a862fc8..0a07e067c 100644 --- a/docs/source/library_map.rst +++ b/docs/source/library_map.rst @@ -132,8 +132,8 @@ Distances across packages ``get_distance()`` computes an exact distance and ``get_distance_bound`` a randomized upper bound, and CSS codes also have separate X and Z distances. The distance of a subsystem code is dressed: its logical operators may be multiplied by gauge operators. For a binary subsystem code, :func:`~qldpc.codes.distance.get_distance_from_checks` says how to find the bare distance: the minimum weight of a nontrivial logical operator that commutes with every gauge operator. -Exact distances of binary codes use an optimized `Brouwer-Zimmermann search `_, and nonbinary codes use exhaustive enumeration. -Binary codes with sparse parity checks can instead use ``method="connected_cluster"``, a `search over supports that are connected in the Tanner graph `_. +Exact distances of binary codes use an optimized `Brouwer-Zimmermann search `_ or, for sparse parity checks, a `search over supports that are connected in the Tanner graph `_, and nonbinary codes use exhaustive enumeration. +By default, ``method="auto"`` chooses a method for each search: exhaustive enumeration if it is cheap, the Brouwer-Zimmermann search for a classical code, and, for the checks of a quantum code or a circuit, a connected-cluster search that may switch to the Brouwer-Zimmermann search (see :data:`~qldpc.codes.distance.DistanceMethod`). All exact methods can take exponential time. A code that encodes nothing, or a circuit with no undetected logical fault, has no distance, and distance methods then return :data:`~qldpc.codes.distance.NO_DISTANCE`, a float ``nan`` (see :class:`~qldpc.codes.distance.NoDistance`). ``get_distance_exact(cutoff=c)`` stops once it finds a nontrivial word of weight at most ``c``. @@ -228,15 +228,15 @@ Support and limitations The following table summarizes support by field; the subsystem column applies to subsystem codes over any field. -=============== ========================================= ============================== =================================== ======================================== -Capability :math:`GF(2)` Prime field :math:`GF(p)` Extension field :math:`GF(p^m)` Subsystem codes -=============== ========================================= ============================== =================================== ======================================== -Codes Yes Yes Yes Yes, inferred from non-commuting checks -Exact distance Brouwer-Zimmermann, or connected cluster Exhaustive enumeration Exhaustive enumeration Yes -Distance bounds Decoder, GAP, or ``sqetch`` Decoder or GAP Decoder or GAP CSS codes only -Decoders All built-in decoders GUF (default), ILP, and lookup GUF (default) and lookup Yes, in code-capacity estimates -Circuits Yes No No No memory experiments -=============== ========================================= ============================== =================================== ======================================== +=============== ===================================================== ============================== =================================== ======================================== +Capability :math:`GF(2)` Prime field :math:`GF(p)` Extension field :math:`GF(p^m)` Subsystem codes +=============== ===================================================== ============================== =================================== ======================================== +Codes Yes Yes Yes Yes, inferred from non-commuting checks +Exact distance Brouwer-Zimmermann, connected cluster, or enumeration Exhaustive enumeration Exhaustive enumeration Yes +Distance bounds Decoder, GAP, or ``sqetch`` Decoder or GAP Decoder or GAP CSS codes only +Decoders All built-in decoders GUF (default), ILP, and lookup GUF (default) and lookup Yes, in code-capacity estimates +Circuits Yes No No No memory experiments +=============== ===================================================== ============================== =================================== ======================================== Other limitations: diff --git a/examples/logical_error_rates/1_code_capacity.ipynb b/examples/logical_error_rates/1_code_capacity.ipynb index 53e758ef7..295fb7031 100644 --- a/examples/logical_error_rates/1_code_capacity.ipynb +++ b/examples/logical_error_rates/1_code_capacity.ipynb @@ -284,7 +284,7 @@ "Each qubit of these codes participates in three checks of each type, so a single error can flip more than two checks and decoding is no longer a matching problem.\n", "We therefore use belief propagation with localized statistics decoding (BP+LSD) instead of MWPM.\n", "Here we set `compute_distance=True` to include exact distances in the labels.\n", - "The Brouwer-Zimmermann search has exponential worst-case cost and can take a long time for larger codes." + "Exact distance searches have exponential worst-case cost and can take a long time for larger codes." ] }, { diff --git a/examples/logical_error_rates/3_quantum_memory_combined.ipynb b/examples/logical_error_rates/3_quantum_memory_combined.ipynb index 1268f3a80..4b3ec6661 100644 --- a/examples/logical_error_rates/3_quantum_memory_combined.ipynb +++ b/examples/logical_error_rates/3_quantum_memory_combined.ipynb @@ -278,7 +278,7 @@ "\n", "A bivariate bicycle (BB) code from [arXiv:2308.07915](https://arxiv.org/abs/2308.07915) provides a higher-rate example with parameters $[[72,12,6]]$: 72 physical qubits, 12 logical qubits, and distance 6.\n", "Here we set `compute_distance=True` to recompute the exact distance for the plot label.\n", - "The Brouwer-Zimmermann search has exponential worst-case cost and can take a long time for larger codes.\n", + "Exact distance searches have exponential worst-case cost and can take a long time for larger codes.\n", "The experiment uses six syndrome-measurement rounds and decodes each CSS sector with BP+LSD.\n", "It tracks 24 logical observables and counts each experiment as a failure if any prediction is wrong." ] diff --git a/examples/logical_error_rates/5_fault_tolerance_certificates.ipynb b/examples/logical_error_rates/5_fault_tolerance_certificates.ipynb index 0dbd96488..ab19e4a25 100644 --- a/examples/logical_error_rates/5_fault_tolerance_certificates.ipynb +++ b/examples/logical_error_rates/5_fault_tolerance_certificates.ipynb @@ -127,9 +127,9 @@ "An error mechanism is an error instruction in the circuit's detector error model (DEM).\n", "Mechanisms with identical detector and observable effects can be merged, so mechanism indices are not physical gate-location indices.\n", "A fault is a set of error mechanisms, and the weight of a fault is the number of error mechanisms it contains.\n", - "`circuits.get_fault_distance` searches for a minimum-weight fault with no detector flips and at least one observable flip.\n", - "This method converts the problem of finding a minimum-weight fault into that of computing the Z-distance of an auxiliary Calderbank-Shor-Steane (CSS) code, and uses the Brouwer-Zimmermann distance-finding algorithm.\n", - "For a derivation of this reduction, see the documentation [here](https://qldpc.readthedocs.io/autoapi/qldpc/circuits/certification/fault_distance/index.html)." + "`circuits.get_fault_distance` searches exactly for a minimum-weight fault with no detector flips and at least one observable flip.\n", + "It computes this weight with `codes.distance.get_distance_from_checks`, as the distance of a code whose parity checks are the detectors, in which faults that flip no observable do not count.\n", + "For details, see the documentation of `get_fault_distance` [here](https://qldpc.readthedocs.io/autoapi/qldpc/circuits/certification/fault_distance/index.html)." ] }, { @@ -141,7 +141,7 @@ "\n", "For comparison, we use a bivariate bicycle code from [arXiv:2308.07915](https://arxiv.org/abs/2308.07915) with parameters $[[72,12,6]]$: 72 physical qubits, 12 logical qubits, and distance 6.\n", "Here each basis has 432 mechanisms after equivalent ones are merged.\n", - "Checking all combinations of up to six would require roughly nine trillion combinations; the code-distance reduction avoids that exhaustive enumeration." + "Checking all combinations of up to six would require roughly nine trillion combinations; this reduction to a code distance avoids that exhaustive enumeration." ] }, { diff --git a/experiments/distance_tuning/trial.py b/experiments/distance_tuning/trial.py index c5d3ae81e..27e781635 100644 --- a/experiments/distance_tuning/trial.py +++ b/experiments/distance_tuning/trial.py @@ -156,7 +156,7 @@ def warm_up(kind: str) -> None: elif kind == "hadamard": hadamard_code = workloads.get_hadamard_code("surface:5") hadamard_code.forget_distance() - hadamard_code.get_distance_exact() + hadamard_code.get_distance_exact(method="brouwer_zimmermann") elif kind == "count": for code, weight in ( (codes.SurfaceCode(5), 5), diff --git a/experiments/distance_tuning/workloads.py b/experiments/distance_tuning/workloads.py index 798b97ac9..cf4eeced6 100644 --- a/experiments/distance_tuning/workloads.py +++ b/experiments/distance_tuning/workloads.py @@ -25,11 +25,11 @@ A workload name is a kind, then a code name, then the arguments of the kind, separated by ``/``: -- ``distance//``: ``get_distance_exact`` of a quantum code, for the sector ``X``, - ``Z``, or ``both``, or of a classical code, whose sector must be ``-``; +- ``distance//``: ``get_distance_exact``, by the BZ method, of a quantum code, for the + sector ``X``, ``Z``, or ``both``, or of a classical code, whose sector must be ``-``; - ``ccdistance//``: the same, by the connected-cluster method; -- ``hadamard/``: ``get_distance_exact`` of the code that Hadamards on the second half of the - qubits of a CSS code make, which is not CSS; +- ``hadamard/``: ``get_distance_exact``, by the BZ method, of the code that Hadamards on the + second half of the qubits of a CSS code make, which is not CSS; - ``count//``: ``CSSLogicalCounter.count_logical_ops(weight)`` by the BZ method; - ``ccount//``: the same count, in a process whose warm-up first calls ``compile_search_setup`` (see ``trial.py``); @@ -52,9 +52,9 @@ - ``certify///``: ``certify_bounded_fault_propagation`` of the encoding circuit of ``|0...0>`` of a CSS code under ``DepolarizingNoiseModel(1e-3)``, by ``css`` or ``physical`` weight, whose result is the fault weight of the smallest violation, if any; -- ``faultdist//``: ``get_fault_distance`` of one round of a memory experiment of a - CSS code in the basis ``X`` or ``Z``, with noise only on its QEC cycle, as in the notebook of - fault-tolerance certificates. +- ``faultdist//``: ``get_fault_distance``, by the BZ method, of one round of a memory + experiment of a CSS code in the basis ``X`` or ``Z``, with noise only on its QEC cycle, as in the + notebook of fault-tolerance certificates. Each workload returns a short string of its result, which every tuning must reproduce. """ @@ -269,10 +269,13 @@ def run_distance( def run_hadamard_distance(code_name: str) -> str: - """The exact distance of a CSS code after Hadamards on the second half of its qubits.""" + """The distance, by the BZ method, of a CSS code after Hadamards. + + The Hadamards act on the second half of its qubits. + """ code = get_hadamard_code(code_name) code.forget_distance() - return str(code.get_distance_exact()) + return str(code.get_distance_exact(method="brouwer_zimmermann")) @functools.cache @@ -426,10 +429,11 @@ def get_noisy_encoding_circuit(code_name: str) -> stim.Circuit: def run_fault_distance(code_name: str, basis: str) -> str: - """The fault distance of one round of a memory experiment with a noisy QEC cycle.""" + """The fault distance, by the BZ method, of one memory round with a noisy QEC cycle.""" if basis not in ("X", "Z"): raise ValueError(f"Unknown basis {basis!r}; choose 'X' or 'Z'") - return str(circuits.get_fault_distance(get_noisy_memory_circuit(code_name, basis))) + circuit = get_noisy_memory_circuit(code_name, basis) + return str(circuits.get_fault_distance(circuit, method="brouwer_zimmermann")) @functools.cache diff --git a/src/qldpc/_deprecations/core.py b/src/qldpc/_deprecations/core.py index ce5b5e345..cbad02624 100644 --- a/src/qldpc/_deprecations/core.py +++ b/src/qldpc/_deprecations/core.py @@ -384,7 +384,7 @@ def get_distance_classical( *, cutoff: int = 1, block_size: int | None = None, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, ) -> int | Literal[NoDistance.NAN]: """Deprecated: the distance of a classical linear binary code, from its generator matrix. @@ -411,7 +411,7 @@ def get_distance_quantum( cutoff: int = 1, block_size: int | None = None, homogeneous: bool = False, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, ) -> int | Literal[NoDistance.NAN]: """Deprecated: the distance of a binary quantum code from its logical operators and stabilizers. diff --git a/src/qldpc/_deprecations/core_test.py b/src/qldpc/_deprecations/core_test.py index 1cb4c416b..d4a9287e4 100644 --- a/src/qldpc/_deprecations/core_test.py +++ b/src/qldpc/_deprecations/core_test.py @@ -270,7 +270,7 @@ def test_deprecated_distance_functions() -> None: ): assert search() == distance - # Both functions pass on their method. + # Both functions pass on their method, which defaults to "auto". get_distance_classical: Any = codes.get_distance_classical # type: ignore[attr-defined] get_distance_quantum: Any = codes.get_distance_quantum # type: ignore[attr-defined] for search in ( @@ -281,6 +281,18 @@ def test_deprecated_distance_functions() -> None: ): with pytest.warns(DeprecationWarning), pytest.raises(ValueError, match="from_checks"): search() + with unittest.mock.patch.object( + codes.distance, + "get_distance_from_generators", + wraps=codes.distance.get_distance_from_generators, + ) as get_distance: + for search in ( + lambda: get_distance_classical(generator), + lambda: get_distance_quantum(generator[:1], generator[1:], homogeneous=True), + ): + with pytest.warns(DeprecationWarning): + assert search() == 3 + assert get_distance.call_args.kwargs["method"] == "auto" # get_distance_quantum required its stabilizers, so it rejects None, and its errors name its own # arguments. diff --git a/src/qldpc/circuits/certification/fault_distance.py b/src/qldpc/circuits/certification/fault_distance.py index 90cbeb934..737911715 100644 --- a/src/qldpc/circuits/certification/fault_distance.py +++ b/src/qldpc/circuits/certification/fault_distance.py @@ -18,7 +18,7 @@ def get_fault_distance( - circuit: stim.Circuit, *, method: DistanceMethod = "brouwer_zimmermann" + circuit: stim.Circuit, *, method: DistanceMethod = "auto" ) -> int | Literal[NoDistance.NAN]: r"""Compute the exact fault distance of a noisy circuit. @@ -44,10 +44,11 @@ def get_fault_distance( Args: circuit: A noisy Stim circuit annotated with detectors and logical observables. - method: The exact-distance method: ``"brouwer_zimmermann"`` (the default, with alias - ``"information_sets"``), ``"brute_force"``, or ``"connected_cluster"``, which searches - the sparse detector matrix :math:`D` for undetected faults that are connected through - its detectors (see :data:`~qldpc.codes.distance.DistanceMethod`). + method: The exact-distance method: ``"auto"`` (the default), ``"brouwer_zimmermann"`` + (with alias ``"information_sets"``), ``"brute_force"``, or ``"connected_cluster"`` (see + :data:`~qldpc.codes.distance.DistanceMethod`). ``"auto"`` chooses among the others. + ``"connected_cluster"`` searches the sparse detector matrix :math:`D` for undetected + faults that are connected through its detectors. Returns: The minimum number of DEM error mechanisms causing an undetected observable flip, or @@ -59,8 +60,8 @@ def get_fault_distance( # Check the method before the costly analysis of the circuit. if method not in get_args(DistanceMethod): raise ValueError( - f"Unknown distance method {method!r}; choose from 'brouwer_zimmermann' (or its alias" - " 'information_sets'), 'brute_force', or 'connected_cluster'." + f"Unknown distance method {method!r}; choose from 'auto', 'brouwer_zimmermann' (or its" + " alias 'information_sets'), 'brute_force', or 'connected_cluster'." ) dem_arrays = decoders.DetectorErrorModelArrays(circuit) detectors = dem_arrays.detector_flip_matrix.toarray().astype(np.uint8) diff --git a/src/qldpc/circuits/certification/fault_distance_test.py b/src/qldpc/circuits/certification/fault_distance_test.py index a67bd5d0d..8eeb252e3 100644 --- a/src/qldpc/circuits/certification/fault_distance_test.py +++ b/src/qldpc/circuits/certification/fault_distance_test.py @@ -10,10 +10,13 @@ import stim from qldpc import circuits, codes +from qldpc.circuits.certification import fault_distance from qldpc.codes.distance import DistanceMethod -@pytest.mark.parametrize("method", ["brouwer_zimmermann", "brute_force", "connected_cluster"]) +@pytest.mark.parametrize( + "method", ["auto", "brouwer_zimmermann", "brute_force", "connected_cluster"] +) @pytest.mark.parametrize("measurement, expected_distance", [(-7, 3), (-4, 4)]) def test_fault_distance(measurement: int, expected_distance: int, method: DistanceMethod) -> None: """Undetected faults flip whole blocks of three or four qubits; only one block is logical.""" @@ -32,6 +35,21 @@ def test_fault_distance(measurement: int, expected_distance: int, method: Distan assert circuits.get_fault_distance(circuit, method=method) == expected_distance +def test_fault_distance_takes_auto_by_default(monkeypatch: pytest.MonkeyPatch) -> None: + """Without a method, the fault distance searches with "auto", the default.""" + methods = [] + get_distance_from_checks = fault_distance.get_distance_from_checks + + def record(*args: Any, **kwargs: Any) -> Any: + methods.append(kwargs["method"]) + return get_distance_from_checks(*args, **kwargs) + + monkeypatch.setattr(fault_distance, "get_distance_from_checks", record) + circuit = stim.Circuit("R 0\nX_ERROR(0.01) 0\nM 0\nOBSERVABLE_INCLUDE(0) rec[-1]") + assert circuits.get_fault_distance(circuit) == 1 + assert methods == ["auto"] + + def test_fault_distance_without_logical_faults() -> None: """Without an undetected observable flip, a circuit has no distance, and gives NO_DISTANCE.""" circuit = stim.Circuit("R 0\nX_ERROR(0.01) 0\nM 0\nDETECTOR rec[-1]") diff --git a/src/qldpc/circuits/certification/fault_propagation.py b/src/qldpc/circuits/certification/fault_propagation.py index 91cc2402d..cb36ef111 100644 --- a/src/qldpc/circuits/certification/fault_propagation.py +++ b/src/qldpc/circuits/certification/fault_propagation.py @@ -103,8 +103,9 @@ def certify_bounded_fault_propagation( result. Default: ``2**30`` (1 GiB). method: The exact search of residual weights: ``"brouwer_zimmermann"``, its alias ``"information_sets"``, or ``"connected_cluster"`` (see - :class:`~qldpc.codes.distance.CosetLeaderSearch`). The methods certify the same - circuits, and find the same smallest violating fault, but the ``output_pauli`` and + :class:`~qldpc.codes.distance.CosetLeaderSearch`); ``"auto"`` takes + ``"brouwer_zimmermann"`` here. The methods certify the same circuits, and find the + same smallest violating fault, but the ``output_pauli`` and ``residual_weight_lower_bound`` of a counterexample can differ (see ``BoundedFaultPropagationCounterexample``). With ``"brouwer_zimmermann"``, the ``output_pauli`` and ``residual_weight_lower_bound`` can also change between versions @@ -261,10 +262,10 @@ class BoundedFaultPropagationCounterexample: No fault that passes post-selection and has a smaller fault weight violates the bound. Equivalence, fault weight, Pauli weight, output error, residual weight, and the bound are as - defined in ``certify_bounded_fault_propagation``. With the method ``"brouwer_zimmermann"``, the - ``output_pauli`` and ``residual_weight_lower_bound`` of a counterexample can change between - versions of qLDPC, as the searches that find them change (see ``method`` in - ``certify_bounded_fault_propagation``). + defined in ``certify_bounded_fault_propagation``. With the method ``"brouwer_zimmermann"``, or + ``"auto"``, which takes it here, the ``output_pauli`` and ``residual_weight_lower_bound`` of a + counterexample can change between versions of qLDPC, as the searches that find them change (see + ``method`` in ``certify_bounded_fault_propagation``). Attributes: fault_indices: The indices of the fault's error mechanisms among the error instructions of @@ -437,13 +438,13 @@ def _check_coset_search_method(method: DistanceMethod) -> None: """ if method == "brute_force": raise ValueError( - "The 'brute_force' distance method has no search of residual weights: use" - " 'brouwer_zimmermann' or 'connected_cluster'." + "The 'brute_force' distance method has no search of residual weights: use 'auto'," + " 'brouwer_zimmermann', or 'connected_cluster'." ) if method not in typing.get_args(DistanceMethod): raise ValueError( - f"Unknown distance method {method!r}; choose from 'brouwer_zimmermann' (or its alias" - " 'information_sets') or 'connected_cluster'." + f"Unknown distance method {method!r}; choose from 'auto', 'brouwer_zimmermann' (or its" + " alias 'information_sets'), or 'connected_cluster'." ) diff --git a/src/qldpc/codes/common.py b/src/qldpc/codes/common.py index c5e4484bb..d564b6c7f 100644 --- a/src/qldpc/codes/common.py +++ b/src/qldpc/codes/common.py @@ -503,7 +503,7 @@ def get_distance( bound: int | bool | None = None, vector: Sequence[int] | npt.NDArray[np.int_] | None = None, decoder: decoders.ErrorDecoderInput = None, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", **bound_kwargs: Any, ) -> int | Literal[NoDistance.NAN]: """Compute (or upper bound) the minimum Hamming weight of nontrivial code words. @@ -546,7 +546,7 @@ def get_distance_exact( *, vector: Sequence[int] | npt.NDArray[np.int_] | None = None, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", ) -> int | Literal[NoDistance.NAN]: """Compute the exact minimum Hamming weight of nontrivial code words. @@ -1966,7 +1966,7 @@ def get_distance( self, *, bound: int | bool | None = None, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", **bound_kwargs: Any, ) -> int | Literal[NoDistance.NAN]: """Compute (or upper bound) the minimum weight of nontrivial logical operators. @@ -2007,7 +2007,7 @@ def get_distance_exact( self, *, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", ) -> int | Literal[NoDistance.NAN]: """Compute the exact minimum weight of nontrivial logical operators. @@ -3594,7 +3594,7 @@ def get_distance( *, bound: int | bool | None = None, decoder: decoders.DeferredErrorDecoderInput = None, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", **bound_kwargs: Any, ) -> int | Literal[NoDistance.NAN]: """Compute (or upper bound) the minimum weight of nontrivial logical operators. @@ -3643,7 +3643,7 @@ def get_distance_exact( pauli: PauliXZLike | None = None, *, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", ) -> int | Literal[NoDistance.NAN]: """Compute the exact minimum weight of nontrivial logical operators. @@ -3690,7 +3690,7 @@ def get_distance_exact( # Brute force searches the two sectors separately, so it may as well remember both. if ( pauli is None - and method in ("brouwer_zimmermann", "connected_cluster") + and method != "brute_force" and self.field is galois.GF2 and type(self)._get_distance_exact is CSSCode._get_distance_exact ): diff --git a/src/qldpc/codes/common_test.py b/src/qldpc/codes/common_test.py index 14a9dc934..dfc774356 100644 --- a/src/qldpc/codes/common_test.py +++ b/src/qldpc/codes/common_test.py @@ -770,9 +770,24 @@ def test_exact_code_distance_method_api() -> None: ("get_distance_pauli", lambda: qudit_code.get_distance(method="brute_force")), ("get_distance_css", lambda: css_code.get_distance(Pauli.X, method="brute_force")), ): - with unittest.mock.patch(f"qldpc.codes.common.{kernel}", return_value=3) as mock_kernel: + with unittest.mock.patch.object(common, kernel, wraps=getattr(common, kernel)) as search: assert get_distance() == 3 - assert mock_kernel.call_args.kwargs["method"] == "brute_force" + assert search.call_args.kwargs["method"] == "brute_force" + + # Without a method, get_distance and get_distance_exact forward "auto", the default. + for kernel, get_distance in ( + ("get_distance_from_generators", classical_code.get_distance), + ("get_distance_from_generators", classical_code.get_distance_exact), + ("get_distance_pauli", qudit_code.get_distance), + ("get_distance_pauli", qudit_code.get_distance_exact), + ("get_distance_css", functools.partial(css_code.get_distance, Pauli.X)), + ("get_distance_css", functools.partial(css_code.get_distance_exact, Pauli.X)), + ): + for code in (classical_code, qudit_code, css_code): + code.forget_distance() + with unittest.mock.patch.object(common, kernel, wraps=getattr(common, kernel)) as search: + assert get_distance() == 3 + assert search.call_args.kwargs["method"] == "auto" redundant_code = codes.ClassicalCode([[1, 1, 1]]).set_generator( [[1, 1, 0], [0, 1, 1], [1, 0, 1]] @@ -786,17 +801,17 @@ def test_exact_code_distance_method_api() -> None: # The "information_sets" alias selects the joint Brouwer-Zimmermann search of a CSS code. hgp_code = codes.HGPCode(codes.RepetitionCode(3), codes.RepetitionCode(4)) css_code = codes.CSSCode(hgp_code.matrix_x, hgp_code.matrix_z) - with unittest.mock.patch( - "qldpc.codes.common.get_distance_css", return_value=3 - ) as mock_joint_distance: + with unittest.mock.patch.object( + common, "get_distance_css", wraps=common.get_distance_css + ) as joint_search: assert css_code.get_distance_exact(method="information_sets") == 3 - mock_joint_distance.assert_called_once() + joint_search.assert_called_once() arguments = ( inspect.signature(get_distance_css) - .bind(*mock_joint_distance.call_args.args, **mock_joint_distance.call_args.kwargs) + .bind(*joint_search.call_args.args, **joint_search.call_args.kwargs) .arguments ) - assert arguments["pauli"] is None + assert arguments["pauli"] is None and arguments["method"] == "brouwer_zimmermann" def test_connected_cluster_method_is_only_for_binary_codes() -> None: @@ -1080,7 +1095,7 @@ def test_distances_with_bliss(monkeypatch: pytest.MonkeyPatch, tune: Tune) -> No for matrix in (bb_code.matrix, bb_code.conjugated(range(36, 72)).matrix): code = codes.QuditCode(np.hstack([matrix[:, shuffle], matrix[:, len(bb_code) + shuffle]])) assert code._found_qudit_automorphisms is not None - assert code.get_distance_exact() == 6 + assert code.get_distance_exact(method="brouwer_zimmermann") == 6 @pytest.mark.parametrize("field", [2, 3]) diff --git a/src/qldpc/codes/distance/README.md b/src/qldpc/codes/distance/README.md index 7902d9298..b0cca19f5 100644 --- a/src/qldpc/codes/distance/README.md +++ b/src/qldpc/codes/distance/README.md @@ -28,6 +28,7 @@ The documentation cites the other names at `qldpc.codes.distance`. - **Basis family** (`BasisFamily`): the information sets that one Brouwer-Zimmermann search or count uses together, stacked into arrays. - **Certificate** (`Certificate`): a lower bound on the weight of the words that a Brouwer-Zimmermann search has neither visited nor skipped, which tells the search when it can stop; with completion tables, it bounds only the words lighter than the best weight `U` of the search (Section 3.9 of `brouwer_zimmermann.md`). - **Connected-cluster search** (`SupportSearch`): the depth-first search that `method="connected_cluster"` selects, which grows supports through the unsatisfied rows of a check matrix `H` to find light binary or symplectic words outside the row space of an excluded matrix `G`, or light vectors with a given syndrome. + `"auto"` runs it for the distances of quantum codes and the fault distances of circuits, unless enumerating is cheap (see "Choosing a method"). - **Scan:** the enumeration of the combinations of rows of each information set, layer by layer, a layer holding the combinations of a given number of rows. Counts have their own scan, in `_counting/scan.py`, and the Brouwer-Zimmermann distance and coset searches (see "The coset-leader search") use the compiled scans of `_brouwer_zimmermann/affine_search.py`. Every scan steps through a layer with `advance_support` in `_information_sets/support_stepping.py`. @@ -36,6 +37,7 @@ The documentation cites the other names at `qldpc.codes.distance`. - `core.py` and `logical_counts.py` are the public interfaces of the distances and the counts. - `coset_leaders.py` is the public interface of the coset-leader search, and `min_weight_bounds.py` defines `MinWeightBound`, which its coset searches build too. +- `_auto.py` holds the parts of the automatic choice of a method, `method="auto"`, that compare the costs of the methods (see "Choosing a method"). - `_brouwer_zimmermann/` holds the Brouwer-Zimmermann searches, which scan information sets, and `_connected_cluster/` the connected-cluster searches, which grow connected supports. Each holds the distance search of its method, and the coset searches with which the coset-leader search answers queries by that method. `_brute_force.py` holds exhaustive enumeration. @@ -52,6 +54,7 @@ The documentation cites the other names at `qldpc.codes.distance`. | To change… | Look in (this directory) | | --- | --- | | A distance method, or the validation of method names | `core.py`, whose validation `coset_leaders.py` also uses; the early checks of `src/qldpc/circuits/certification/fault_propagation.py` and `src/qldpc/circuits/certification/fault_distance.py`; where `logical_counts.py` and `get_distance_exact` in `src/qldpc/codes/common.py` dispatch on the method name, and where `get_distance` validates it; and the `brute_force` flag that `logical_counts.py` passes to `_counting/scan.py` | +| The automatic choice of a method (`method="auto"`): which searches take which method, and when a connected-cluster search switches to the BZ method | `core.py`, where the distance functions dispatch on the method; `_auto.py`, which estimates and compares the costs; and `_get_coset_search_method` of `coset_leaders.py`, where coset searches take BZ | | The counter, its deformed counters, or `compile_search_setup` | `logical_counts.py` | | The coset-leader search, which `src/qldpc/circuits/certification/` and `src/qldpc/experimental/circuits/benchmarking/residual_weights.py` use, and how its queries split into coset searches | `coset_leaders.py` | | The Brouwer-Zimmermann (BZ) distance search | `_brouwer_zimmermann/distance.py` | @@ -86,11 +89,12 @@ From the top: 1. `logical_counts.py`, which takes `NoDistance` from `core.py`, and `coset_leaders.py`, which takes `DistanceMethod` from it. 2. `core.py`. -3. The methods `_brouwer_zimmermann/`, `_connected_cluster/`, `_counting/`, and `_brute_force.py`. -4. `_information_sets/`, `_cosets.py`, and `min_weight_bounds.py`, which the methods build on. +3. `_auto.py`, the automatic choice of a method. +4. The methods `_brouwer_zimmermann/`, `_connected_cluster/`, `_counting/`, and `_brute_force.py`. +5. `_information_sets/`, `_cosets.py`, and `min_weight_bounds.py`, which the methods build on. The connected-cluster search uses `infer_weight_divisor`, `verify_automorphisms`, and `get_orbits` of `_information_sets/`. -5. `_tuning.py`. -6. `_binary.py`. +6. `_tuning.py`. +7. `_binary.py`. A top-level module or subpackage imports only from lower levels, and a module also from its own subpackage. `_layering_test.py` checks this. @@ -103,6 +107,76 @@ Modules of every level may also import `src/qldpc/_binary.py`, the bit-packed li `src/qldpc/codes/common.py` imports this package, so `logical_counts.py` imports the code classes of `codes.common` inside the functions that use them. Modules elsewhere in qLDPC, such as `src/qldpc/codes/common.py` and the deprecation registry in `src/qldpc/_deprecations/core.py`, use private names of this package, so grep `src/qldpc` for a private name before you move or rename it. +## Choosing a method + +`method="auto"` is the default of the distance functions, of `get_distance` and `get_distance_exact` of the code classes, and of `get_fault_distance`. +It chooses a method for each search, in `_search_css`, `_search_checks`, and `_search_generators` of `core.py`. +A code class that searches other codes, as `HGPCode` searches its seed codes, passes the method down, so each of those searches chooses too. +The coset searches of `CosetLeaderSearch` take the BZ method for `"auto"` (`_get_coset_search_method` of `coset_leaders.py`). +The rule aims to avoid a method that is very slow on a search, not to find the fastest one, and it decides from the shapes of the sectors and from the progress of the searches, without fitted thresholds. + +1. Brute force runs if `is_cheap_to_enumerate` finds that it examines at most `auto_brute_force_max_words` packed 64-bit integers. + Unlike that of the other methods, its cost is known before it runs. +2. Otherwise, a search of binary words that excludes no nonzero word, as for a classical code, takes the BZ method, and so does every search of a function that takes generators, since it has no checks. + A connected-cluster search can be much slower than BZ on classical codes whose checks are only moderately sparse. +3. Every other search, as of a quantum code or of the faults of a circuit, runs `search_by_cost` in `_auto.py`. + It advances a connected-cluster search one weight limit at a time, through `ConnectedClusterSearch`, and counts work in BZ candidate/set evaluations, a state being `connected_cluster_state_evaluations` of them. + The expected work of a limit is the work that the search has spent plus the predicted work of that limit. + - In its free phase, the search runs on until the expected work of its next limit exceeds `auto_free_work`. + Below that, fixed costs that the estimates leave out, such as setup, dominate the costs that a switch would compare. + `SupportSearch._predict_states` predicts `MAX_VALUE - 1` states for a first limit that `cutoff` raises, so such a limit ends the free phase at once. + - `_probe` then runs a BZ search by whole layers within `auto_probe_work`. + It skips the costlier stages of setup: overlapping and partitioned families, and so the certificates from linear programs that only those take. + If the probe finishes, its result is the answer. + Otherwise, its shared upper bound `U`, the weight of a nontrivial word that it has found or the caller's upper bound if that is less, caps the connected-cluster search, which then stops once its lower bounds reach `U`. + The search of a sector can hold a lighter witness from its setup, which joins `U` only after the probe advances that search. + - Before each limit whose expected work exceeds `auto_free_work`, starting with the limit that ended the free phase, `_should_switch` may switch to a BZ search with the full tuning and the upper bound `U`. + The level of a limit is the weight that it certifies, one weight divisor above it. + The search switches only while its work grows faster per unit of weight than BZ's would at the level of its next limit. + It then switches if its spent and projected work exceeds the estimated work of BZ, to certify that level or to reach `U`. + A search has a growth only after two limits without a witness, so it cannot switch before then. + So a search with a cutoff, whose first limit is the cutoff, runs through it and one more limit before it can switch. + With a cutoff within about two weight divisors of the distance, it may never switch. + +The choice keeps every result exact, and `cutoff` and `upper_bound` mean what they mean for each method. +Since `U` is the weight of a nontrivial word or the caller's upper bound, a search with `U` as its upper bound returns the smaller of the distance and the caller's upper bound, as a search without it does. +When the connected-cluster search switches, a BZ search of the same sectors, in the same order, replaces it, and takes nothing from it but `U`. +For Pauli strings, that BZ search searches their images under `symplectic_to_hamming` (Section 2.7 of `brouwer_zimmermann.md`) and halves the weight that it finds. +The tuning of the probe changes only its family of information sets and their certificate, which no result depends on. +Which method runs can still change which word an early exit at `cutoff` finds. + +The estimates are crude. +The connected-cluster search predicts the states of a limit by `SupportSearch._predict_states` until two successive limits without a witness have visited states from the same number of roots. +It then extrapolates the states of its last limit without a witness by the growth of its states per unit of weight (`ConnectedClusterSearch.get_growth`). +`_BrouwerZimmermannCosts` estimates the work of BZ from the rank `r` and width `n` of each basis alone: a family that covers the columns evenly certifies about `(t + 1) n / r` after layer `t`, which scans `C(r, t)` supports of each set. +It counts one set per layer, as with an automorphism family, which underestimates a family of several sets. +It also leaves out completion tables, which can skip most supports of a costly layer, so it can overestimate the work of a hard BZ search. +To reach `U`, the projection counts half of the states predicted for the limit at weight `U` of the search of the next limit, as for a search that must find a witness of weight `U` itself. +A search with `U` as its upper bound stops before that limit, so this overestimates the connected-cluster work. +These two overestimates push the choice in opposite directions, so a refinement that removes one should remove the other. + +### Refining the choice + +The known ways for the rule to lose are these: + +- A classical code whose checks are sparse enough for a connected-cluster search to be much faster takes BZ. +- A quantum code on which BZ is much faster pays for the connected-cluster search up to the switch, the probe, and the setup of a BZ search that starts again, since the switch discards the probe. +- A quantum code on which the connected-cluster search would win switches to BZ if the estimates make BZ look cheaper than it is, as when a basis has columns that its information sets cannot cover evenly. +- A search with a cutoff near the distance may never switch (see above). + It then pays for the whole connected-cluster search unless its probe finishes. + +A better rule could use: + +- the work of each layer that a BZ search plans (`get_layer_work`), which counts the sets of its family, in place of the closed form; +- the work that the completion tables of a BZ search are predicted to skip; +- a projection to `U` that stops before the limit at `U`; +- a race that interleaves the two searches; +- more features of the sectors, such as the weights of their checks. + +It could also resume the probe rather than start a new BZ search. +To check a refinement, record its choice and its predicted cost for each code before you time it, and then time it on codes other than those it was designed on, against each method alone. +A refinement should never make a choice that is much slower, in absolute time, than the one it replaces. + ## The coset-leader search `CosetLeaderSearch` answers each query with one or two coset searches, each over the cosets of a binary code. diff --git a/src/qldpc/codes/distance/_auto.py b/src/qldpc/codes/distance/_auto.py new file mode 100644 index 000000000..078b1ba26 --- /dev/null +++ b/src/qldpc/codes/distance/_auto.py @@ -0,0 +1,264 @@ +# SPDX-License-Identifier: Apache-2.0 + +"""The parts of ``method="auto"`` that compare the costs of exact distance searches. + +``core.py`` chooses a method for each search. It enumerates every word if ``is_cheap_to_enumerate`` +says that this is cheap. Otherwise, it takes the Brouwer-Zimmermann (BZ) method for a search given +by generators, or for one that excludes no nonzero word, as for a classical code. Other searches, +as of a quantum code or of the faults of a circuit, run ``search_by_cost``: a connected-cluster +search that switches to a BZ search once that is predicted to cost less. The section "Choosing a +method" of ``README.md`` in ``src/qldpc/codes/distance/`` explains the rule, why it keeps results +exact, and its estimates. +""" + +from __future__ import annotations + +import dataclasses +import math +from collections.abc import Callable, Sequence + +import numpy as np +import numpy.typing as npt + +from ._brouwer_zimmermann.distance import ( + BrouwerZimmermannSearch, + get_distance_brouwer_zimmermann, +) +from ._connected_cluster.search import ConnectedClusterSearch +from ._tuning import MAX_VALUE, SearchTuning + +#################################################################################################### +# brute force + + +def is_cheap_to_enumerate(dimensions: Sequence[int], width: int, tuning: SearchTuning) -> bool: + """Whether brute force examines at most ``auto_brute_force_max_words`` packed 64-bit integers. + + Brute force enumerates the ``2**dimension`` words of each space of the given dimensions, and + packs each word of ``width`` bits into ``ceil(width / 64)`` integers of 64 bits. + """ + num_words = sum(1 << dimension for dimension in dimensions) + return num_words * -(-width // 64) <= tuning.auto_brute_force_max_words + + +#################################################################################################### +# searches by cost + + +@dataclasses.dataclass(frozen=True) +class BrouwerZimmermannProblem: + """The sectors of a distance search, as a BZ search takes them. + + Attributes: + sectors: Nested-code bases and their quotient labels, as + ``get_distance_brouwer_zimmermann`` takes them, one for each sector. + weight_scale: The factor that converts the weights of the searched words into those of + their BZ words: two for Pauli strings, whose BZ words are their images under + ``symplectic_to_hamming``, and one otherwise. + weight_divisor: A proven divisor of the weights of BZ words. Certificates round up to its + multiples (Section 3.2 of ``brouwer_zimmermann.md``). It equals ``weight_scale`` for + the problems that ``core.py`` builds. + get_automorphisms: A function that returns automorphisms of the coordinates of the BZ + words, or None, as ``get_distance_brouwer_zimmermann`` takes it. + """ + + sectors: Sequence[tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]] + weight_scale: int = 1 + weight_divisor: int = 1 + get_automorphisms: Callable[[], npt.NDArray[np.int_] | None] | None = None + + +def search_by_cost( + cluster_sectors: Sequence[tuple[npt.ArrayLike, npt.ArrayLike]], + brouwer_zimmermann_problem: BrouwerZimmermannProblem, + *, + symplectic: bool, + cutoff: int, + upper_bound: int | None, + get_automorphisms: Callable[[], npt.NDArray[np.int_] | None] | None, + tuning: SearchTuning, +) -> int: + """The minimum distance across sectors, by a connected-cluster search that may switch to BZ. + + The function runs a connected-cluster search, probes with a BZ search, and may switch to one, as + the section "Choosing a method" of ``README.md`` describes. The sectors and other arguments, + and the result, are those of ``get_distance_connected_cluster``. ``brouwer_zimmermann_problem`` + holds the same sectors, in the same order, for a BZ search. Each caller builds it as its own + route builds a BZ search, so it is an argument rather than derived here. Every sector must have + a witness: ``ConnectedClusterSearch`` keeps only the sectors that do, and its searches must be + numbered as the sectors of ``brouwer_zimmermann_problem``. + """ + clusters = ConnectedClusterSearch( + cluster_sectors, + cutoff=cutoff, + upper_bound=upper_bound, + symplectic=symplectic, + get_automorphisms=get_automorphisms, + tuning=tuning, + ) + assert len(clusters.searches) == len(brouwer_zimmermann_problem.sectors) + costs = _BrouwerZimmermannCosts(brouwer_zimmermann_problem) + state_cost = tuning.connected_cluster_state_evaluations + scale = brouwer_zimmermann_problem.weight_scale + probe_bound: int | None = None + while not clusters.finished: + # Decide only once the expected work of the next limit exceeds the free work. + index, max_weight = clusters.next_limit + spent = clusters.num_states * state_cost + if spent + clusters.predict_states(index, max_weight) * state_cost > tuning.auto_free_work: + # Probe once, and let the probe's bound end the connected-cluster search once every + # sector's lower bound reaches it. + if probe_bound is None: + probe = _probe(brouwer_zimmermann_problem, cutoff, upper_bound, tuning) + if probe.finished: + return probe.best // scale + probe_bound = probe.best // scale + clusters.update_upper_bound(probe_bound) + continue + + # Switch to a BZ search with the full tuning, which starts again, capped by the probe's + # bound. + if _should_switch(clusters, costs, probe_bound, state_cost): + return ( + get_distance_brouwer_zimmermann( + brouwer_zimmermann_problem.sectors, + cutoff=cutoff * scale, + weight_divisor=brouwer_zimmermann_problem.weight_divisor, + upper_bound=probe_bound * scale, + get_automorphisms=brouwer_zimmermann_problem.get_automorphisms, + tuning=tuning, + ) + // scale + ) + clusters.advance() + return clusters.result + + +class _BrouwerZimmermannCosts: + """Estimates of the work of a BZ search, from the shapes of its bases alone. + + A family of information sets of a basis of rank ``r`` and width ``n`` that covers the columns + evenly certifies the weight ``ceil((t + 1) n / r)`` after layer ``t``, rounded up to the weight + divisor (Section 2.5 of ``brouwer_zimmermann.md``), and scans ``C(r, t)`` supports in layer + ``t``. The estimates count one set per layer, as with an automorphism family, and leave out + completion tables. + """ + + def __init__(self, problem: BrouwerZimmermannProblem) -> None: + self._shapes = [basis.shape for basis, _ in problem.sectors] + self._scale = problem.weight_scale + self._divisor = problem.weight_divisor + + def get_work(self, level: int) -> int: + """The evaluations, over every sector, that certify that no witness weighs below ``level``. + + ``level`` is a weight of the sectors' words, before the scale. + """ + work = 0 + for rank, width in self._shapes: + num_layers = self._get_num_layers(rank, width, level) + work += sum(math.comb(rank, weight) for weight in range(1, num_layers + 1)) + return work + + def get_growth(self, index: int, level: int) -> float: + """The factor by which the work of sector ``index`` grows per unit of weight near ``level``. + + The layer ``t`` that certifies ``level`` is followed by layer ``t + 1``, which scans + ``(r - t) / (t + 1)`` times as many supports, and raises the certified weight by about + ``n / r`` in the weights of BZ words, or ``n / (r * scale)`` in those of the sector. The + factor is zero if no layer certifies ``level``. + """ + rank, width = self._shapes[index] + num_layers = self._get_num_layers(rank, width, level) + if num_layers >= rank: + return 0.0 + return ((rank - num_layers) / (num_layers + 1)) ** (rank * self._scale / width) + + def _get_num_layers(self, rank: int, width: int, level: int) -> int: + """The number of layers, from layer 1, that a search scans to certify ``level``. + + They run up to the first whose certified weight reaches ``level``, or to ``rank`` if none + does. + """ + target = level * self._scale + for layer in range(1, rank): + bound = -(-(layer + 1) * width // rank) + if bound + -bound % self._divisor >= target: + return layer + return rank + + +def _probe( + problem: BrouwerZimmermannProblem, + cutoff: int, + upper_bound: int | None, + tuning: SearchTuning, +) -> BrouwerZimmermannSearch: + """A BZ search of the sectors, advanced by whole layers within ``auto_probe_work``. + + The probe skips the costlier stages of setup: families of overlapping and partitioned + information sets, and so the certificates from linear programs, which only those take. + """ + scale = problem.weight_scale + search = BrouwerZimmermannSearch( + problem.sectors, + cutoff=cutoff * scale, + weight_divisor=problem.weight_divisor, + upper_bound=None if upper_bound is None else upper_bound * scale, + get_automorphisms=problem.get_automorphisms, + tuning=dataclasses.replace( + tuning, overlapping_minimum_work=MAX_VALUE, partition_work_ratio=MAX_VALUE + ), + ) + work = 0 + while not search.finished: + layer_work = search.get_next_layer_work() + if work + layer_work > tuning.auto_probe_work: + break + work += layer_work + search.advance() + return search + + +def _should_switch( + clusters: ConnectedClusterSearch, + costs: _BrouwerZimmermannCosts, + probe_bound: int, + state_cost: int, +) -> bool: + """Whether the connected-cluster search is predicted to cost more than a BZ search. + + The level of the next weight limit is the weight that the limit certifies, one weight divisor + above it. The search switches only while its growth exceeds BZ's at that level. It then + switches if its spent and projected work to certify that level, or to reach the probe's bound, + exceeds BZ's. To reach the bound, the projection also counts half of the states predicted for + the next limit's search at the weight limit ``probe_bound``. + """ + index, max_weight = clusters.next_limit + growth = clusters.get_growth(index) + level = max_weight + clusters.searches[index].weight_divisor + if growth is None or growth <= costs.get_growth(index, level): + return False + spent = clusters.num_states * state_cost + if spent + _project_states(clusters, level) * state_cost > costs.get_work(level): + return True + # A bound within the next limit leaves only the level to compare. search_by_cost never calls + # this function then, since its connected-cluster search, capped by the bound, has finished. + if probe_bound <= max_weight: + return False + projected = _project_states(clusters, probe_bound) + projected += clusters.predict_states(index, probe_bound) / 2 + return spent + projected * state_cost > costs.get_work(probe_bound) + + +def _project_states(clusters: ConnectedClusterSearch, level: int) -> float: + """The states that the searches are predicted to visit until each lower bound reaches ``level``. + + Each search runs its weight limits from its next one to the last below ``level``. + """ + states = 0.0 + for index, search in enumerate(clusters.searches): + first = clusters.get_next_limit(index) + for max_weight in range(first, level, search.weight_divisor): + states += clusters.predict_states(index, max_weight) + return states diff --git a/src/qldpc/codes/distance/_auto_test.py b/src/qldpc/codes/distance/_auto_test.py new file mode 100644 index 000000000..7f46fd2f6 --- /dev/null +++ b/src/qldpc/codes/distance/_auto_test.py @@ -0,0 +1,441 @@ +# SPDX-License-Identifier: Apache-2.0 + +"""Unit tests for the automatic choice of an exact distance search.""" + +from __future__ import annotations + +import dataclasses +import functools +import math +from collections.abc import Iterator +from typing import Any +from unittest import mock + +import numpy as np +import numpy.typing as npt +import pytest +from sympy.abc import x, y, z + +import qldpc +from qldpc._binary import get_kernel +from qldpc.objects import Pauli + +from . import _auto, core +from ._brouwer_zimmermann.distance import BrouwerZimmermannSearch +from ._connected_cluster.search import ConnectedClusterSearch, SupportSearch +from ._information_sets import basis_families +from ._tuning import DEFAULT_TUNING, MAX_VALUE, use_tuning +from .conftest import CostlyStages, get_costly_basis, get_golay_checks, get_sector + +# Tunings that skip enumeration and the free phase, with a probe of unlimited work, or of none. +UNLIMITED = {"auto_brute_force_max_words": 0, "auto_free_work": 0, "auto_probe_work": MAX_VALUE} +NO_PROBE = {"auto_brute_force_max_words": 0, "auto_free_work": 0, "auto_probe_work": 0} + + +def get_nested_sector( + checks: npt.NDArray[np.uint8], excluded: npt.NDArray[np.uint8] +) -> tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]: + """The nested basis and labels of the words that satisfy binary checks, outside the excluded.""" + return qldpc.codes.distance._binary.get_nested_code_basis(get_kernel(checks), excluded) + + +def get_shapes(rank: int, width: int) -> _auto.BrouwerZimmermannProblem: + """A problem of one sector of the given rank and width, which is all that costs read.""" + return _auto.BrouwerZimmermannProblem( + [(np.zeros((rank, width), np.uint8), np.zeros((rank, 1), np.uint8))] + ) + + +def test_brute_force_is_cheap_up_to_its_budget_of_packed_integers() -> None: + """Enumeration is cheap if it examines at most the budget of integers over all of its spaces.""" + tuning = dataclasses.replace(DEFAULT_TUNING, auto_brute_force_max_words=1 << 10) + assert _auto.is_cheap_to_enumerate([10], 64, tuning) + assert not _auto.is_cheap_to_enumerate([10], 65, tuning) + assert _auto.is_cheap_to_enumerate([9, 9], 64, tuning) + assert not _auto.is_cheap_to_enumerate([9, 9, 0], 64, tuning) + assert _auto.is_cheap_to_enumerate([8], 256, tuning) + + +def test_brouwer_zimmermann_costs_grow_with_the_level_and_add_over_sectors() -> None: + """The estimated work grows with the level, adds over sectors, and scales with the weights. + + A basis of rank 4 and width 8 certifies 4, 6, and 8 after its layers 1 to 3, so no layer + certifies 9, which then has no growth. The level of a Pauli string is half that of its BZ word. + """ + costs = _auto._BrouwerZimmermannCosts(get_shapes(4, 8)) + assert 0 < costs.get_work(4) < costs.get_work(6) < costs.get_work(8) + assert costs.get_growth(0, 6) > 0 and costs.get_growth(0, 9) == 0 + pair = _auto._BrouwerZimmermannCosts( + _auto.BrouwerZimmermannProblem(2 * list(get_shapes(4, 8).sectors)) + ) + assert pair.get_work(6) == 2 * costs.get_work(6) + sectors = get_shapes(5, 12).sectors + paulis = _auto._BrouwerZimmermannCosts(_auto.BrouwerZimmermannProblem(sectors, weight_scale=2)) + words = _auto._BrouwerZimmermannCosts(_auto.BrouwerZimmermannProblem(sectors)) + assert [paulis.get_work(level) for level in (3, 5)] == [words.get_work(6), words.get_work(10)] + + +@dataclasses.dataclass +class Routes: + """What searches by cost did: their probes, switches, weight limits, and witnesses. + + Attributes: + probes: Whether each probe finished, and its best weight. + switch_bounds: The upper bound of each switch to a BZ search, in the weights of BZ words. + switch_sectors: The sectors of each switch. + switch_tunings: The tuning of each switch. + limits: The weight limit of each call of ``SupportSearch.find_witness``, in order. + witnesses: The weight of each witness that a connected-cluster search found. + """ + + probes: list[tuple[bool, int]] = dataclasses.field(default_factory=list) + switch_bounds: list[int] = dataclasses.field(default_factory=list) + switch_sectors: list[Any] = dataclasses.field(default_factory=list) + switch_tunings: list[Any] = dataclasses.field(default_factory=list) + limits: list[int] = dataclasses.field(default_factory=list) + witnesses: list[int] = dataclasses.field(default_factory=list) + + +@pytest.fixture +def routes(monkeypatch: pytest.MonkeyPatch) -> Iterator[Routes]: + """Record the routes of searches by cost, through pass-through wrappers.""" + recorded = Routes() + probe = _auto._probe + get_distance = _auto.get_distance_brouwer_zimmermann + find_witness = SupportSearch.find_witness + + def recording_probe(*args: Any) -> BrouwerZimmermannSearch: + search = probe(*args) + recorded.probes.append((search.finished, search.best)) + return search + + def recording_switch(sectors: Any, **kwargs: Any) -> int: + recorded.switch_bounds.append(kwargs["upper_bound"]) + recorded.switch_sectors.append(sectors) + recorded.switch_tunings.append(kwargs["tuning"]) + return get_distance(sectors, **kwargs) + + def recording_find_witness( + search: SupportSearch, max_weight: int + ) -> npt.NDArray[np.int64] | None: + recorded.limits.append(max_weight) + witness = find_witness(search, max_weight) + if witness is not None: + recorded.witnesses.append(len(witness)) + return witness + + monkeypatch.setattr(_auto, "_probe", recording_probe) + monkeypatch.setattr(_auto, "get_distance_brouwer_zimmermann", recording_switch) + monkeypatch.setattr(SupportSearch, "find_witness", recording_find_witness) + yield recorded + + +def test_a_search_by_cost_can_end_in_its_free_phase(routes: Routes) -> None: + """A connected-cluster search that stays within its free work neither probes nor switches.""" + checks = get_golay_checks() + surface_code = qldpc.codes.SurfaceCode(3).conjugated([0, 4]) + with use_tuning(auto_brute_force_max_words=0): + assert core.get_distance_from_checks(checks, checks, method="auto") == 7 + assert core.get_distance_pauli(surface_code.matrix, method="auto") == 3 + assert routes.probes == routes.switch_bounds == [] + + +def test_the_free_phase_ends_with_the_expected_work(routes: Routes) -> None: + """The search probes once the expected work of a limit, at a state cost, exceeds the free work. + + The expected work of a limit is the work spent plus the predicted work of that limit. A free + work above the work spent before every limit of the Golay sector's search, but below the + largest expected work, makes it probe only through the prediction. + """ + checks = get_golay_checks() + clusters = ConnectedClusterSearch([(checks, checks)], cutoff=0) + spent, expected = [], [] + while not clusters.finished: + index, max_weight = clusters.next_limit + spent.append(clusters.num_states) + expected.append(clusters.num_states + clusters.predict_states(index, max_weight)) + clusters.advance() + assert 2 * max(spent) < max(expected) + + state_cost = 3 + for free_work, probes in ( + (state_cost * (max(spent) + max(expected)) / 2, [(True, 7)]), + (state_cost * 2 * max(expected), []), + ): + routes.probes.clear() + with use_tuning( + auto_brute_force_max_words=0, + auto_free_work=int(free_work), + connected_cluster_state_evaluations=state_cost, + ): + assert core.get_distance_from_checks(checks, checks, method="auto") == 7 + assert routes.probes == probes + + +def test_a_cutoff_ends_the_free_phase_at_once(routes: Routes) -> None: + """A first limit that a cutoff raises ends the free phase at once, whatever the free work. + + ``_predict_states`` predicts ``MAX_VALUE - 1`` states for that limit, so the search probes + before it. + """ + checks = get_golay_checks() + with use_tuning(auto_brute_force_max_words=0, auto_free_work=10**12, auto_probe_work=0): + assert core.get_distance_from_checks(checks, checks, method="auto", cutoff=4) == 7 + assert routes.probes == [(False, 7)] and routes.switch_bounds == [] + + +def test_a_finished_probe_gives_the_distance(routes: Routes) -> None: + """A probe that finishes within its work gives the result, also by its cutoff or upper bound. + + Pauli strings take a probe of their images, whose weights, cutoff, and upper bound are doubled. + """ + checks = get_golay_checks() + golay = qldpc.codes.CSSCode(checks.astype(int), checks.astype(int)) + pauli_checks = golay.conjugated(range(0, 23, 2)).matrix + bacon_shor_code = qldpc.codes.BaconShorCode(3) + hgp_code = qldpc.codes.HGPCode(qldpc.codes.RepetitionCode(4), qldpc.codes.RepetitionCode(5)) + hgp_checks = hgp_code.conjugated(range(0, len(hgp_code), 2)).matrix + with use_tuning(**UNLIMITED): + for search in ( + functools.partial(core.get_distance_from_checks, checks, checks, method="auto"), + functools.partial(core.get_distance_pauli, pauli_checks, method="auto"), + ): + assert search() == 7 + assert search(max_weight=10) == 7 + assert search(max_weight=5) == 6 + assert search(cutoff=7) == 7 + assert 7 <= search(cutoff=9) <= 9 + assert core.get_distance_pauli(bacon_shor_code.matrix, method="auto") == 3 + assert [finished for finished, _ in routes.probes] == [True] * 11 and routes.switch_bounds == [] + + # The lightest nontrivial row of the basis of the probe of hgp_checks weighs 14 as a BZ word, or + # 7 as a Pauli string, more than the distance 4. The setup of the probe's search finds a + # witness of the distance, which joins the probe's bound only once that search advances. So a + # probe that advances no layer has the bound 14 in the weights of BZ words, or 7 in those of + # Pauli strings. A cutoff below 7 must not return 7, and a cutoff of 7, doubled for the BZ + # words, returns 7 before the setup. + with use_tuning(**NO_PROBE): + core.get_distance_pauli(hgp_checks, method="auto") + assert routes.probes[-1] == (False, 14) + routes.probes.clear() + with use_tuning(**UNLIMITED): + assert core.get_distance_pauli(hgp_checks, method="auto", cutoff=4) == 4 + assert 4 <= core.get_distance_pauli(hgp_checks, method="auto", cutoff=6) <= 6 + assert core.get_distance_pauli(hgp_checks, method="auto", cutoff=7) == 7 + assert routes.probes == [(True, 8), (True, 8), (True, 14)] + + # A CSS search passes its cutoff on too: with the cutoff 7, a search of the Golay code probes + # before its first weight limit, and a row of the probe's basis weighs 7, so the probe finishes + # before its setup. + routes.probes.clear() + routes.limits.clear() + with use_tuning(**NO_PROBE): + assert core.get_distance_css(golay.matrix_x, golay.matrix_z, method="auto", cutoff=7) == 7 + assert routes.probes == [(True, 7)] and routes.limits == [] + + +def test_an_unfinished_probe_caps_the_search_by_its_bound(routes: Routes) -> None: + """The probe's bound ends the connected-cluster search. + + The search of the surface code finds no witness itself: it stops once its lower bounds reach the + probe's bound. + """ + surface_code = qldpc.codes.SurfaceCode(5) + with use_tuning(**NO_PROBE): + distance = core.get_distance_css( + surface_code.matrix_x, surface_code.matrix_z, method="auto" + ) + assert distance == 5 + assert routes.probes == [(False, 5)] and routes.switch_bounds == routes.witnesses == [] + + +def test_a_switch_takes_the_probe_bound_and_finds_the_distance(routes: Routes) -> None: + """A switch to BZ starts again with the full tuning and the bound of the probe. + + With a probe that does nothing, the Golay sector's search switches, as binary words and as + Pauli strings, also with an upper bound or a cutoff below the distance. The bound of a switch + is in the weights of BZ words, which are twice those of Pauli strings. + """ + checks = get_golay_checks() + golay = qldpc.codes.CSSCode(checks.astype(int), checks.astype(int)) + pauli_checks = golay.conjugated(range(0, 23, 2)).matrix + with use_tuning(**NO_PROBE) as tuning: + for search in ( + functools.partial(core.get_distance_from_checks, checks, checks, method="auto"), + functools.partial(core.get_distance_pauli, pauli_checks, method="auto"), + ): + assert search() == 7 + assert search(max_weight=5) == 6 + assert search(cutoff=3) == 7 + probe_bounds = [best for finished, best in routes.probes if not finished] + assert len(routes.switch_bounds) == 6 and routes.switch_bounds == probe_bounds + assert routes.switch_bounds[3:] == [14, 12, 14] + assert all(switch_tuning == tuning for switch_tuning in routes.switch_tunings) + + +def test_a_switch_searches_every_sector(routes: Routes) -> None: + """A search of several sectors switches to a BZ search of all of them, in the same order. + + The Golay sector has distance 7, and the sector of the costly basis, of a classical code, 6. + """ + checks, basis = get_golay_checks(), get_costly_basis() + cluster_sectors = [(checks, checks), (get_kernel(basis), np.zeros((0, 21), np.uint8))] + problem = _auto.BrouwerZimmermannProblem( + [get_nested_sector(checks, checks), (basis, np.empty((len(basis), 0), np.uint8))] + ) + tuning = dataclasses.replace( + DEFAULT_TUNING, + auto_free_work=0, + auto_probe_work=0, + connected_cluster_state_evaluations=10**6, + ) + distance = _auto.search_by_cost( + cluster_sectors, + problem, + symplectic=False, + cutoff=0, + upper_bound=None, + get_automorphisms=None, + tuning=tuning, + ) + assert distance == 6 + (switched,) = routes.switch_sectors + assert len(switched) == len(problem.sectors) == 2 + for (switched_basis, labels), (expected_basis, expected_labels) in zip( + switched, problem.sectors + ): + assert np.array_equal(switched_basis, expected_basis) + assert np.array_equal(labels, expected_labels) + + +def test_searches_by_cost_use_lifted_automorphisms(routes: Routes) -> None: + """The probe and the switched search verify and use automorphisms, lifted for Pauli strings. + + The shifts of the Golay code, and the translations of a bicycle code with Hadamards on its + second block, which is not CSS, preserve the codes. A transposition does not, and fails the + verification. The Golay code is searched by its two sectors and as Pauli strings. Only the BZ + searches ask for automorphisms here. + """ + checks = get_golay_checks() + golay = qldpc.codes.CSSCode(checks.astype(int), checks.astype(int)) + shifts = np.roll(np.arange(23), 1)[None] + bicycle_code = qldpc.codes.MBCode([7], 1 + x + x**3, 1 + x**2 + x**3 + x**4) + conjugated = bicycle_code.conjugated(range(7, 14)) + search_css = functools.partial(core.get_distance_css, golay.matrix_x, golay.matrix_z) + search_golay = functools.partial( + core.get_distance_from_checks, golay.matrix, golay.get_stabilizer_ops(), symplectic=True + ) + search_bicycle = functools.partial( + core.get_distance_from_checks, + conjugated.matrix, + conjugated.get_stabilizer_ops(), + symplectic=True, + ) + asks = {"automorphism_minimum_states": MAX_VALUE, "automorphism_minimum_layer_work": 0} + for changes, search, automorphisms, distance in ( + (NO_PROBE, search_css, shifts, 7), + (NO_PROBE, search_golay, shifts, 7), + (UNLIMITED, search_bicycle, bicycle_code._get_qudit_automorphisms(), 3), + ): + num_switches = len(routes.switch_bounds) + get_automorphisms = mock.Mock(return_value=automorphisms) + with use_tuning(**changes, **asks): + assert search(method="auto", automorphisms=get_automorphisms) == distance + get_automorphisms.assert_called_once() + if changes is NO_PROBE: + assert len(routes.switch_bounds) == num_switches + 1 + else: + assert routes.probes[-1][0] + + transposition = np.arange(len(automorphisms[0])) + transposition[[0, 1]] = [1, 0] + with use_tuning(**changes, **asks), pytest.raises(ValueError, match="not an automorphism"): + search(method="auto", automorphisms=transposition[None]) + + +def test_the_probe_skips_the_costly_stages_of_setup( + monkeypatch: pytest.MonkeyPatch, costly_stages: CostlyStages +) -> None: + """The probe takes no overlapping or partitioned family, and so no certificate from LP. + + The tuning asks for those stages. A BZ search of a sector of this MB code then takes an + overlapping family, and one of this basis a partitioned family with a certificate from LP, but + their probes do not. + """ + overlapping_families: list[basis_families.BasisFamily] = [] + get_overlapping_family = basis_families.get_overlapping_family + + def recording_get_overlapping_family(*args: Any, **kwargs: Any) -> basis_families.BasisFamily: + family = get_overlapping_family(*args, **kwargs) + overlapping_families.append(family) + return family + + monkeypatch.setattr(basis_families, "get_overlapping_family", recording_get_overlapping_family) + eager = dataclasses.replace( + DEFAULT_TUNING, + overlapping_minimum_work=0, + partition_work_ratio=0, + lp_minimum_work=0, + max_information_sets=16, + auto_probe_work=0, + ) + mb_code = qldpc.codes.MBCode({x: 4, y: 4, z: 2}, 1 + x + y + x * y * z, 1 + y + x * z + x * y) + basis = get_costly_basis() + for sector, families, uses_lp in ( + (get_sector(mb_code, Pauli.X), overlapping_families, False), + ((basis, np.empty((len(basis), 0), np.uint8)), costly_stages.partitioned_families, True), + ): + recordings = (overlapping_families, *costly_stages) + for recording in recordings: + recording.clear() + problem = _auto.BrouwerZimmermannProblem([sector]) + BrouwerZimmermannSearch(problem.sectors, cutoff=0, tuning=eager) + assert any(len(family.ranks) for family in families) + certificates = costly_stages.search_certificates + assert any(certificate.pivot_blocks for certificate in certificates) is uses_lp + + for recording in recordings: + recording.clear() + _auto._probe(problem, 0, None, eager) + assert families and not any(len(family.ranks) for family in families) + assert certificates and not any(certificate.pivot_blocks for certificate in certificates) + + +def test_a_switch_needs_faster_growth_and_more_work_to_a_level_or_to_the_probe_bound() -> None: + """The search switches while it grows faster than BZ, once it would work more than BZ. + + The work to certify a level counts the states spent and projected for the limits of every + sector below it, each weighing the state cost, and the projection to the probe's bound adds + half the limit at the bound. After its limits 1 to 4, the Golay sector's search grows faster + than a BZ search of a basis of rank and width 16, and more slowly than one of rank and width + 23. A state cost between the thresholds of level 6 and of the bound 8 switches only by the + bound. + """ + checks = get_golay_checks() + clusters = ConnectedClusterSearch([(checks, checks)], cutoff=0) + slow = _auto._BrouwerZimmermannCosts(get_shapes(16, 16)) + assert not _auto._should_switch(clusters, slow, 20, 10**6) + for _ in range(4): + clusters.advance() + assert clusters.next_limit == (0, 5) + + # Two copies of the sector project twice the states. + pair = ConnectedClusterSearch([(checks, checks), (checks, checks)], cutoff=0) + for _ in range(8): + pair.advance() + assert _auto._project_states(pair, 8) == pytest.approx(2 * _auto._project_states(clusters, 8)) + + spent = clusters.num_states + predicted = {weight: clusters.predict_states(0, weight) for weight in range(5, 9)} + level_threshold = slow.get_work(6) / (spent + predicted[5]) + bound_threshold = slow.get_work(8) / ( + spent + sum(predicted[w] for w in (5, 6, 7)) + predicted[8] / 2 + ) + assert 2 * bound_threshold < level_threshold + state_cost = round(math.sqrt(level_threshold * bound_threshold)) + assert not _auto._should_switch(clusters, slow, 5, state_cost) + assert _auto._should_switch(clusters, slow, 8, state_cost) + assert _auto._should_switch(clusters, slow, 5, math.ceil(2 * level_threshold)) + + fast = _auto._BrouwerZimmermannCosts(get_shapes(23, 23)) + assert not _auto._should_switch(clusters, fast, 20, 10**6) diff --git a/src/qldpc/codes/distance/_brouwer_zimmermann/distance.py b/src/qldpc/codes/distance/_brouwer_zimmermann/distance.py index 5f288127c..a2a6818d9 100644 --- a/src/qldpc/codes/distance/_brouwer_zimmermann/distance.py +++ b/src/qldpc/codes/distance/_brouwer_zimmermann/distance.py @@ -175,6 +175,16 @@ def finished(self) -> bool: """Whether the best upper bound is at most the cutoff, or every sector's search finished.""" return self.best <= self._cutoff or all(search.finished for search in self._searches) + def get_next_layer_work(self) -> int: + """The candidate/set evaluations of the layer that ``advance`` scans next, or zero if none. + + This plans the layer as ``advance`` does, but leaves out a switch to an automorphism family, + after which the layer scans one set. + """ + if self.finished: + return 0 + return self._searches[self._get_next_index()].get_layer_work() + def advance(self) -> None: """Advance the next unfinished sector's search by one layer, round-robin, unless finished. @@ -188,13 +198,19 @@ def advance(self) -> None: """ if self.finished: return - while self._searches[self._next].finished: - self._next = (self._next + 1) % len(self._searches) + self._next = self._get_next_index() search = self._searches[self._next] self._next = (self._next + 1) % len(self._searches) self.best = min(self.best, search.advance()) self._share_best() + def _get_next_index(self) -> int: + """The index of the next unfinished search in round-robin order, while one is unfinished.""" + index = self._next + while self._searches[index].finished: + index = (index + 1) % len(self._searches) + return index + def _share_best(self) -> None: """Give each sector's search the best upper bound; one that finishes frees its tables.""" for search in self._searches: @@ -253,6 +269,11 @@ def finished(self) -> bool: or self.next_weight > self.dimension ) + def get_layer_work(self) -> int: + """The candidate/set evaluations of the sets that ``advance`` plans to scan next.""" + num_scanned, _ = self._plan_layer() + return num_scanned * math.comb(self.dimension, self.next_weight) + def update_upper_bound(self, upper_bound: int) -> None: """Import a witness found by another search, and free the tables of a finished search.""" self.best = min(self.best, upper_bound) @@ -271,19 +292,14 @@ def advance(self) -> int: # Plan to scan the sets up to the first whose step certifies the best weight. weight = self.next_weight - instance = self.instance - certificate = instance.certificate + certificate = self.instance.certificate num_sets = len(certificate.ranks) - if self.uses_automorphism_family: - num_scanned, num_completed = 1, num_sets - else: - num_completed = certificate.get_num_sets_to_certify(weight, self.best) - num_scanned = num_completed + num_scanned, num_completed = self._plan_layer() # Before a costly layer, consider switching to an automorphism family (Section 3.7). The # layer work saturates at ``WORK_CAP``, as work estimates do, so that a threshold above it # forbids the switch. - layer_work = min(num_scanned * math.comb(self.dimension, weight), certificates.WORK_CAP) + layer_work = min(self.get_layer_work(), certificates.WORK_CAP) if ( self._considers_automorphism_family and layer_work >= self.tuning.automorphism_minimum_layer_work @@ -309,6 +325,20 @@ def advance(self) -> int: self.next_weight += 1 return self.best + def _plan_layer(self) -> tuple[int, int]: + """The numbers of sets that the next layer scans and completes, as ``advance`` plans. + + The layer scans the sets up to the first whose step certifies the best weight, or the one + set of an automorphism family, which completes the layer for every image. + """ + num_sets = len(self.instance.certificate.ranks) + if self.uses_automorphism_family: + return 1, num_sets + num_completed = self.instance.certificate.get_num_sets_to_certify( + self.next_weight, self.best + ) + return num_completed, num_completed + @property def _scan_cutoff(self) -> int: """The cutoff that the compiled scans take, which is ``cutoff`` clamped below at -1. diff --git a/src/qldpc/codes/distance/_brouwer_zimmermann/distance_test.py b/src/qldpc/codes/distance/_brouwer_zimmermann/distance_test.py index d087028d4..fd2715310 100644 --- a/src/qldpc/codes/distance/_brouwer_zimmermann/distance_test.py +++ b/src/qldpc/codes/distance/_brouwer_zimmermann/distance_test.py @@ -29,6 +29,7 @@ from ..conftest import ( MISSED_SET_CODE, CostlyStages, + get_costly_basis, get_random_css_code, get_random_full_rank_matrix, get_sector, @@ -180,7 +181,7 @@ def test_brouwer_zimmermann_setup_exits( if overlap: tune(overlapping_minimum_work=0) distance = qldpc.codes.distance.get_distance_from_generators( - logicals, stabilizers, cutoff=cutoff + logicals, stabilizers, cutoff=cutoff, method="brouwer_zimmermann" ) assert expected <= distance <= max(cutoff, expected) @@ -200,7 +201,9 @@ def test_searches_never_enumerate_every_word() -> None: with mock.patch.object( _brute_force, "_get_distance_brute_force", wraps=_brute_force._get_distance_brute_force ) as enumerate_words: - distance = qldpc.codes.distance.get_distance_from_generators(logicals, stabilizers) + distance = qldpc.codes.distance.get_distance_from_generators( + logicals, stabilizers, method="brouwer_zimmermann" + ) assert distance == expected enumerate_words.assert_not_called() @@ -371,20 +374,25 @@ def test_brouwer_zimmermann_subsystem_code() -> None: See https://arxiv.org/abs/quant-ph/0506023. """ code = qldpc.codes.QuditCode(qldpc.codes.BaconShorCode(3).matrix, is_subsystem_code=True) - assert code.get_distance_exact() == 3 + assert code.get_distance_exact(method="brouwer_zimmermann") == 3 @pytest.mark.parametrize("length", [64, 65, 257]) def test_brouwer_zimmermann_packed_widths(length: int) -> None: """Information-set weights are exact across uint64 words and for weights above 255.""" generators = np.ones((1, length), dtype=np.uint8) - assert qldpc.codes.distance.get_distance_from_generators(generators, cutoff=0) == length + assert ( + qldpc.codes.distance.get_distance_from_generators( + generators, cutoff=0, method="brouwer_zimmermann" + ) + == length + ) def test_dependent_logical_rows_give_the_distance_of_their_span() -> None: """Repeated logical rows are reduced to a basis of the quotient before any search.""" distance = qldpc.codes.distance.get_distance_from_generators( - [[1, 1, 1], [1, 1, 1]], [], cutoff=0 + [[1, 1, 1], [1, 1, 1]], [], cutoff=0, method="brouwer_zimmermann" ) assert distance == 3 @@ -397,9 +405,9 @@ def test_brouwer_zimmermann_reduces_enumeration_work( generators = np.asarray(qldpc.codes.HammingCode(5).generator, dtype=np.uint8) if extended: generators = np.column_stack([generators, np.bitwise_xor.reduce(generators, axis=1)]) - assert qldpc.codes.distance.get_distance_from_generators(generators, cutoff=0) == ( - 4 if extended else 3 - ) + assert qldpc.codes.distance.get_distance_from_generators( + generators, cutoff=0, method="brouwer_zimmermann" + ) == (4 if extended else 3) assert 0 < 1000 * get_candidate_count() < 2 ** len(generators) @@ -411,13 +419,14 @@ def test_the_lightest_eligible_row_excludes_stabilizers() -> None: def test_brouwer_zimmermann_overlap_mb(get_candidate_count: Callable[[], int]) -> None: - """The default X-sector search of this 64-qubit MB code stays within a broad work budget.""" + """The BZ X-sector search of this 64-qubit MB code stays within a broad work budget.""" code = qldpc.codes.MBCode( {x: 4, y: 4, z: 2}, 1 + x + y + x * y * z, 1 + x * y + x**2 * y**3 * z + x**3 * y**2, ) - assert code.forget_distance().get_distance_exact("X", cutoff=0) == 8 + distance = code.forget_distance().get_distance_exact("X", cutoff=0, method="brouwer_zimmermann") + assert distance == 8 assert 0 < get_candidate_count() < 1_000_000 @@ -455,7 +464,9 @@ def select(*args: Any, **kwargs: Any) -> basis_families.BasisFamily: basis_families, "get_overlapping_family", side_effect=select ) as select_mock: assert ( - qldpc.codes.distance.get_distance_from_generators(logicals, stabilizers, cutoff=0) + qldpc.codes.distance.get_distance_from_generators( + logicals, stabilizers, cutoff=0, method="brouwer_zimmermann" + ) == expected == 5 ) @@ -508,7 +519,10 @@ def test_a_search_scans_every_set_that_its_certificate_counts() -> None: qldpc.codes.distance.get_distance_from_generators(basis, cutoff=0, method="brute_force") == 5 ) - assert qldpc.codes.distance.get_distance_from_generators(basis, cutoff=0) == 5 + distance = qldpc.codes.distance.get_distance_from_generators( + basis, cutoff=0, method="brouwer_zimmermann" + ) + assert distance == 5 def test_a_search_through_the_costly_stages_finds_the_distance( @@ -521,12 +535,15 @@ def test_a_search_through_the_costly_stages_finds_the_distance( partition_work_ratio=0, max_information_sets=16, ) - basis = _get_costly_basis() + basis = get_costly_basis() assert ( qldpc.codes.distance.get_distance_from_generators(basis, cutoff=0, method="brute_force") == 6 ) - assert qldpc.codes.distance.get_distance_from_generators(basis, cutoff=0) == 6 + distance = qldpc.codes.distance.get_distance_from_generators( + basis, cutoff=0, method="brouwer_zimmermann" + ) + assert distance == 6 assert any( len(family.ranks) and family.max_coverage > 1 for family in costly_stages.partitioned_families @@ -548,7 +565,9 @@ def test_a_coverage_limit_of_one_partitions_into_disjoint_sets( partition_max_coverage=1, partition_work_ratio=0, ) - distance = qldpc.codes.distance.get_distance_from_generators(MISSED_SET_CODE, cutoff=0) + distance = qldpc.codes.distance.get_distance_from_generators( + MISSED_SET_CODE, cutoff=0, method="brouwer_zimmermann" + ) assert ( distance == qldpc.codes.distance.get_distance_from_generators(MISSED_SET_CODE, method="brute_force") @@ -775,6 +794,64 @@ def recording_prepare(*args: Any, **kwargs: Any) -> Any: assert tunings == [tuning, tuning] +def test_the_next_layer_work_is_that_of_the_layer_that_advance_scans( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Searches report the work of the layer that ``advance`` scans next, and none at the end. + + The work counts the supports of the sets that the next unfinished search plans to scan, an + upper bound on the supports that it scans. A layer of an automorphism family scans its one + set. The surface code scans several sets per layer, the random code skips the search of a + finished sector, and the gross code switches to automorphism families. + """ + asked: list[bz._SectorSearch] = [] + scanned: list[tuple[bz._SectorSearch, int]] = [] + get_layer_work = bz._SectorSearch.get_layer_work + scan_set = bz._SectorSearch._scan_set + + def recording_get_layer_work(search: bz._SectorSearch) -> int: + asked.append(search) + return get_layer_work(search) + + def recording_scan_set(search: bz._SectorSearch, index: int, weight: int) -> bool: + scanned.append((search, math.comb(search.dimension, weight))) + return scan_set(search, index, weight) + + monkeypatch.setattr(bz._SectorSearch, "get_layer_work", recording_get_layer_work) + monkeypatch.setattr(bz._SectorSearch, "_scan_set", recording_scan_set) + layers: list[tuple[int, int, int, bool]] = [] + for code, minimum_layer_work, distance in ( + (qldpc.codes.SurfaceCode(9), MAX_VALUE, 9), + (get_random_css_code(np.random.default_rng(4), 60, 20, 25), MAX_VALUE, 4), + (qldpc.codes.GrossCode(), 0, None), + ): + sectors = [get_sector(code, pauli) for pauli in PAULIS_XZ] + with use_tuning(automorphism_minimum_layer_work=minimum_layer_work): + search = bz.BrouwerZimmermannSearch( + sectors, cutoff=0, get_automorphisms=code._get_qudit_automorphisms + ) + while not search.finished: + asked.clear() + scanned.clear() + work = search.get_next_layer_work() + (sector_search,) = asked + uses_family = sector_search.uses_automorphism_family + search.advance() + assert scanned and all(scanner is sector_search for scanner, _ in scanned) + layers.append( + (work, sum(supports for _, supports in scanned), len(scanned), uses_family) + ) + # Stop the gross code at its first family layer, which keeps the test fast. + if uses_family: + break + if distance is not None: + assert search.best == distance and search.get_next_layer_work() == 0 + assert all(supports <= work for work, supports, _, _ in layers) + assert any(supports == work and num_sets > 1 for work, supports, num_sets, _ in layers) + family_layers = [(work, supports) for work, supports, _, uses_family in layers if uses_family] + assert family_layers and all(work == supports for work, supports in family_layers) + + def test_automorphism_family_searches_scan_their_new_set_from_layer_two( tune: Tune, completion_tables: CompletionRecord ) -> None: @@ -1007,7 +1084,10 @@ def _spy_switch(search: bz._SectorSearch) -> None: ): assert ( qldpc.codes.distance.get_distance_from_generators( - _get_costly_basis(), cutoff=0, automorphisms=np.arange(21)[None] + get_costly_basis(), + cutoff=0, + automorphisms=np.arange(21)[None], + method="brouwer_zimmermann", ) == 6 ) @@ -1064,21 +1144,6 @@ def test_searches_reject_wrong_automorphisms() -> None: ) -def _get_costly_basis() -> npt.NDArray[np.uint8]: - """A basis of a [21, 8, 6] code whose search reaches matroid partitioning at lowered gates.""" - rows = [ - "100000010110010010011", - "010000010100101111100", - "001000000111011111100", - "000100010101000101010", - "000010010110000011000", - "000001010011000111011", - "000000110001110101011", - "000000001110001010110", - ] - return np.array([[int(bit) for bit in row] for row in rows], dtype=np.uint8) - - def _no_labels(basis: npt.NDArray[np.uint8]) -> npt.NDArray[np.uint8]: """Quotient labels with no columns, which exclude no subcode.""" return np.empty((len(basis), 0), dtype=np.uint8) diff --git a/src/qldpc/codes/distance/_connected_cluster/search.py b/src/qldpc/codes/distance/_connected_cluster/search.py index 854648155..d977262d1 100644 --- a/src/qldpc/codes/distance/_connected_cluster/search.py +++ b/src/qldpc/codes/distance/_connected_cluster/search.py @@ -12,6 +12,7 @@ from __future__ import annotations +import itertools import math from collections.abc import Callable, Sequence @@ -99,7 +100,8 @@ class ConnectedClusterSearch: The arguments are those of ``get_distance_connected_cluster``, which advances the search until it finishes. Each advance searches the sector with the lowest lower bound on its witnesses - through its next weight limit (Section 3.6 of ``connected_cluster.md``). + through its next weight limit (Section 3.6 of ``connected_cluster.md``). The searches of the + sectors that have witnesses are numbered in the order of those sectors. Raises: ValueError: If there are neither sectors nor an upper bound, or if a sector is not valid @@ -149,6 +151,11 @@ def __init__( ) self._num_limits = 0 + # The weight limit, states, and number of roots of each limit without a witness, per search. + self._limits_without_witness: list[list[tuple[int, int, int]]] = [ + [] for _ in self._searches + ] + @property def finished(self) -> bool: """Whether no sector has witnesses, or the best weight is at most the cutoff or each bound. @@ -168,6 +175,72 @@ def result(self) -> int: bounds = (self._best_weight, *self._no_distance_bounds) return min(bound for bound in bounds if bound is not None) + @property + def searches(self) -> tuple[SupportSearch, ...]: + """The search of each sector that has witnesses.""" + return tuple(self._searches) + + @property + def num_states(self) -> int: + """The states that the searches have visited, over all their weight limits.""" + return sum(search.num_states for search in self._searches) + + @property + def next_limit(self) -> tuple[int, int]: + """The index of the search that the next advance runs, and that advance's weight limit.""" + index = self._lower_bounds.index(min(self._lower_bounds)) + return index, self.get_next_limit(index) + + def get_next_limit(self, index: int) -> int: + """The next weight limit of a search: its lower bound, or the cutoff if that is higher. + + A cutoff raises the limit to the largest multiple of the search's divisor within the cutoff. + """ + divisor = self._searches[index].weight_divisor + return max(self._lower_bounds[index], self._cutoff - self._cutoff % divisor) + + def get_growth(self, index: int) -> float | None: + """The factor by which the states of a search grow per unit of its weight limit. + + A ratio compares the states of two successive limits without a witness, both nonzero and + from the same number of roots, per unit of weight between them. The factor is the geometric + mean of the last two ratios, which evens out the alternation between limits of odd and even + weight. With only one ratio, the factor is that ratio, and with none it is None. + """ + limits = self._limits_without_witness[index] + ratios = [ + (states / previous_states) ** (1 / (weight - previous_weight)) + for (previous_weight, previous_states, previous_roots), (weight, states, roots) in ( + itertools.pairwise(limits) + ) + if previous_states and states and roots == previous_roots + ] + if not ratios: + return None + return math.sqrt(ratios[-1] * ratios[-2]) if len(ratios) > 1 else ratios[-1] + + def predict_states(self, index: int, max_weight: int) -> float: + """The states that a search is expected to visit at one later weight limit. + + The states of its last limit without a witness grow by ``get_growth`` for each unit of + weight, and a prediction too large for a float is infinite. Without a growth factor, the + prediction for any later limit is the search's own prediction for its next limit (Section + 3.5 of ``connected_cluster.md``). + """ + growth = self.get_growth(index) + if growth is None: + return self._searches[index]._predict_states(max_weight) + last_weight, last_states, _ = self._limits_without_witness[index][-1] + try: + return last_states * growth ** (max_weight - last_weight) + except OverflowError: + return math.inf + + def update_upper_bound(self, upper_bound: int) -> None: + """Lower the best weight to the weight of a witness, or to a cap, found elsewhere.""" + if self._best_weight is None or upper_bound < self._best_weight: + self._best_weight = upper_bound + def advance(self) -> None: """Search the sector with the lowest lower bound through its next limit, unless finished. @@ -187,17 +260,20 @@ def advance(self) -> None: ) # Search the sector with the lowest lower bound through its next limit, which the cutoff - # can raise. - index = self._lower_bounds.index(min(self._lower_bounds)) + # can raise (see ``get_next_limit``). + index, max_weight = self.next_limit search = self._searches[index] divisor = search.weight_divisor - max_weight = max(self._lower_bounds[index], self._cutoff - self._cutoff % divisor) + num_states = search.num_states witness = search.find_witness(max_weight) self._num_limits += 1 - # A limit without a witness raises the lower bound past it, and a witness may lower the best - # weight. + # A limit without a witness raises the lower bound past it, and records its states for + # get_growth. A witness may lower the best weight. if witness is None: + self._limits_without_witness[index].append( + (max_weight, search.num_states - num_states, len(search._roots)) + ) self._lower_bounds[index] = (max_weight // divisor + 1) * divisor elif self._best_weight is None or len(witness) < self._best_weight: self._best_weight = len(witness) diff --git a/src/qldpc/codes/distance/_connected_cluster/search_test.py b/src/qldpc/codes/distance/_connected_cluster/search_test.py index 89f53b877..971a70a12 100644 --- a/src/qldpc/codes/distance/_connected_cluster/search_test.py +++ b/src/qldpc/codes/distance/_connected_cluster/search_test.py @@ -7,6 +7,7 @@ import dataclasses import inspect import itertools +import math import unittest.mock from collections.abc import Iterator from typing import Any @@ -21,7 +22,7 @@ from qldpc.objects import PAULIS_XZ, Pauli from .. import _tuning -from ..conftest import get_span +from ..conftest import get_golay_checks, get_span from ..core import get_distance_css, get_distance_pauli from . import search as cs @@ -55,13 +56,8 @@ def get_sectors(code: codes.CSSCode) -> list[Problem]: def get_golay_problem() -> Problem: - """A sector of the CSS code of the cyclic [23, 12, 7] Golay code, which contains its dual. - - Its checks and excluded rows are both a basis of the dual, so cyclic shifts are automorphisms. - """ - generator = np.zeros(23, dtype=np.uint8) - generator[[0, 1, 5, 6, 7, 9, 11]] = 1 - checks = get_kernel(np.array([np.roll(generator, shift) for shift in range(12)])) + """A sector of the CSS code of the cyclic [23, 12, 7] Golay code (see ``get_golay_checks``).""" + checks = get_golay_checks() return checks, checks @@ -767,6 +763,90 @@ def record_limit(search: cs.SupportSearch, max_weight: int) -> npt.NDArray[np.in assert cs.get_distance_connected_cluster([components], cutoff=3) == 3 +def test_stepped_searches_report_their_progress_and_take_caps() -> None: + """Stepped searches report their states and the growth of their states, and stop at a cap.""" + sector = get_golay_problem() + assert get_brute_force_distance(sector) == 7 + search = cs.ConnectedClusterSearch([sector], cutoff=0) + (sector_search,) = search.searches + assert search.next_limit == (0, 1) + assert search.num_states == 0 and search.get_growth(0) is None + assert search.predict_states(0, 1) == sector_search._predict_states(1) + + # Each limit without a witness raises the lower bound past it, and records its states. The + # growth is the ratio of states per unit of weight while there is only one, and then the + # geometric mean of the last two ratios; the first limit visits no states, so it gives no ratio. + states = [] + for max_weight in range(1, 5): + assert search.next_limit == (0, max_weight) + num_states = search.num_states + search.advance() + states.append(search.num_states - num_states) + assert search.next_limit == (0, max_weight + 1) and not search.finished + if max_weight == 3: + assert search.get_growth(0) == pytest.approx(states[2] / states[1]) + assert states[0] == 0 and all(states[1:]) + growth = search.get_growth(0) + assert growth is not None and growth == pytest.approx(np.sqrt(states[3] / states[1])) + assert search.predict_states(0, 7) == pytest.approx(states[3] * growth**3) + assert search.predict_states(0, 10**4) == math.inf + + # A weaker cap changes nothing, and a cap at the next lower bound ends the search there. + search.update_upper_bound(30) + search.update_upper_bound(6) + search.update_upper_bound(8) + assert not search.finished + search.advance() + assert search.finished and search.result == 6 + + # A cap within the cutoff ends the search at once. + search = cs.ConnectedClusterSearch([sector], cutoff=5) + search.update_upper_bound(5) + assert search.finished and search.result == 5 and search.num_states == 0 + + +def test_the_growth_of_states_skips_a_change_of_roots() -> None: + """The growth compares only limits from the same number of roots, which automorphisms reduce. + + The search asks for the cyclic shifts before limit 4, which then searches from fewer roots. + """ + shifts = np.roll(np.arange(23), 1)[None] + tuning = dataclasses.replace(_tuning.DEFAULT_TUNING, automorphism_minimum_states=100) + search = cs.ConnectedClusterSearch( + [get_golay_problem()], cutoff=0, get_automorphisms=lambda: shifts, tuning=tuning + ) + states, growths = [], [] + for _ in range(5): + num_states = search.num_states + search.advance() + states.append(search.num_states - num_states) + growths.append(search.get_growth(0)) + assert growths[2] == growths[3] == pytest.approx(states[2] / states[1]) + assert growths[4] == pytest.approx(np.sqrt(states[2] / states[1] * states[4] / states[3])) + + +def test_stepped_searches_add_the_states_of_their_sectors_and_grow_per_unit_of_weight() -> None: + """The states add over sectors, and the growth is per unit of weight, not per weight limit. + + Both sectors of the toric code of size 6 have only words of even weight, so the limits of each + rise in steps of two, and the growth is the square root of the ratio of their states. + """ + search = cs.ConnectedClusterSearch(get_sectors(codes.ToricCode(6)), cutoff=0) + assert [sector_search.weight_divisor for sector_search in search.searches] == [2, 2] + records = [] + for _ in range(4): + index, max_weight = search.next_limit + num_states = search.searches[index].num_states + search.advance() + records.append((index, max_weight, search.searches[index].num_states - num_states)) + assert [record[:2] for record in records] == [(0, 2), (1, 2), (0, 4), (1, 4)] + assert all(states for *_, states in records) and not search.finished + assert search.num_states == sum(states for *_, states in records) + growth = (records[2][2] / records[0][2]) ** (1 / 2) + assert search.get_growth(0) == pytest.approx(growth) + assert search.predict_states(0, 8) == pytest.approx(records[2][2] * growth**4) + + def test_a_problem_without_logical_operators_has_no_distance() -> None: """Without witnesses, a sector counts as one more than its number of variables.""" trivial = (np.array([[1, 1]], dtype=np.uint8), np.array([[1, 1]], dtype=np.uint8)) diff --git a/src/qldpc/codes/distance/_layering_test.py b/src/qldpc/codes/distance/_layering_test.py index 454b78448..3f1bf80cf 100644 --- a/src/qldpc/codes/distance/_layering_test.py +++ b/src/qldpc/codes/distance/_layering_test.py @@ -13,6 +13,7 @@ ("_tuning",), ("_information_sets", "_cosets", "min_weight_bounds"), ("_brouwer_zimmermann", "_connected_cluster", "_counting", "_brute_force"), + ("_auto",), ("core",), ("logical_counts", "coset_leaders"), ) diff --git a/src/qldpc/codes/distance/_tuning.py b/src/qldpc/codes/distance/_tuning.py index d83bd648b..68451e299 100644 --- a/src/qldpc/codes/distance/_tuning.py +++ b/src/qldpc/codes/distance/_tuning.py @@ -6,7 +6,8 @@ logical operators, or a ``CosetLeaderSearch`` chooses its route and limits its batches and memory. The page :doc:`/exact_search_tuning` renders this docstring after tables that lead from symptoms and routes to these values. Section 4.4 of ``brouwer_zimmermann.md`` and Sections 3.4, 3.5, 3.8, and -3.9 of ``connected_cluster.md`` explain how searches use the fields. +3.9 of ``connected_cluster.md`` explain how searches use the fields, and the section "Choosing a +method" of ``README.md`` how ``method="auto"`` uses its fields. This module is private: its names, fields, defaults, and constants can change in any release without warning, so code that uses it should pin the exact version of qLDPC. It is meant for contributors, @@ -37,7 +38,7 @@ code = codes.BBCode({x: 6, y: 6}, x**3 + y + y**2, y**3 + x + x**2) with _tuning.use_tuning(max_information_sets=8, lp_minimum_work=0): - distance = code.get_distance_exact() + distance = code.get_distance_exact(method="brouwer_zimmermann") A search reads the active tuning once, when it starts. The active tuning is a context variable (see the ``contextvars`` module). So copies of the context, as in asyncio tasks and @@ -84,8 +85,9 @@ - A limit for experiments restricts what searches may build, and its default, ``MAX_VALUE``, restricts nothing. -The sizes of caches, ``coset_route_minimum_work``, and ``macwilliams_shared_walk_rank`` were chosen -by hand, and the comments of the two fields say why. +The sizes of caches, ``auto_brute_force_max_words``, ``auto_free_work``, +``connected_cluster_state_evaluations``, ``coset_route_minimum_work``, and +``macwilliams_shared_walk_rank`` were chosen by hand, and the comments of those fields say why. Change a default only when its heuristic changes, or when a check shows that its assumption, such as the cost of a step, is off by more than a factor of about 3; only a maintainer changes a budget or a @@ -93,7 +95,7 @@ scripts of ``experiments/distance_tuning`` check the tuning and set no value: ``constants.py`` checks the orders of magnitude that the heuristics assume, ``trace.py`` shows which decisions a search makes, and ``sweep.py`` catches gross regressions or bugs. The README of that directory says -how to run them. +how to run them. They do not check the fields of the automatic choice of a method. """ from __future__ import annotations @@ -222,6 +224,40 @@ class SearchTuning: #: code, several million evaluations (Sections 3.7 and 4.4 of ``brouwer_zimmermann.md``). automorphism_minimum_layer_work: int = 5_000_000 + # Fields of the automatic choice of a method. + + #: ``method="auto"`` enumerates every word, as ``"brute_force"`` does, if the enumeration + #: examines at most this many packed 64-bit integers: ``2**dim * ceil(width / 64)`` for words of + #: ``width`` bits in a space of dimension ``dim``, summed over the spaces of a search, such as + #: the two sectors of a CSS code. Raising it enumerates larger spaces. + #: Defaults to ``2**30``, chosen by hand: an enumeration of about a second, whose cost, unlike + #: that of the other methods, is known before it runs. + auto_brute_force_max_words: int = 2**30 + + #: The work, in candidate/set evaluations, below which a search by cost of ``method="auto"`` + #: runs its connected-cluster search without probing or switching: it runs on while its spent + #: work plus the predicted work of its next weight limit stays within this. Beyond it, the + #: search probes with a Brouwer-Zimmermann search, and then considers switching to one. + #: Connected-cluster work counts each state as ``connected_cluster_state_evaluations``. + #: Defaults to ``10**8``, chosen by hand: a fraction of a second, below which fixed costs that + #: the estimates leave out, such as setup, dominate the costs that a switch would compare + #: (section "Choosing a method" of ``README.md``). + auto_free_work: int = 100_000_000 + + #: The most work, in candidate/set evaluations, that the Brouwer-Zimmermann probe of + #: ``method="auto"`` may do. The weight of the lightest word that it finds caps the + #: connected-cluster search. The probe skips the costlier stages of setup, and stops before a + #: layer that would take it over this work. + #: Defaults to ``10**8``, a budget for the cost of choosing: a fraction of a second (section + #: "Choosing a method" of ``README.md``). + auto_probe_work: int = 100_000_000 + + #: The cost of a state of a connected-cluster search in candidate/set evaluations, which lets + #: ``method="auto"`` compare the work of the two searches (section "Choosing a method" of + #: ``README.md``). Raising it switches to a Brouwer-Zimmermann search sooner. Defaults to 500, + #: chosen by hand: the cost of a state, hundreds of evaluations. + connected_cluster_state_evaluations: int = 500 + # Fields of connected-cluster searches. #: A connected-cluster search asks its caller for automorphisms before the first weight limit at diff --git a/src/qldpc/codes/distance/_tuning_test.py b/src/qldpc/codes/distance/_tuning_test.py index 6a2c35b73..fe58da97c 100644 --- a/src/qldpc/codes/distance/_tuning_test.py +++ b/src/qldpc/codes/distance/_tuning_test.py @@ -23,7 +23,7 @@ from qldpc import codes from qldpc._binary import get_kernel -from qldpc.objects import Pauli +from qldpc.objects import PAULIS_XZ, Pauli from . import _tuning, core from . import logical_counts as lc @@ -176,7 +176,9 @@ def test_tunings_change_routes_but_not_results(costly_stages: CostlyStages) -> N with _tuning.use_tuning(**changes): # New codes and counters, since codes and counters remember their distances. fresh = codes.CSSCode(code.matrix_x, code.matrix_z) - distances = (fresh.get_distance_exact(Pauli.X), fresh.get_distance_exact(Pauli.Z)) + distances = tuple( + fresh.get_distance_exact(pauli, method="brouwer_zimmermann") for pauli in PAULIS_XZ + ) counts = lc.CSSLogicalCounter(code.matrix_x, code.matrix_z).count_logical_ops( count_pauli=True ) @@ -222,7 +224,8 @@ def test_limits_on_partitioned_coverage_change_routes_but_not_results( partition_max_coverage=limit, ): # A new code, since codes remember their distances. - distance = codes.CSSCode(css_code.matrix_x, css_code.matrix_z).get_distance_exact() + fresh = codes.CSSCode(css_code.matrix_x, css_code.matrix_z) + distance = fresh.get_distance_exact(method="brouwer_zimmermann") counts = lc.CSSLogicalCounter(css_code.matrix_x, css_code.matrix_z).count_logical_ops(5) results.append((distance, counts)) max_coverages.append( @@ -312,9 +315,9 @@ def test_every_field_reaches_the_code_that_reads_it(monkeypatch: pytest.MonkeyPa Tuning keeps results, so recording the arguments of each reader is what shows that a value arrives. The tuning takes every route, with values that differ from the defaults. Fields that this test does not follow are read in one of two ways. Some are read within one function from - the tuning that it receives, and their tests in ``_information_sets/completion_bounds_test.py`` - and ``_information_sets/basis_families_test.py`` change them there. Others are read once by - ``CosetLeaderSearch``, which passes their values down to its coset searches: + the tuning that it receives, and their tests in ``_information_sets/completion_bounds_test.py``, + ``_information_sets/basis_families_test.py``, and ``_auto_test.py`` change them there. Others + are read once by ``CosetLeaderSearch``, which passes their values down to its coset searches: ``max_css_single_pairs`` and ``max_sparsifying_rounds``. Tests in ``coset_leaders_test.py``, ``_brouwer_zimmermann/coset_test.py``, and ``_connected_cluster/coset_test.py`` change or record these, and the fields that the oracle passes down to the routes of its BZ searches. The @@ -352,6 +355,8 @@ def recording(*args: Any, **kwargs: Any) -> Any: "restrict_viable_symbols", ) record(core, "get_sparser_rows", "max_rounds", "rounds_without_improvement") + record(core, "is_cheap_to_enumerate", "tuning", prefix="auto ") + record(core, "search_by_cost", "tuning", prefix="auto ") original_prepare = bz._prepare_sector_search def prepare(*args: Any, **kwargs: Any) -> Any: @@ -379,8 +384,12 @@ def prepare(*args: Any, **kwargs: Any) -> Any: restrict_viable_symbols=False, max_distance_sparsifying_rounds=4, sparsifying_rounds_without_improvement=2, + auto_brute_force_max_words=0, ) as tuning: - codes.CSSCode(code.matrix_x, code.matrix_z).get_distance_exact(Pauli.X) + codes.CSSCode(code.matrix_x, code.matrix_z).get_distance_exact( + Pauli.X, method="brouwer_zimmermann" + ) + codes.CSSCode(code.matrix_x, code.matrix_z).get_distance_exact(Pauli.Z, method="auto") codes.CSSCode(code.matrix_x, code.matrix_z).get_distance_exact(method="connected_cluster") counter = lc.CSSLogicalCounter(code.matrix_x, code.matrix_z) counter.count_logical_ops() @@ -403,6 +412,7 @@ def prepare(*args: Any, **kwargs: Any) -> Any: "restrict_viable_symbols": {False}, "max_rounds": {4}, "rounds_without_improvement": {2}, + "auto tuning": {tuning}, } assert {key: set(values) for key, values in seen.items()} == expected diff --git a/src/qldpc/codes/distance/brouwer_zimmermann.md b/src/qldpc/codes/distance/brouwer_zimmermann.md index bc8fbed1b..d2c56f212 100644 --- a/src/qldpc/codes/distance/brouwer_zimmermann.md +++ b/src/qldpc/codes/distance/brouwer_zimmermann.md @@ -750,7 +750,7 @@ The Retuning section of [_tuning.py](_tuning.py) defines the other classes of fi Binary input (`symplectic=False`) is a classical problem, or a pure-X or pure-Z problem with one bit per qubit, and symplectic input (`symplectic=True`) has general Pauli rows. For symplectic input, BZ maps rows by `T` ([Section 2.7](#27-pauli-weight-as-hamming-weight)), doubles `cutoff` and the bound of a weight limit, passes `weight_divisor=2`, and halves the result, while `method="brute_force"` uses symplectic weights and the original `cutoff`. - **Independence.** - `_search_generators` in `core.py` makes the generators of a caller independent, with `get_independent_rows`, or with `get_nested_code_basis` when the caller gives `S`, for every BZ and brute-force search but the BZ search of `_search_css`, which calls `get_nested_code_basis` itself. + `_get_nested_basis` in `core.py` makes the generators of a caller independent, with `get_independent_rows`, or with `get_nested_code_basis` when the caller gives `S`, for every BZ and brute-force search of `_search_generators` and `_search_checks`, including those of `search_by_cost`; `_search_css` calls `get_nested_code_basis` itself for its BZ searches and those of `search_by_cost`. - **Resumable layers.** A compiled scan of a distance search returns after at most `batch_size` supports, or supports and lookups together with completion tables, so that it stays responsive while it skips most supports, and successive batches reuse its `support` and `prefixes` arrays. Once the search bases that `advance` scans have completed a layer, it may raise `lower_bound` ([Section 2.5](#25-the-stopping-certificate)), but it increments `next_weight` only once every search basis has. diff --git a/src/qldpc/codes/distance/conftest.py b/src/qldpc/codes/distance/conftest.py index c2c23d8e2..c4539352a 100644 --- a/src/qldpc/codes/distance/conftest.py +++ b/src/qldpc/codes/distance/conftest.py @@ -178,6 +178,33 @@ def get_random_css_code( return codes.CSSCode(checks_x.astype(int), checks_z.astype(int)) +def get_costly_basis() -> npt.NDArray[np.uint8]: + """A basis of a [21, 8, 6] code whose search reaches matroid partitioning at low thresholds.""" + return get_binary_matrix( + [ + "100000010110010010011", + "010000010100101111100", + "001000000111011111100", + "000100010101000101010", + "000010010110000011000", + "000001010011000111011", + "000000110001110101011", + "000000001110001010110", + ] + ) + + +def get_golay_checks() -> npt.NDArray[np.uint8]: + """A basis of the dual of the cyclic [23, 12, 7] Golay code, which contains its dual. + + As checks and as excluded rows, it gives a sector of the [[23, 1, 7]] CSS code of the Golay + code, which cyclic shifts preserve. + """ + generator = np.zeros(23, dtype=np.uint8) + generator[[0, 1, 5, 6, 7, 9, 11]] = 1 + return get_kernel(np.array([np.roll(generator, shift) for shift in range(12)])) + + def get_sector( code: codes.CSSCode, pauli: PauliXZ ) -> tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]: diff --git a/src/qldpc/codes/distance/connected_cluster.md b/src/qldpc/codes/distance/connected_cluster.md index e324d3062..b8c6d6708 100644 --- a/src/qldpc/codes/distance/connected_cluster.md +++ b/src/qldpc/codes/distance/connected_cluster.md @@ -4,8 +4,10 @@ The connected-cluster search computes the exact distance of a binary code from the Tanner graph of its parity checks. The code can be classical, CSS, or a qubit code that is not CSS, and a quantum code can be a subsystem code. -The search is `method="connected_cluster"` of `get_distance_exact` on the code classes, which defaults to Brouwer-Zimmermann (BZ, see [brouwer_zimmermann.md](brouwer_zimmermann.md)). -It also finds light vectors with a given syndrome for `CosetLeaderSearch`, which `certify_bounded_fault_propagation` uses. +The search is `method="connected_cluster"` of `get_distance_exact` on the code classes. +The default, `method="auto"`, runs it for the distances of quantum codes and the fault distances of circuits, unless enumerating every word is cheap. +`"auto"` switches to Brouwer-Zimmermann (BZ, see [brouwer_zimmermann.md](brouwer_zimmermann.md)) when BZ is predicted to cost less (section "Choosing a method" of [README.md](README.md)). +The connected-cluster search also finds light vectors with a given syndrome for `CosetLeaderSearch`, which `certify_bounded_fault_propagation` uses. The search is the connected-cluster algorithm of Dumer, Kovalev, and Pryadko, and later works cite their 2017 paper for it (see the [references](#references)). Its earliest known application to quantum codes is the linked-cluster technique of Kovalev, Dumer, and Pryadko (2013), for the distances of quantum LDPC codes. Dumer, Kovalev, and Pryadko (2014) then grew each cluster through the checks that it leaves unsatisfied, as qLDPC does. diff --git a/src/qldpc/codes/distance/core.py b/src/qldpc/codes/distance/core.py index 134a77665..ee449aebe 100644 --- a/src/qldpc/codes/distance/core.py +++ b/src/qldpc/codes/distance/core.py @@ -16,6 +16,7 @@ from qldpc._binary import get_independent_rows, get_kernel from qldpc.objects import PAULIS_XZ, Pauli, PauliXZ, PauliXZLike +from ._auto import BrouwerZimmermannProblem, is_cheap_to_enumerate, search_by_cost from ._binary import ( as_binary_matrix, as_binary_matrix_pair, @@ -34,11 +35,24 @@ from ._tuning import SearchTuning, get_tuning DistanceMethod = Literal[ - "brouwer_zimmermann", "information_sets", "connected_cluster", "brute_force" + "auto", "brouwer_zimmermann", "information_sets", "connected_cluster", "brute_force" ] """The methods of exact distance searches. -- ``"brouwer_zimmermann"`` (the default) and its alias ``"information_sets"``: the +- ``"auto"`` (the default of distance searches): a choice of the methods below, made for each + search, which aims to avoid a method that is very slow on the search, rather than to find the + fastest. It takes ``"brute_force"`` if that is cheap. Otherwise, a search of binary words that + excludes no nonzero word, as for a classical code, takes ``"brouwer_zimmermann"``. Any other + search, as for a quantum code or a circuit, runs ``"connected_cluster"``, and switches to + ``"brouwer_zimmermann"`` if that is predicted to cost less. A short ``"brouwer_zimmermann"`` + probe may give the result first, or cap the connected-cluster search (`Choosing a method + `_ + in the guide to the distance package). Functions that take generators choose between + ``"brute_force"`` and ``"brouwer_zimmermann"``, and coset searches, such as those of + :class:`~qldpc.codes.distance.CosetLeaderSearch`, take ``"brouwer_zimmermann"``. Every method + gives the same result unless the search exits early. Which method runs, and so which weight an + early exit at a ``cutoff`` returns, can depend on the tuning and on the version of qLDPC. +- ``"brouwer_zimmermann"`` and its alias ``"information_sets"``: the `Brouwer-Zimmermann algorithm `_, which enumerates words of increasing weight in several information sets, and stops once a lower @@ -98,10 +112,11 @@ def _missing_(cls, value: object) -> NoDistance | None: It behaves as a float ``nan``. Test for it with ``distance is NO_DISTANCE``. """ -_CanonicalDistanceMethod = Literal["brouwer_zimmermann", "connected_cluster", "brute_force"] +_CanonicalDistanceMethod = Literal["auto", "brouwer_zimmermann", "connected_cluster", "brute_force"] # The canonical name of every accepted method name, including aliases. _DISTANCE_METHODS: dict[str, _CanonicalDistanceMethod] = { + "auto": "auto", "brouwer_zimmermann": "brouwer_zimmermann", "information_sets": "brouwer_zimmermann", "connected_cluster": "connected_cluster", @@ -119,7 +134,7 @@ def get_distance_from_generators( *, symplectic: bool = False, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, automorphisms: npt.ArrayLike | Callable[[], npt.ArrayLike | None] | None = None, ) -> int | Literal[NoDistance.NAN]: @@ -146,9 +161,10 @@ def get_distance_from_generators( number of qubits on which it acts nontrivially. cutoff: Exit early once a nontrivial word of weight at most ``cutoff`` is found (see "Returns"). - method: ``"brouwer_zimmermann"`` (the default), its alias ``"information_sets"``, or - ``"brute_force"`` (see :data:`DistanceMethod`). ``"connected_cluster"`` needs parity - checks: use :func:`get_distance_from_checks`. + method: ``"auto"`` (the default), ``"brouwer_zimmermann"``, its alias + ``"information_sets"``, or ``"brute_force"`` (see :data:`DistanceMethod`). Here + ``"auto"`` chooses between ``"brute_force"`` and ``"brouwer_zimmermann"``. + ``"connected_cluster"`` needs parity checks: use :func:`get_distance_from_checks`. max_weight: If given, a nonnegative integer limit on the weights to search (see "Returns"). The search stops once it proves that every nontrivial word is heavier than the limit, which can be much faster than finding a large distance exactly. The limit caps @@ -203,7 +219,7 @@ def get_distance_from_checks( *, symplectic: bool = False, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, automorphisms: npt.ArrayLike | Callable[[], npt.ArrayLike | None] | None = None, ) -> int | Literal[NoDistance.NAN]: @@ -247,8 +263,9 @@ def get_distance_from_checks( :func:`get_distance_from_generators`). cutoff: Exit early once a nontrivial word of weight at most ``cutoff`` is found (see :func:`get_distance_from_generators`). - method: ``"brouwer_zimmermann"`` (the default), its alias ``"information_sets"``, - ``"connected_cluster"``, or ``"brute_force"`` (see :data:`DistanceMethod`). + method: ``"auto"`` (the default), ``"brouwer_zimmermann"``, its alias + ``"information_sets"``, ``"connected_cluster"``, or ``"brute_force"`` (see + :data:`DistanceMethod`). max_weight: If given, a nonnegative integer limit on the weights to search (see :func:`get_distance_from_generators`). automorphisms: Permutations of the bits, or of the qubits if ``symplectic`` is True, that @@ -308,7 +325,7 @@ def get_distance_css( pauli: PauliXZLike | None = None, *, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, automorphisms: npt.ArrayLike | Callable[[], npt.ArrayLike | None] | None = None, ) -> int | Literal[NoDistance.NAN]: @@ -336,9 +353,9 @@ def get_distance_css( Brouwer-Zimmermann and connected-cluster methods find faster than two calls would. cutoff: Exit early once a nontrivial logical operator of weight at most ``cutoff`` is found (see :func:`get_distance_from_generators`). - method: ``"brouwer_zimmermann"`` (the default), its alias ``"information_sets"``, - ``"connected_cluster"``, or ``"brute_force"`` (see :data:`DistanceMethod`, which also - covers subsystem codes). + method: ``"auto"`` (the default), ``"brouwer_zimmermann"``, its alias + ``"information_sets"``, ``"connected_cluster"``, or ``"brute_force"`` (see + :data:`DistanceMethod`, which also covers subsystem codes). max_weight: If given, a nonnegative integer limit on the weights to search (see :func:`get_distance_from_generators`). automorphisms: Permutations of the qubits that map the row space of ``checks_x``, and that @@ -379,7 +396,7 @@ def get_distance_pauli( checks: npt.ArrayLike, *, cutoff: int = 1, - method: DistanceMethod = "brouwer_zimmermann", + method: DistanceMethod = "auto", max_weight: SupportsIndex | None = None, automorphisms: npt.ArrayLike | Callable[[], npt.ArrayLike | None] | None = None, ) -> int | Literal[NoDistance.NAN]: @@ -404,9 +421,9 @@ def get_distance_pauli( checks: A binary matrix whose rows represent Pauli strings, which may be redundant. cutoff: Exit early once a nontrivial logical operator of weight at most ``cutoff`` is found (see :func:`get_distance_from_generators`). - method: ``"brouwer_zimmermann"`` (the default), its alias ``"information_sets"``, - ``"connected_cluster"``, or ``"brute_force"`` (see :data:`DistanceMethod`, which also - covers subsystem codes). + method: ``"auto"`` (the default), ``"brouwer_zimmermann"``, its alias + ``"information_sets"``, ``"connected_cluster"``, or ``"brute_force"`` (see + :data:`DistanceMethod`, which also covers subsystem codes). max_weight: If given, a nonnegative integer limit on the weights to search (see :func:`get_distance_from_generators`). automorphisms: Permutations of the qubits that map the row space of ``checks`` onto itself, @@ -456,7 +473,7 @@ def get_distance_pauli( # generators that commute with every gauge generator are stabilizers, which join the checks # (Section 3.9 of connected_cluster.md). kept_stabilizers = None - if method == "connected_cluster" and stabilizers is not gauge: + if method in ("auto", "connected_cluster") and stabilizers is not gauge: overlaps = multiply_binary(gauge, swap_pauli_halves(gauge).T) kept_stabilizers = gauge[~np.any(overlaps, axis=1)] @@ -489,8 +506,10 @@ def _search_css( ``_get_automorphism_getter``. """ # Each sector takes the stabilizers of the other type as checks, and excludes the gauge - # generators of its own type. + # generators of its own type. get_css_stabilizers returns gauge_x and gauge_z themselves unless + # the code is a subsystem code. stabilizers_x, stabilizers_z = get_css_stabilizers(gauge_x, gauge_z) + subsystem = stabilizers_x is not gauge_x sector_of_type = {Pauli.X: (stabilizers_z, gauge_x), Pauli.Z: (stabilizers_x, gauge_z)} sectors = [sector_of_type[sector_pauli] for sector_pauli in paulis] @@ -504,29 +523,38 @@ def _search_css( if num_logical_x == 0: return NO_DISTANCE + # "auto" enumerates the sectors if that is cheap, and searches them by the Brouwer-Zimmermann + # method if none excludes a nonzero word, as for a classical code. Otherwise, it searches them + # by cost, with a connected-cluster search that may switch to the Brouwer-Zimmermann method. + tuning = get_tuning() + if method == "auto": + dimensions = [checks.shape[1] - len(get_independent_rows(checks)) for checks, _ in sectors] + if is_cheap_to_enumerate(dimensions, gauge_x.shape[1], tuning): + method = "brute_force" + elif not any(np.any(excluded) for _, excluded in sectors): + method = "brouwer_zimmermann" + else: + return search_by_cost( + _get_cluster_sectors(gauge_x, gauge_z, paulis, sectors, subsystem, tuning), + _get_brouwer_zimmermann_problem( + [ + get_nested_code_basis(get_kernel(checks), excluded) + for checks, excluded in sectors + ], + symplectic=False, + get_automorphisms=get_automorphisms, + ), + symplectic=False, + cutoff=cutoff, + upper_bound=upper_bound, + get_automorphisms=get_automorphisms, + tuning=tuning, + ) + # Search the requested sectors at once, except by brute force, which searches each in turn. if method == "connected_cluster": - tuning = get_tuning() - if stabilizers_x is not gauge_x: - # get_css_stabilizers returns gauge_x and gauge_z themselves unless the code is a - # subsystem code, whose derived basis can be dense, so search sparser checks with the - # same span (Section 3.9 of connected_cluster.md). - overlaps = multiply_binary(gauge_x, gauge_z.T) - kept_stabilizers = { - Pauli.X: gauge_z[~np.any(overlaps, axis=0)], - Pauli.Z: gauge_x[~np.any(overlaps, axis=1)], - } - sectors = [ - ( - _get_sparser_stabilizers( - checks, kept_stabilizers[sector_pauli], symplectic=False, tuning=tuning - ), - excluded, - ) - for sector_pauli, (checks, excluded) in zip(paulis, sectors, strict=True) - ] return get_distance_connected_cluster( - sectors, + _get_cluster_sectors(gauge_x, gauge_z, paulis, sectors, subsystem, tuning), cutoff=cutoff, upper_bound=upper_bound, get_automorphisms=get_automorphisms, @@ -550,10 +578,41 @@ def _search_css( cutoff=cutoff, upper_bound=upper_bound, get_automorphisms=get_automorphisms, - tuning=get_tuning(), + tuning=tuning, ) +def _get_cluster_sectors( + gauge_x: npt.NDArray[np.uint8], + gauge_z: npt.NDArray[np.uint8], + paulis: list[PauliXZ], + sectors: list[tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]], + subsystem: bool, + tuning: SearchTuning, +) -> list[tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]]: + """The sectors of ``_search_css``, with the checks that a connected-cluster search takes. + + The derived stabilizers of a subsystem code can be dense, so its sectors take sparser checks + with the same span (Section 3.9 of ``connected_cluster.md``). Other sectors are unchanged. + """ + if not subsystem: + return sectors + overlaps = multiply_binary(gauge_x, gauge_z.T) + kept_stabilizers = { + Pauli.X: gauge_z[~np.any(overlaps, axis=0)], + Pauli.Z: gauge_x[~np.any(overlaps, axis=1)], + } + return [ + ( + _get_sparser_stabilizers( + checks, kept_stabilizers[sector_pauli], symplectic=False, tuning=tuning + ), + excluded, + ) + for sector_pauli, (checks, excluded) in zip(paulis, sectors, strict=True) + ] + + def _search_checks( checks: npt.NDArray[np.uint8], excluded: npt.NDArray[np.uint8], @@ -572,6 +631,19 @@ def _search_checks( ``kept_stabilizers`` is given, the checks are the derived stabilizers of a subsystem code, which a connected-cluster search replaces with those of ``_get_sparser_stabilizers``. """ + # "auto" searches Pauli strings, and binary words if the search excludes a nonzero word, by + # brute force or by cost. Other searches, as of a classical code, go on to _search_generators + # below, which enumerates if that is cheap and takes the Brouwer-Zimmermann method otherwise. + if method == "auto" and (symplectic or np.any(excluded)): + return _search_checks_by_cost( + checks, + excluded, + symplectic=symplectic, + cutoff=cutoff, + upper_bound=upper_bound, + get_automorphisms=get_automorphisms, + kept_stabilizers=kept_stabilizers, + ) if method != "connected_cluster": kernel = get_kernel(swap_pauli_halves(checks) if symplectic else checks) return _search_generators( @@ -604,6 +676,53 @@ def _search_checks( ) +def _search_checks_by_cost( + checks: npt.NDArray[np.uint8], + excluded: npt.NDArray[np.uint8], + *, + symplectic: bool, + cutoff: int, + upper_bound: int | None, + get_automorphisms: Callable[[], npt.NDArray[np.int_] | None] | None, + kept_stabilizers: npt.NDArray[np.uint8] | None, +) -> int | Literal[NoDistance.NAN]: + """Search the words that satisfy the checks by brute force if that is cheap, or by cost. + + The arguments are those of ``_search_checks``. ``search_by_cost`` runs a connected-cluster + search, which may switch to a Brouwer-Zimmermann search. + """ + tuning = get_tuning() + kernel = get_kernel(swap_pauli_halves(checks) if symplectic else checks) + nested = _get_nested_basis(kernel, excluded) + if nested is None: + return NO_DISTANCE + basis, labels = nested + if is_cheap_to_enumerate([len(basis)], basis.shape[1], tuning): + distance = get_nested_distance_brute_force( + basis, + labels, + cutoff=cutoff, + batch_exponent=tuning.enumeration_batch_exponent, + symplectic=symplectic, + ) + return _apply_upper_bound(distance, upper_bound) + if kept_stabilizers is not None: + checks = _get_sparser_stabilizers( + checks, kept_stabilizers, symplectic=symplectic, tuning=tuning + ) + return search_by_cost( + [(checks, excluded)], + _get_brouwer_zimmermann_problem( + [(basis, labels)], symplectic=symplectic, get_automorphisms=get_automorphisms + ), + symplectic=symplectic, + cutoff=cutoff, + upper_bound=upper_bound, + get_automorphisms=get_automorphisms, + tuning=tuning, + ) + + def _get_sparser_stabilizers( stabilizers: npt.NDArray[np.uint8], kept_stabilizers: npt.NDArray[np.uint8], @@ -642,26 +761,23 @@ def _search_generators( symplectic: bool, cutoff: int, upper_bound: int | None, - method: Literal["brouwer_zimmermann", "brute_force"], + method: Literal["auto", "brouwer_zimmermann", "brute_force"], get_automorphisms: Callable[[], npt.NDArray[np.int_] | None] | None, ) -> int | Literal[NoDistance.NAN]: """Search the words that the generators and excluded rows span, outside the excluded row space. ``upper_bound`` and ``get_automorphisms`` are as for ``_search_css``, and the other arguments - are those of ``get_distance_from_generators``, validated. + are those of ``get_distance_from_generators``, validated. Here ``"auto"`` chooses brute force + if that is cheap, and the Brouwer-Zimmermann method otherwise. """ tuning = get_tuning() - if len(excluded): - # An adapted basis and its quotient labels (Section 3.4 of brouwer_zimmermann.md). - basis, labels = get_nested_code_basis(generators, excluded) - if labels.shape[1] == 0: - return NO_DISTANCE - else: - basis = get_independent_rows(generators) - if not len(basis): - return NO_DISTANCE - # Labels with no columns exclude nothing. - labels = np.empty((len(basis), 0), dtype=np.uint8) + nested = _get_nested_basis(generators, excluded) + if nested is None: + return NO_DISTANCE + basis, labels = nested + if method == "auto": + cheap = is_cheap_to_enumerate([len(basis)], basis.shape[1], tuning) + method = "brute_force" if cheap else "brouwer_zimmermann" if method == "brute_force": distance = get_nested_distance_brute_force( @@ -673,26 +789,64 @@ def _search_generators( ) return _apply_upper_bound(distance, upper_bound) - # A Pauli string weighs half the Hamming weight of its image under symplectic_to_hamming - # (Section 2.7 of brouwer_zimmermann.md). - divisor = 1 - if symplectic: - num_qubits = basis.shape[1] // 2 - basis = symplectic_to_hamming(basis) - divisor = 2 - if get_automorphisms is not None: - get_automorphisms = functools.partial( - _get_lifted_automorphisms, get_automorphisms, num_qubits - ) + problem = _get_brouwer_zimmermann_problem( + [(basis, labels)], symplectic=symplectic, get_automorphisms=get_automorphisms + ) + scale = problem.weight_scale distance = get_distance_brouwer_zimmermann( - [(basis, labels)], - cutoff=cutoff * divisor, - weight_divisor=divisor, - upper_bound=None if upper_bound is None else upper_bound * divisor, - get_automorphisms=get_automorphisms, + problem.sectors, + cutoff=cutoff * scale, + weight_divisor=problem.weight_divisor, + upper_bound=None if upper_bound is None else upper_bound * scale, + get_automorphisms=problem.get_automorphisms, tuning=tuning, ) - return distance // divisor + return distance // scale + + +def _get_nested_basis( + generators: npt.NDArray[np.uint8], excluded: npt.NDArray[np.uint8] +) -> tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]] | None: + """A basis of the span of both matrices' rows and its labels, or None if every word is trivial. + + The trivial words are those in the row space of the excluded rows. With excluded rows, the + basis is adapted to them (Section 3.4 of ``brouwer_zimmermann.md``). Without excluded rows, + the labels have no columns, which exclude nothing. + """ + if len(excluded): + basis, labels = get_nested_code_basis(generators, excluded) + return None if labels.shape[1] == 0 else (basis, labels) + basis = get_independent_rows(generators) + if not len(basis): + return None + return basis, np.empty((len(basis), 0), dtype=np.uint8) + + +def _get_brouwer_zimmermann_problem( + sectors: list[tuple[npt.NDArray[np.uint8], npt.NDArray[np.uint8]]], + *, + symplectic: bool, + get_automorphisms: Callable[[], npt.NDArray[np.int_] | None] | None, +) -> BrouwerZimmermannProblem: + """The problem of a BZ search of sectors, given by their nested bases and labels. + + A Pauli string weighs half the Hamming weight of its image under ``symplectic_to_hamming`` + (Section 2.7 of ``brouwer_zimmermann.md``), so for Pauli strings the bases are mapped to those + images, and the automorphisms are lifted to their coordinates. + """ + if not symplectic: + return BrouwerZimmermannProblem(sectors, get_automorphisms=get_automorphisms) + num_qubits = sectors[0][0].shape[1] // 2 + if get_automorphisms is not None: + get_automorphisms = functools.partial( + _get_lifted_automorphisms, get_automorphisms, num_qubits + ) + return BrouwerZimmermannProblem( + [(symplectic_to_hamming(basis), labels) for basis, labels in sectors], + weight_scale=2, + weight_divisor=2, + get_automorphisms=get_automorphisms, + ) def _as_matrix_and_excluded( @@ -797,15 +951,15 @@ def _validate_distance_method( """Validate the name of a distance method, and return the canonical name of the method.""" if not isinstance(method, str) or method not in _DISTANCE_METHODS: raise ValueError( - f"Unknown distance method {method!r}; choose from 'brouwer_zimmermann' (or its alias" - " 'information_sets'), 'brute_force', or 'connected_cluster'." + f"Unknown distance method {method!r}; choose from 'auto', 'brouwer_zimmermann' (or its" + " alias 'information_sets'), 'brute_force', or 'connected_cluster'." ) return _DISTANCE_METHODS[method] def _reject_connected_cluster( method: _CanonicalDistanceMethod, -) -> Literal["brouwer_zimmermann", "brute_force"]: +) -> Literal["auto", "brouwer_zimmermann", "brute_force"]: """Return the method, unless it is ``"connected_cluster"``, which needs parity checks.""" if method == "connected_cluster": raise ValueError( diff --git a/src/qldpc/codes/distance/core_test.py b/src/qldpc/codes/distance/core_test.py index 97fcfa844..2173cccde 100644 --- a/src/qldpc/codes/distance/core_test.py +++ b/src/qldpc/codes/distance/core_test.py @@ -29,17 +29,30 @@ from ._tuning import use_tuning from .conftest import get_random_full_rank_matrix, get_span +# "auto" enumerates the small codes of the tests that take every method, unless its tuning makes +# enumeration costly (test_auto_without_enumeration_gives_the_same_distances). +METHODS: list[core.DistanceMethod] = [ + "auto", + "brouwer_zimmermann", + "connected_cluster", + "brute_force", +] + def test_brouwer_zimmermann_information_set_search() -> None: """Information-set search excludes stabilizers, finds logical products, and honors cutoffs.""" # The stabilizer of weight one is lighter than every logical operator, and with one support per # batch, some batches hold only that stabilizer. with use_tuning(information_set_batch_exponent=0): - distance = core.get_distance_from_generators([[1, 1, 1, 1]], [[1, 0, 0, 0]], cutoff=0) + distance = core.get_distance_from_generators( + [[1, 1, 1, 1]], [[1, 0, 0, 0]], cutoff=0, method="brouwer_zimmermann" + ) assert distance == 3 # The lightest logical operator is the product of both logical generators. - distance = core.get_distance_from_generators([[1, 1, 1, 0], [1, 1, 0, 1]], [], cutoff=0) + distance = core.get_distance_from_generators( + [[1, 1, 1, 0], [1, 1, 0, 1]], [], cutoff=0, method="brouwer_zimmermann" + ) assert distance == 2 # Every row of every information set weighs at least three, so the codeword of weight two needs @@ -51,16 +64,26 @@ def test_brouwer_zimmermann_information_set_search() -> None: [0, 1, 1, 0, 0, 0, 1, 1, 1, 0], [0, 1, 0, 0, 1, 0, 0, 1, 1, 1], ] - assert core.get_distance_from_generators(generators, cutoff=0) == 2 + assert core.get_distance_from_generators(generators, cutoff=0, method="brouwer_zimmermann") == 2 # Both reduced generators weigh three, and their sum of weight two reaches the cutoff. - assert core.get_distance_from_generators([[1, 0, 1, 1], [0, 1, 1, 1]], cutoff=2) == 2 + assert ( + core.get_distance_from_generators( + [[1, 0, 1, 1], [0, 1, 1, 1]], cutoff=2, method="brouwer_zimmermann" + ) + == 2 + ) # Row reduction exposes a Hamming-weight-one generator before any search runs. - assert core.get_distance_from_generators([[1, 1, 0], [0, 1, 1], [1, 1, 1]], cutoff=1) == 1 + assert ( + core.get_distance_from_generators( + [[1, 1, 0], [0, 1, 1], [1, 1, 1]], cutoff=1, method="brouwer_zimmermann" + ) + == 1 + ) -@pytest.mark.parametrize("method", ["brouwer_zimmermann", "connected_cluster", "brute_force"]) +@pytest.mark.parametrize("method", METHODS) def test_css_distances(method: core.DistanceMethod) -> None: """CSS distances of sectors, of codes, of subsystem codes, and of codes that encode nothing.""" checks_x = [[1, 1, 1, 1, 1, 1, 1], [1, 0, 1, 1, 1, 0, 0]] @@ -91,7 +114,7 @@ def test_css_distances(method: core.DistanceMethod) -> None: assert core.get_distance_css(np.zeros((0, 0)), [], method=method) is core.NO_DISTANCE -@pytest.mark.parametrize("method", ["brouwer_zimmermann", "connected_cluster", "brute_force"]) +@pytest.mark.parametrize("method", METHODS) def test_distances_from_checks(method: core.DistanceMethod) -> None: """Checks give classical, CSS, and stabilizer distances, and excluded rows must satisfy them.""" hamming_code = qldpc.codes.HammingCode(3) @@ -122,7 +145,7 @@ def test_distances_from_checks(method: core.DistanceMethod) -> None: assert core.get_distance_from_checks([], [[1, 1, 0]], method=method) == 1 -@pytest.mark.parametrize("method", ["brouwer_zimmermann", "connected_cluster", "brute_force"]) +@pytest.mark.parametrize("method", METHODS) def test_distances_of_stabilizer_and_subsystem_codes(method: core.DistanceMethod) -> None: """Pauli checks give the distances of stabilizer codes, and dressed ones of subsystem codes.""" five_qubit_code = qldpc.codes.FiveQubitCode() @@ -149,6 +172,137 @@ def test_distances_of_stabilizer_and_subsystem_codes(method: core.DistanceMethod assert core.get_distance_from_checks(gauge, no_stabilizers, symplectic=True, method=method) == 3 +def test_auto_without_enumeration_gives_the_same_distances() -> None: + """With enumeration never cheap, "auto" searches otherwise, and finds the same distances.""" + with use_tuning(auto_brute_force_max_words=0): + test_css_distances("auto") + test_distances_from_checks("auto") + test_distances_of_stabilizer_and_subsystem_codes("auto") + test_bare_and_dressed_distances("auto") + + +def test_auto_takes_the_method_of_its_rule(monkeypatch: pytest.MonkeyPatch) -> None: + """By default, a search enumerates if that is cheap, and otherwise takes the method of its kind. + + Words that only generators give, and binary words that exclude nothing nonzero, take the BZ + method, and other words a search by cost, including Pauli strings that exclude nothing and CSS + sectors of which only some exclude nothing. + """ + taken: list[str] = [] + for name in ("get_nested_distance_brute_force", "get_distance_brouwer_zimmermann"): + monkeypatch.setattr(core, name, _record_call(taken, name, getattr(core, name))) + monkeypatch.setattr(core, "search_by_cost", _record_call(taken, "by cost", core.search_by_cost)) + + hamming_code = qldpc.codes.HammingCode(3) + steane_code = qldpc.codes.SteaneCode() + five_qubit_code = qldpc.codes.FiveQubitCode() + # The X checks of the last two searches are empty. So their X sector excludes nothing, and + # every Z-type string satisfies the checks, so the Z sector has words of weight 1. + searches: list[tuple[Callable[[], Any], str, int]] = [ + ( + lambda: core.get_distance_from_checks(hamming_code.matrix), + "get_distance_brouwer_zimmermann", + 3, + ), + ( + lambda: core.get_distance_from_generators(hamming_code.generator), + "get_distance_brouwer_zimmermann", + 3, + ), + ( + lambda: core.get_distance_from_generators( + steane_code.get_logical_ops(Pauli.X), steane_code.get_stabilizer_ops(Pauli.X) + ), + "get_distance_brouwer_zimmermann", + 3, + ), + ( + lambda: core.get_distance_from_checks(steane_code.matrix_z, steane_code.matrix_x), + "by cost", + 3, + ), + (lambda: core.get_distance_css(steane_code.matrix_x, steane_code.matrix_z), "by cost", 3), + (lambda: core.get_distance_pauli(five_qubit_code.matrix), "by cost", 3), + ( + lambda: core.get_distance_from_checks( + five_qubit_code.matrix, np.zeros((0, 10), int), symplectic=True + ), + "by cost", + 3, + ), + ( + lambda: core.get_distance_css(np.zeros((0, 7), int), hamming_code.matrix, "X"), + "get_distance_brouwer_zimmermann", + 3, + ), + ( + lambda: core.get_distance_css(np.zeros((0, 7), int), hamming_code.matrix), + "by cost", + 1, + ), + ] + for search, method, distance in searches: + taken.clear() + assert search() == distance + assert taken[:1] == ["get_nested_distance_brute_force"] + taken.clear() + with use_tuning(auto_brute_force_max_words=0): + assert search() == distance + assert taken[:1] == [method] + + +def test_auto_enumerates_up_to_its_budget_of_packed_integers( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A search enumerates if all of its spaces fit the budget, and not with one integer less. + + The two sectors of the Steane code have dimension 4 each, the Hamming code has dimension 4, and + the Pauli strings that commute with the five-qubit code have dimension 6, all within one 64-bit + integer per word. The two words of the repetition code of length 65 take two integers each. + A word has a bit for each bit or qubit of a CSS or classical code, and two for each qubit of a + Pauli string. + """ + taken: list[str] = [] + brute_force = core.get_nested_distance_brute_force + monkeypatch.setattr( + core, "get_nested_distance_brute_force", _record_call(taken, "", brute_force) + ) + widths: list[int] = [] + is_cheap_to_enumerate = core.is_cheap_to_enumerate + + def record_width(dimensions: list[int], width: int, tuning: Any) -> bool: + widths.append(width) + return is_cheap_to_enumerate(dimensions, width, tuning) + + monkeypatch.setattr(core, "is_cheap_to_enumerate", record_width) + steane_code = qldpc.codes.SteaneCode() + hamming_code = qldpc.codes.HammingCode(3) + five_qubit_code = qldpc.codes.FiveQubitCode() + searches: list[tuple[Callable[[], Any], int, int, int]] = [ + (lambda: core.get_distance_css(steane_code.matrix_x, steane_code.matrix_z), 2 * 2**4, 7, 3), + (lambda: core.get_distance_from_generators(hamming_code.generator), 2**4, 7, 3), + (lambda: core.get_distance_pauli(five_qubit_code.matrix), 2**6, 10, 3), + (lambda: core.get_distance_from_generators(np.ones((1, 65), int)), 2 * 2, 65, 65), + ] + for search, num_integers, width, distance in searches: + for max_words, enumerates in ((num_integers, True), (num_integers - 1, False)): + taken.clear() + widths.clear() + with use_tuning(auto_brute_force_max_words=max_words): + assert search() == distance + assert bool(taken) is enumerates and widths == [width] + + +def _record_call(taken: list[str], name: str, function: Callable[..., Any]) -> Callable[..., Any]: + """Wrap a function to record its name in ``taken`` whenever it is called.""" + + def recording(*args: Any, **kwargs: Any) -> Any: + taken.append(name) + return function(*args, **kwargs) + + return recording + + def test_pauli_checks_of_css_codes_are_searched_as_css_codes( monkeypatch: pytest.MonkeyPatch, ) -> None: @@ -173,7 +327,7 @@ def record(sectors: Any, **kwargs: Any) -> Any: assert searched == [([7, 7], False)] -@pytest.mark.parametrize("method", ["brouwer_zimmermann", "connected_cluster", "brute_force"]) +@pytest.mark.parametrize("method", METHODS) def test_bare_and_dressed_distances(method: core.DistanceMethod) -> None: """The recipes of get_distance_from_checks give the bare and dressed distances of a code.""" # The only stabilizer of this subsystem code is Z on its last qubit, and X on one of its first @@ -258,7 +412,10 @@ def test_distance_functions_check_their_input() -> None: def test_automorphisms_are_checked_and_asked_for_once(tune: Tune) -> None: - """Automorphisms must be permutations, and a function returning them is called at most once.""" + """Automorphisms are checked, and a function returning them is called at most once. + + They must be permutations, and they must preserve the code. + """ # With automorphism_minimum_states=0, a connected-cluster search asks for automorphisms before # its first weight limit, so both sectors of the bicycle code below would ask. The code words # of the checks below are 000 and 111, so the cyclic shift [1, 2, 0] is an automorphism. @@ -273,14 +430,33 @@ def test_automorphisms_are_checked_and_asked_for_once(tune: Tune) -> None: assert distance == 3 get_automorphisms.assert_called_once() - # The searches of both sectors of a CSS code share one call. + # The searches of both sectors of a CSS code share one call, also when "auto" searches by cost + # and its connected-cluster search asks. A transposition is not an automorphism. code = qldpc.codes.MBCode([7], 1 + x + x**3, 1 + x**2 + x**3 + x**4) + transposition = np.arange(14) + transposition[[0, 1]] = [1, 0] + tune(auto_brute_force_max_words=0) + for css_method in ("connected_cluster", "auto"): + get_automorphisms = mock.Mock(return_value=code._get_qudit_automorphisms()) + distance = core.get_distance_css( + code.matrix_x, code.matrix_z, method=css_method, automorphisms=get_automorphisms + ) + assert distance == 3 + get_automorphisms.assert_called_once() + with pytest.raises(ValueError, match="not an automorphism"): + core.get_distance_css( + code.matrix_x, code.matrix_z, method=css_method, automorphisms=transposition[None] + ) + + # A search of Pauli strings by cost asks once too, from the connected-cluster search of its free + # phase. Hadamards on the second block of the code keep the automorphisms above. + pauli_checks = code.conjugated(range(7, 14)).matrix get_automorphisms = mock.Mock(return_value=code._get_qudit_automorphisms()) - distance = core.get_distance_css( - code.matrix_x, code.matrix_z, method="connected_cluster", automorphisms=get_automorphisms - ) + distance = core.get_distance_pauli(pauli_checks, method="auto", automorphisms=get_automorphisms) assert distance == 3 get_automorphisms.assert_called_once() + with pytest.raises(ValueError, match="not an automorphism"): + core.get_distance_pauli(pauli_checks, method="auto", automorphisms=transposition[None]) for automorphisms in ([[1, 0]], [[0, 0, 1]], [[[1, 2, 0]]]): with pytest.raises(ValueError, match="permutations of range"): @@ -291,8 +467,18 @@ def test_automorphisms_are_checked_and_asked_for_once(tune: Tune) -> None: ) -def test_connected_cluster_searches_sparse_stabilizers(monkeypatch: pytest.MonkeyPatch) -> None: - """A connected-cluster search of a subsystem code searches sparse checks of its stabilizers.""" +@pytest.mark.parametrize( + ("method", "searcher"), + [("connected_cluster", "get_distance_connected_cluster"), ("auto", "search_by_cost")], +) +def test_connected_cluster_searches_sparse_stabilizers( + monkeypatch: pytest.MonkeyPatch, tune: Tune, method: core.DistanceMethod, searcher: str +) -> None: + """A connected-cluster search of a subsystem code searches sparse checks of its stabilizers. + + So does a search by cost, which "auto" runs when enumeration is not cheap. + """ + tune(auto_brute_force_max_words=0) # With X and Z on a bulk qubit of the [[16, 1, 4]] surface code as gauge generators, the two # checks of each type on that qubit merge into one stabilizer of weight 6, the dressed distance # drops to 3, and the row-reduced stabilizers have rows of weight 8. The other checks are @@ -314,13 +500,13 @@ def get_stabilizers(checks: npt.NDArray[np.uint8]) -> npt.NDArray[np.uint8]: searched: list[npt.NDArray[np.uint8]] = [] - def record(sectors: Any, **kwargs: Any) -> Any: + def record(sectors: Any, *args: Any, **kwargs: Any) -> Any: searched.extend(np.asarray(checks) for checks, _ in sectors) - return get_distance_connected_cluster(sectors, **kwargs) + return search(sectors, *args, **kwargs) - get_distance_connected_cluster = core.get_distance_connected_cluster - monkeypatch.setattr(core, "get_distance_connected_cluster", record) - distance = core.get_distance_css(gauge_x, gauge_z, method="connected_cluster") + search = getattr(core, searcher) + monkeypatch.setattr(core, searcher, record) + distance = core.get_distance_css(gauge_x, gauge_z, method=method) assert distance == core.get_distance_css(gauge_x, gauge_z, method="brute_force") == 3 for checks, stabilizers in zip( searched, (get_stabilizers(checks_z), get_stabilizers(checks_x)), strict=True @@ -328,10 +514,10 @@ def record(sectors: Any, **kwargs: Any) -> Any: assert checks.sum(axis=1).max() <= 6 assert set(map(tuple, stabilizers[:-1])) <= set(map(tuple, checks)) assert np.array_equal(get_independent_rows(checks), get_independent_rows(stabilizers)) - assert core.get_distance_css(gauge_x, gauge_z, Pauli.Z, method="connected_cluster") == 3 + assert core.get_distance_css(gauge_x, gauge_z, Pauli.Z, method=method) == 3 searched.clear() - assert core.get_distance_css(checks_x, checks_z, method="connected_cluster") == 4 + assert core.get_distance_css(checks_x, checks_z, method=method) == 4 assert [checks.tolist() for checks in searched] == [checks_z.tolist(), checks_x.tolist()] # Hadamards on half of the qubits make a Pauli form that is not CSS. @@ -345,7 +531,7 @@ def apply_hadamards(rows: npt.NDArray[np.uint8]) -> npt.NDArray[np.uint8]: return rows gauge = apply_hadamards(scipy.linalg.block_diag(gauge_x, gauge_z)) - assert core.get_distance_pauli(gauge, method="connected_cluster") == 3 + assert core.get_distance_pauli(gauge, method=method) == 3 (checks,) = searched assert (checks[:, : len(code)] | checks[:, len(code) :]).sum(axis=1).max() <= 6 stabilizers = scipy.linalg.block_diag(get_stabilizers(checks_x), get_stabilizers(checks_z)) @@ -411,15 +597,16 @@ def test_special_inputs_of_generators() -> None: (qldpc.codes.classical.RingCode(8), 8), ], ) +@pytest.mark.parametrize("method", ["auto", "brouwer_zimmermann"]) def test_distances_of_known_classical_codes( - code: qldpc.codes.ClassicalCode, expected_distance: int + code: qldpc.codes.ClassicalCode, expected_distance: int, method: core.DistanceMethod ) -> None: """Classical distances match known values for Hamming, repetition, and ring codes.""" - assert core.get_distance_from_generators(code.generator) == expected_distance - assert core.get_distance_from_checks(code.matrix) == expected_distance + assert core.get_distance_from_generators(code.generator, method=method) == expected_distance + assert core.get_distance_from_checks(code.matrix, method=method) == expected_distance code._distance = None - assert code.get_distance_exact() == expected_distance + assert code.get_distance_exact(method=method) == expected_distance @pytest.mark.parametrize( @@ -433,26 +620,31 @@ def test_distances_of_known_classical_codes( (qldpc.codes.quantum.ToricCode(4), 4), ], ) -def test_distances_of_known_css_codes(code: qldpc.codes.CSSCode, expected_distance: int) -> None: +@pytest.mark.parametrize("method", ["auto", "brouwer_zimmermann"]) +def test_distances_of_known_css_codes( + code: qldpc.codes.CSSCode, expected_distance: int, method: core.DistanceMethod +) -> None: """Quantum distances match known values for C4, C6, Steane, and small surface/toric codes.""" distance_x = core.get_distance_from_generators( code.get_logical_ops(qldpc.objects.Pauli.X), code.get_stabilizer_ops(qldpc.objects.Pauli.X), + method=method, ) distance_z = core.get_distance_from_generators( code.get_logical_ops(qldpc.objects.Pauli.Z), code.get_stabilizer_ops(qldpc.objects.Pauli.Z), + method=method, ) distance_all = core.get_distance_from_generators( - code.get_logical_ops(), code.get_stabilizer_ops(), symplectic=True + code.get_logical_ops(), code.get_stabilizer_ops(), symplectic=True, method=method ) - distance_css = core.get_distance_css(code.matrix_x, code.matrix_z) + distance_css = core.get_distance_css(code.matrix_x, code.matrix_z, method=method) assert distance_x == distance_z == distance_all == distance_css == expected_distance # CSSCode.forget_distance also clears the cached distances of the X and Z sectors, which the # method would otherwise reuse. code.forget_distance() - assert code.get_distance_exact() == expected_distance + assert code.get_distance_exact(method=method) == expected_distance @pytest.mark.parametrize( @@ -461,19 +653,18 @@ def test_distances_of_known_css_codes(code: qldpc.codes.CSSCode, expected_distan (qldpc.codes.quantum.FiveQubitCode(), 3), ], ) +@pytest.mark.parametrize("method", ["auto", "brouwer_zimmermann"]) def test_distances_of_known_noncss_codes( - code: qldpc.codes.QuditCode, expected_distance: int + code: qldpc.codes.QuditCode, expected_distance: int, method: core.DistanceMethod ) -> None: """Quantum distances match known values for small non-CSS codes.""" distance = core.get_distance_from_generators( - code.get_logical_ops(), - code.get_stabilizer_ops(), - symplectic=True, + code.get_logical_ops(), code.get_stabilizer_ops(), symplectic=True, method=method ) - assert distance == core.get_distance_pauli(code.matrix) == expected_distance + assert distance == core.get_distance_pauli(code.matrix, method=method) == expected_distance code._distance = None - assert code.get_distance_exact() == expected_distance + assert code.get_distance_exact(method=method) == expected_distance def test_quantum_distances_use_qubit_automorphisms(tune: Tune) -> None: @@ -497,25 +688,28 @@ def test_quantum_distances_use_qubit_automorphisms(tune: Tune) -> None: expected = search(method="brute_force") for automorphisms in (translations, None): get_automorphisms = mock.Mock(return_value=automorphisms) - assert search(automorphisms=get_automorphisms) == expected == 3 + distance = search(automorphisms=get_automorphisms, method="brouwer_zimmermann") + assert distance == expected == 3 get_automorphisms.assert_called_once() with pytest.raises(ValueError, match="not an automorphism"): - search(automorphisms=transposition[None]) + search(automorphisms=transposition[None], method="brouwer_zimmermann") def test_brouwer_zimmermann_random_cross_checks() -> None: """Information-set search agrees with exhaustive enumeration on random nested codes.""" + BZ: core.DistanceMethod = "brouwer_zimmermann" rng = np.random.default_rng(118) for length in range(5, 10): basis = get_random_full_rank_matrix(rng, min(5, length - 1), length) expected = core.get_distance_from_generators(basis, cutoff=0, method="brute_force") - assert core.get_distance_from_generators(basis, cutoff=0) == expected + assert core.get_distance_from_generators(basis, cutoff=0, method=BZ) == expected stabilizers, logical_ops = basis[:2], basis[2:] expected = core.get_distance_from_generators( logical_ops, stabilizers, cutoff=0, method="brute_force" ) - assert core.get_distance_from_generators(logical_ops, stabilizers, cutoff=0) == expected + distance = core.get_distance_from_generators(logical_ops, stabilizers, cutoff=0, method=BZ) + assert distance == expected for num_qubits in range(3, 7): basis = get_random_full_rank_matrix(rng, min(6, 2 * num_qubits - 1), 2 * num_qubits) @@ -523,10 +717,10 @@ def test_brouwer_zimmermann_random_cross_checks() -> None: expected = core.get_distance_from_generators( logical_ops, stabilizers, symplectic=True, cutoff=0, method="brute_force" ) - assert ( - core.get_distance_from_generators(logical_ops, stabilizers, symplectic=True, cutoff=0) - == expected + distance = core.get_distance_from_generators( + logical_ops, stabilizers, symplectic=True, cutoff=0, method=BZ ) + assert distance == expected def test_max_weight() -> None: @@ -562,9 +756,13 @@ def check(distance_with_limit: Callable[..., float], distance: int, method: str) def test_max_weight_caps_cutoff() -> None: """A large cutoff cannot make the search give up on a word within the limit.""" - # Both rows weigh more than the limit, but their sum weighs 3. + # Both rows weigh more than the limit, but their sum weighs 3, which the BZ search finds after + # the rows. The test pins BZ, since "auto" would enumerate so small a code. basis = [[1, 0, 1, 1, 1, 1], [0, 1, 1, 1, 1, 0]] - assert core.get_distance_from_generators(basis, cutoff=6, max_weight=3) == 3 + distance = core.get_distance_from_generators( + basis, cutoff=6, max_weight=3, method="brouwer_zimmermann" + ) + assert distance == 3 def test_max_weight_conventions_and_validation() -> None: @@ -599,7 +797,7 @@ def test_max_weight_conventions_and_validation() -> None: def test_distance_method_validation() -> None: """Exact-method selectors are restricted to documented values, and resolve aliases.""" - for method in ["brouwer_zimmermann", "brute_force", "connected_cluster"]: + for method in ["auto", "brouwer_zimmermann", "brute_force", "connected_cluster"]: assert core._validate_distance_method(method) == method # type: ignore[arg-type] assert core._validate_distance_method("information_sets") == "brouwer_zimmermann" with pytest.raises(ValueError, match="Unknown distance method"): diff --git a/src/qldpc/codes/distance/coset_leaders.py b/src/qldpc/codes/distance/coset_leaders.py index b4b79a2cd..1899bda35 100644 --- a/src/qldpc/codes/distance/coset_leaders.py +++ b/src/qldpc/codes/distance/coset_leaders.py @@ -76,10 +76,10 @@ class CosetLeaderSearch: its physical weight. method: ``"brouwer_zimmermann"`` (the default), with alias ``"information_sets"``, searches the cosets of binary codes by information sets (see the `Brouwer-Zimmermann guide - `_). - ``"connected_cluster"`` makes generators of the stabilizer group sparser, and grows - errors from the syndrome through them, so it can be fast only if those generators are - sparse (see the `connected-cluster guide + `_), + and so does ``"auto"``. ``"connected_cluster"`` makes generators of the stabilizer + group sparser, and grows errors from the syndrome through them, so it can be fast only + if those generators are sparse (see the `connected-cluster guide `_). It weighs errors by CSS weight only on CSS states. For a class without an error within a positive maximum weight, its lower bound is that maximum weight plus one, whereas the @@ -464,20 +464,24 @@ def _join_part_bounds(self, x_bound: MinWeightBound, z_bound: MinWeightBound) -> def _get_coset_search_method( method: DistanceMethod, ) -> Literal["brouwer_zimmermann", "connected_cluster"]: - """Validate a distance method that has coset searches, and return its canonical name.""" + """Validate a distance method that has coset searches, and return the method that they use. + + ``"auto"`` chooses the Brouwer-Zimmermann method, whose coset searches often prove higher + lower bounds on the weights of coset leaders. + """ try: canonical = _validate_distance_method(method) except ValueError: raise ValueError( - f"Unknown distance method {method!r}; choose from 'brouwer_zimmermann' (or its alias" - " 'information_sets') or 'connected_cluster'." + f"Unknown distance method {method!r}; choose from 'auto', 'brouwer_zimmermann' (or its" + " alias 'information_sets'), or 'connected_cluster'." ) from None if canonical == "brute_force": raise ValueError( - "The 'brute_force' distance method has no coset-leader search: use" - " 'brouwer_zimmermann' or 'connected_cluster'." + "The 'brute_force' distance method has no coset-leader search: use 'auto'," + " 'brouwer_zimmermann', or 'connected_cluster'." ) - return canonical + return "brouwer_zimmermann" if canonical == "auto" else canonical def _as_state_stabilizers(stabilizers: npt.ArrayLike) -> npt.NDArray[np.uint8]: diff --git a/src/qldpc/codes/distance/coset_leaders_test.py b/src/qldpc/codes/distance/coset_leaders_test.py index af9882967..d75dfca55 100644 --- a/src/qldpc/codes/distance/coset_leaders_test.py +++ b/src/qldpc/codes/distance/coset_leaders_test.py @@ -31,7 +31,7 @@ split_syndromes_into_words, ) from .core import DistanceMethod -from .coset_leaders import CosetLeaderSearch +from .coset_leaders import CosetLeaderSearch, _get_coset_search_method from .min_weight_bounds import MinWeightBound @@ -475,6 +475,11 @@ def test_methods_take_the_names_of_distance_methods() -> None: if method != "brute_force": assert_exact_answers(CosetLeaderSearch(matrix, method=method), matrix, False) + # This state is not CSS, so only the Brouwer-Zimmermann method weighs its errors by CSS weight, + # and "auto" takes that method. + assert _get_coset_search_method("auto") == "brouwer_zimmermann" + assert_exact_answers(CosetLeaderSearch(matrix, css_weight=True, method="auto"), matrix, True) + def test_properties_are_read_only() -> None: """The number of qubits and the measure of weight of a search are fixed when it is built.""" diff --git a/src/qldpc/codes/quantum/hypergraph_product_test.py b/src/qldpc/codes/quantum/hypergraph_product_test.py index bc3226ad5..c187db36b 100644 --- a/src/qldpc/codes/quantum/hypergraph_product_test.py +++ b/src/qldpc/codes/quantum/hypergraph_product_test.py @@ -48,7 +48,7 @@ def test_hypergraph_product( code.set_logical_ops(code.get_logical_ops(), skip_validation=False) # the closed-form X and Z distances agree with those of a plain CSS code with the same checks - if field == 2: # the Brouwer-Zimmermann search behind the generic route is binary + if field == 2: # the exact searches behind the generic route are binary plain = codes.CSSCode(code.matrix_x, code.matrix_z) with unittest.mock.patch("qldpc.external.gap.is_installed", return_value=False): assert code.get_distance(Pauli.X) == plain.get_distance(Pauli.X) diff --git a/src/qldpc/codes/quantum/self_orthogonal_test.py b/src/qldpc/codes/quantum/self_orthogonal_test.py index bdb2c13d6..da5ae6add 100644 --- a/src/qldpc/codes/quantum/self_orthogonal_test.py +++ b/src/qldpc/codes/quantum/self_orthogonal_test.py @@ -51,7 +51,8 @@ def test_reed_muller_css_brute_force_agreement() -> None: for (order, size), dist in [((0, 2), 2), ((1, 4), 4)]: code = codes.QuantumReedMullerCode(order, size) assert code.get_distance() == dist - assert codes.CSSCode(code.matrix_x, code.matrix_z).get_distance() == dist + plain = codes.CSSCode(code.matrix_x, code.matrix_z) + assert plain.get_distance(method="brute_force") == dist def test_iceberg_code_requires_an_even_length() -> None: diff --git a/src/qldpc/experimental/circuits/benchmarking/residual_weights.py b/src/qldpc/experimental/circuits/benchmarking/residual_weights.py index 7e23d7b4a..88aa6edc5 100644 --- a/src/qldpc/experimental/circuits/benchmarking/residual_weights.py +++ b/src/qldpc/experimental/circuits/benchmarking/residual_weights.py @@ -392,8 +392,9 @@ def estimate_residual_weight_histogram( seed: A seed for the random number generator. Default: None, for fresh randomness. method: The exact search of residual weights where the tables do not apply: ``"brouwer_zimmermann"``, its alias ``"information_sets"``, or ``"connected_cluster"`` - (see :class:`~qldpc.codes.distance.CosetLeaderSearch`). With the same ``seed``, the - methods give the same histogram. Default: ``"brouwer_zimmermann"``. + (see :class:`~qldpc.codes.distance.CosetLeaderSearch`); ``"auto"`` takes + ``"brouwer_zimmermann"`` here. With the same ``seed``, the methods give the same + histogram. Default: ``"brouwer_zimmermann"``. Returns: A ``ResidualWeightHistogram``.