feat: dc solver multi outage import by spetznick-elia · Pull Request #723 · eliagroup/ToOp

spetznick-elia · 2026-09-08T12:49:00Z

Checklist

Please check if the PR fulfills these requirements:

PR Title follows conventional commit messages
Tests for the changes have been added (for bug fixes / features)
Docs have been added / updated (for bug fixes / features)
All commits in this PR are DCO signed-off (see CONTRIBUTING.md)

Does this PR already have an issue describing the problem?

Fixes #

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change?

Yes
No

Copilot

🟡 Changes recommended

Unresolved moderate findings affect AC sanitisation, DC projection, parsing, and configuration behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds grouped contingency import, canonical AC definitions, and DC-compatible projected definitions with provenance, SPPS handling, tests, and documentation.

Changes:

Imports grouped contingencies and SPPS rules.
Generates DC-specific contingency artifacts.
Updates AC sanitisation, preprocessing, tests, and documentation.

File summaries

File	Summary
`packages/topology_optimizer_pkg/tests/ac/test_ac_acceptance_integration.py`	Updates AC acceptance coverage.
`packages/topology_optimizer_pkg/src/toop_engine_topology_optimizer/ac/optimizer.py`	Filters AC contingencies; moderate (1 vote): non-complex definitions may lose intentional switch-only contingencies.
`packages/interfaces_pkg/tests/test_nminus1_definition.py`	Tests N-1 definition helpers.
`packages/interfaces_pkg/src/toop_engine_interfaces/nminus1_definition.py`	Adds copy/filter helpers; moderate (3 votes): filtering contingencies can leave orphaned SPPS rules.
`packages/interfaces_pkg/src/toop_engine_interfaces/messages/preprocess/preprocess_commands.py`	Adds complex schema selection.
`packages/interfaces_pkg/src/toop_engine_interfaces/messages/lf_service/loadflow_commands.py`	Adds SPPS sanitisation configuration; moderate (3 votes): the setting is not wired through.
`packages/interfaces_pkg/src/toop_engine_interfaces/folder_structure.py`	Adds the DC definition artifact path.
`packages/interfaces_pkg/src/toop_engine_interfaces/backend.py`	Adds contingency provenance mapping.
`packages/importer_pkg/tests/pypowsybl_import/test_preprocessing.py`	Tests complex definition persistence.
`packages/importer_pkg/src/toop_engine_importer/pypowsybl_import/preprocessing.py`	Loads and persists canonical definitions.
`packages/importer_pkg/src/toop_engine_importer/pypowsybl_import/powsybl_masks.py`	Handles complex contingency masks.
`packages/importer_pkg/src/toop_engine_importer/pypowsybl_import/contingency_from_file/complex_contingency_file.py`	Parses grouped files; moderate (2 votes): empty-ID neutral points may still be resolved.
`packages/dc_solver_pkg/tests/preprocessing/test_write_aux_data.py`	Tests artifact persistence.
`packages/dc_solver_pkg/tests/preprocessing/test_powsybl_backend.py`	Tests DC projection behavior.
`packages/dc_solver_pkg/tests/preprocessing/test_parallel_switch_edge_cases.py`	Covers projected parallel-switch cases.
`packages/dc_solver_pkg/tests/preprocessing/test_convert_to_jax.py`	Updates DC artifact expectations.
`packages/dc_solver_pkg/tests/preprocessing/test_complex_contingency_end_to_end.py`	Covers importer-to-DC integration.
`packages/dc_solver_pkg/tests/postprocessing/test_postprocess_powsybl.py`	Updates postprocessing inputs.
`packages/dc_solver_pkg/tests/jax/test_aggregate_results.py`	Updates aggregate-result fixtures.
`packages/dc_solver_pkg/tests/jax/benchmarks/test_bench_postprocessing.py`	Updates benchmark inputs.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/powsybl/powsybl_backend.py`	Projects contingencies; moderate findings: mixed non-switch cases can be misclassified (3 votes), mask-derived three-winding outages can be dropped (1 vote), and busbar provenance can be lost (2 votes).
`packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/network_data.py`	Preserves contingency provenance.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/convert_to_jax.py`	Writes the DC-specific definition.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/postprocess/write_aux_data.py`	Manages canonical and DC artifacts.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/jax/disconnections.py`	Updates JAX disconnection handling.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/jax/benchmarks/bench_postprocessing.py`	Updates postprocessing benchmarks.
`packages/dc_solver_pkg/src/toop_engine_dc_solver/example_grids.py`	Updates example grid definitions.
`packages/contingency_analysis_pkg/tests/powsybl/test_ca_powsybl_helpers.py`	Covers Powsybl helper behavior.
`packages/contingency_analysis_pkg/tests/ac_service/test_spps_sanitisation_boundary.py`	Tests AC SPPS sanitisation boundaries.
`packages/contingency_analysis_pkg/src/toop_engine_contingency_analysis/ac_loadflow_service/ac_loadflow_service.py`	Sanitises AC inputs; moderate (1 vote): schema inequality can remove deliberate switch-only contingencies.
`notebooks/example4_data_contracts.ipynb`	Demonstrates the canonical/DC data contract.
`docs/usage.md`	Documents the updated workflow.
`docs/dc_solver/quickstart.md`	Updates DC solver setup guidance.
`docs/dc_solver/preprocessing.md`	Documents DC preprocessing artifacts.
`data/test_station.json`	Removes obsolete test fixture data.

Review details

Suppressed comments (3)

packages/contingency_analysis_pkg/src/toop_engine_contingency_analysis/ac_loadflow_service/ac_loadflow_service.py:90

source_schema != "complex" is being used as a proxy for "mask-derived", but source_schema is optional and explicitly supplied non-complex definitions (including BaseGrid.n_1_definition) also have None. This removes deliberate switch-only contingencies from those inputs and changes AC behavior. Use explicit provenance or gate this cleanup behind an explicit sanitisation option instead of the schema inequality.

        # A mask-derived definition carries one contingency per switch, which for a node-breaker grid
        # means every disconnector. Those outage nothing but the equipment behind them and cannot
        # converge, so they are dropped here. A curated complex list is left untouched: a switch-only
        # case there is deliberate.
        n_minus_1_definition = copy_without_switch_only_contingencies(n_minus_1_definition)

packages/dc_solver_pkg/src/toop_engine_dc_solver/preprocess/powsybl/powsybl_backend.py:314

Mask-derived three-winding cases are still emitted by create_nminus1_definition_from_masks as one GridElement with the original transformer id and type THREE_WINDINGS_TRANSFORMER, while the converted Powsybl network exposes only the -Leg1/-Leg2/-Leg3 branch ids. branches is therefore empty here and those N-1 cases are dropped as unsupported, so default/mask-derived 3W outages disappear from DC. Expand or normalize this representation to the three leg ids before projection and cover a mask-generated 3W fixture.

                if element.kind == "branch" and element.id in supported_branch_ids
            ]

packages/topology_optimizer_pkg/src/toop_engine_topology_optimizer/ac/optimizer.py:255

The same source_schema != "complex" proxy is applied to every definition loaded by the optimizer. A curated non-complex nminus1_definition.json can contain intentional switch-only contingencies, but this path silently removes them before AC validation. Tie the cleanup to explicit mask-derived provenance or an explicit sanitisation setting rather than treating every non-complex definition as generated.

Files reviewed: 35/35 changed files
Comments generated: 5
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        branch_contingency_ids, injection_contingency_ids, _ = self._get_dc_contingency_projection()
+        return {**branch_contingency_ids, **injection_contingency_ids}


… solver Cut the Powsybl backend's dependency on *_for_nminus1 masks: outages are now taken exclusively from nminus1_definition.json (or the DC-written dc_nminus1_definition.json fallback for folders without an importer run), so grouped/multi-element contingencies written by an importer (including the complex parser added in a previous commit) reach the DC solver unchanged. _get_definition_mask lost its mask fallback and the uses_dc_definition switch; all six call sites now take outages from the definition only. get_busbar_outage_map is built from kind=="bus" contingencies instead of the busbar_for_nminus1 mask. Because producers must now write the definition before load_grid() reads it, save_nminus1_definition_from_masks() derives one from the masks a fixture already wrote and is wired into the Powsybl example-grid builders and into write_aux_data() as a non-overwriting provisioning step. DC's own projection of the canonical definition -- exactly the contingencies DC computes, in solver order -- is now written to dc_nminus1_definition.json as an output only; it is never read back as input, which used to shrink the busbar set on every run. NetworkData.contingency_ids now labels single outages by their source contingency id (BackendInterface.get_contingency_id_by_element_id, defaults to {} so PandaPower is unaffected) instead of by element id, and the single-outage vs. multi-outage split now keys off the *projected* arity instead of the source arity, so a line plus its two breakers collapses to one branch outage instead of a degenerate multi-outage row. Fixes a multi-outage bug found while wiring this up: a live multi-outage that islands the unsplit grid made the BSDF/LODF action filter reject every split of every substation, because modf_success is conjoined over all multi-outages and the MODF denominator is singular for any group that islands the base grid. exclude_bridges_from_outage_masks now runs an islanding test over whole multi-outage groups (find_islanding_branch_groups), not just single branches: an *imported* group that contains a bridge or is a cut set in its own right is dropped whole and reported at warning level (a curated contingency is computed as declared or not at all); a *synthesised* group (trafo3w, bus - one physical element that islands its own star node/busbar by construction) is repaired instead by sparing the minimal set of branches that resolves the islanding (find_branches_to_spare_from_groups), replacing the previous unconditional "blank the first branch" hack. The gap between a declared and a computed synthesised group is now recorded explicitly in NetworkData.multi_outage_spared_branch_mask. Also removes multi_outage_nodes end to end (BackendInterface method, both backend implementations, NetworkData/DynamicInformation fields, HDF5 datasets): it was produced, reduced, padded, serialized and shape-asserted but never read by any computation, since build_modf_matrices and apply_modf_matrices only ever took branches. Note: static_information.hdf5 no longer contains multi_outage_nodes_{idx}. Old code reads new files fine (lookup by key), but older code cannot read files written after this change. New end-to-end regression: tests/preprocessing/test_complex_contingency_end_to_end.py covers contingency_list_complex.json -> convert_file -> canonical -> projection -> PowsyblBackend -> NetworkData/StaticInformation and pins the switch-collapse table (a line + its breakers projects to a single branch outage; a converted 3W transformer's three legs project to a 3-branch multi-outage; a group with no DC-supported element is dropped). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

…pipeline create_nminus1_definition() now dispatches on importer_parameters.schema_format: for "ContingencyImportSchemaComplex" it calls load_nminus1_definition_from_file() (added in a previous commit) instead of deriving the definition from masks, using the mask-derived definition only for its monitored_elements and base_case. make_masks() becomes a no-op for the complex schema, since a mask cannot express grouped N-k outages -- it stays the importer's internal derivation step for every other input, never a transport to DC. save_shared_nminus1_definition() replaces the direct save_pydantic_model_fs() call in convert_file() / compute_network_masks_and_n_1_definition(): it persists the canonical definition and immediately reloads it, asserting the round trip is lossless, so a serialization gap can never silently change what DC and CA read back. BaseImporterParameters.schema_format gains the "ContingencyImportSchemaComplex" literal alongside the existing PowerFactory and CSV schemas. New regression: test_convert_file_complex_contingencies_persists_grouped_definition drives convert_file() over data/complex_grid/contingency_list_complex.json end to end and asserts the persisted definition keeps every grouped contingency, its SPPS rules, and source_schema == "complex". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

…evaluation Neither AC backend may execute a complex list's SPPS rules in this work: Pandapower would feed them to its SPPS engine and Powsybl has no engine for them at all. get_ac_loadflow_results() now strips spps_rules from an in-memory copy (copy_without_spps_rules) whenever n_minus_1_definition.source_schema == "complex"; the canonical definition on disk keeps its rules untouched. For every other input, a mask-derived definition auto-generates one contingency per switch in the N-1 area, so a node-breaker grid yields a contingency for every disconnector -- these outage only the equipment behind them, cannot converge, and used to make CA's ">len/2" non-convergence guard abort every run once DC stopped silently shrinking the definition it forwarded. get_ac_loadflow_results() and the AC optimizer's definition load now both apply copy_without_switch_only_contingencies() to drop these, gated on source_schema != "complex" so a curated list's deliberate switch-only case is kept. Both call sites need the filter independently: the topology optimizer reaches Powsybl CA through PowsyblRunner and bypasses get_ac_loadflow_results() entirely. Nminus1Definition gains the two copy helper functions used above (copy_without_spps_rules, copy_without_switch_only_contingencies); the source_schema field, the SPPS scheme-name validator and the widened condition_limit_value type were added by the parser commit. BaseGrid gains a sanitize_spps_rules flag for message-level callers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

…notebook Update docs/dc_solver/preprocessing.md, docs/dc_solver/quickstart.md and docs/usage.md to describe nminus1_definition.json as the DC solver's outage source instead of the *_for_nminus1 masks. Add notebooks/example4_data_contracts.ipynb, walking through importer -> canonical definition -> DC projection -> CA for a complex grouped-contingency import. Remove the unreferenced data/test_station.json fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

spetznick-elia changed the base branch from main to feat/complex-contingency-parser September 9, 2026 09:03

spetznick-elia added this pull request to stack #724 September 9, 2026 09:04

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from eb4aa01 to 18aa071 Compare September 9, 2026 09:05

spetznick-elia removed this pull request from stack #724 September 9, 2026 13:28

spetznick-elia added this pull request to stack #727 September 9, 2026 13:29

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from e3c3d38 to 4406bbd Compare September 10, 2026 11:35

spetznick-elia mentioned this pull request Sep 10, 2026

test: add test suite for ac validation after dc solver multi outage #728

Open

6 tasks

spetznick-elia marked this pull request as ready for review September 10, 2026 20:42

Copilot AI lite review requested due to automatic review settings September 10, 2026 20:43

Copilot started reviewing on behalf of spetznick-elia September 10, 2026 20:43 View session

Copilot AI reviewed Sep 10, 2026

View reviewed changes

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from 98cc8ca to 3c9bae7 Compare September 11, 2026 11:32

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from 3c9bae7 to 21d867b Compare September 14, 2026 09:04

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from 21d867b to 978ec42 Compare September 17, 2026 13:06

spetznick-elia and others added 9 commits September 17, 2026 15:15

fix: missed rename

797e6e6

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

refactor: naming

e116677

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

chore: remove stale sanitization, filter orphaned SPPS rules

11d3170

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

chore: remove stale docstring

bbfdad0

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

refactor: remova spps rules when preparing for Powsybl

d37bde6

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

spetznick-elia force-pushed the feat/dc-solver-multi-outage-import branch from 978ec42 to d37bde6 Compare September 17, 2026 13:16

chore: add import

6e3bc23

Signed-off-by: Sascha Petznick <229719644+spetznick-elia@users.noreply.github.com>

Navigation Menu

feat: dc solver multi outage import - #723

feat: dc solver multi outage import#723
spetznick-elia wants to merge 10 commits into
feat/complex-contingency-parserfrom
feat/dc-solver-multi-outage-import

spetznick-elia commented Sep 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		branch_contingency_ids, injection_contingency_ids, _ = self._get_dc_contingency_projection()
		return {branch_contingency_ids, injection_contingency_ids}

Conversation

spetznick-elia commented Sep 8, 2026

Checklist

Does this PR already have an issue describing the problem?

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (3)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants