Skip to content

chore(differential): the PyPSA parity gate certifies the pinned mathspec corpus - #2101

Draft
FBumann wants to merge 2 commits into
claude/tender-pasteur-zc4z96from
chore/parity-corpus-main
Draft

FBumann wants to merge 2 commits into
claude/tender-pasteur-zc4z96from
chore/parity-corpus-main

Conversation

@FBumann

@FBumann FBumann commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Prompt: "Separate pr" (on moving the parity corpus to mathspec main, asked on #2099)

Note

The following content was generated by AI.

Parity checks the corpus at the mathspec commit #2099 pins (5f1464c3) against PyPSA 3625c816: 80 rungs match, none known failing. Stacked on #2099.

What changed, evidence, decisions
  • Pins: corpus ref: 5f1464c334c1012cdee2947d4d55b4d1d6bef40a; PyPSA 3625c8168b0b45a7936ddcb3a4da99dd608125ca, the SHA in that corpus's examples/references/pypsa/reference.py
  • Rungs: 66 → 80 (71–82, 85, 86 new; rung 48 replaced by rung_48_start_up_period_weight); objective-only as before: 14, 70
  • prep.py: the ~30 parameters, one relation and two dims main's pypsa.yaml declares — store ratings/efficiencies/split dispatch, energy limits for Link and Process, commitment history after PyPSA's (*_brought_in), Bus_attached, Cycle_in_basis, flow-based tables, marginal-cost curve breakpoints
  • parity.py: expands each rung's curves before the build; names the expanded rows after linopy's; maps flow_based to GlobalConstraint-ram
  • deviations.yaml 7 → 14:
    • removed the two negated energy-balance entries: against this corpus the duals equal PyPSA's directly
    • nine new Generator-marginal_cost_piecewise_* entries: mathspec's .expand() keys and pads segments differently from linopy's formulation (rows per curve, padded segments, delta <= 1 on an ungated curve); objectives match on 81, 82, 86
  • untested.txt regenerated: −21 / +50 lines, mostly declarations no rung varies
  • tools/ladder.py: rung links point at headings in the corpus README on blob/main — they drift if mathspec renames a heading
  • Gates run: the parity runner as the workflow runs it, exit 0, and a second run left differential/pypsa byte-identical; pixi run check (5919 passed, 604 skipped, 5 xfailed); pixi run -e docs docs-build
  • Departure: the type table's topmost row touched is docs/ (regenerated ladder pages, mkdocs.yml); typed chore per "a new rung of the ladder is chore", so no changelog line

Generated by Claude Code

…pec corpus

The workflow checks out the corpus at the mathspec commit the project pins,
5f1464c3, and PyPSA at 3625c816, the commit that corpus's reference pins.
The prep learns what that corpus declares: the stores' power rating,
efficiencies and split dispatch, energy limits on every dispatching
component, the commitment a unit brings into the horizon, which buses and
cycles have a row, flow-based market coupling and piecewise marginal cost
curves. The runner expands each curve before the build and names the rows
the expansion writes after linopy's; the formulation differences between the
two are recorded in deviations.yaml. All 80 rungs match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018B3nA1xxh6NMJN6mrYY34Q
@coderabbitai

coderabbitai Bot commented Nov 19, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@read-the-docs-community

read-the-docs-community Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

@codspeed

codspeed Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 28.65%

1 warning

⚠️ 3 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 111 untouched benchmarks
⏩ 307 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_read[commitment-s-specsolve-parquet] 218.4 KB 684.9 KB -68.11%
⚡ test_window[curves-s-specsolve-highs-shape] 65.7 MB 60.9 MB +7.82%
⚡ test_window[curves-s-specsolve-highs-one] 65.9 MB 62.4 MB +5.65%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing chore/parity-corpus-main (8d253a4) with claude/tender-pasteur-zc4z96 (2417c7d)

Open in CodSpeed

Footnotes

  1. 307 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

…inopy pads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018B3nA1xxh6NMJN6mrYY34Q

FBumann commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator Author

Note

The following content was generated by AI.

The piecewise deviations, row by row. Same feasible set, same objective (rung 81: 30890 both; rung 82: 26135 both). Every recorded difference is one of four things:

  1. Key — mathspec has no segment dim, so a segment stands at one of its breakpoints; linopy numbers pieces _breakpoint_piece = lower breakpoint
  2. Scale — the chord row is multiplied through by the segment's width
  3. Vacuous rows — linopy keeps rows its bounds already imply, or with no variable at all
  4. Split — the status gate splits a link row into two blocks
row linopy 0.10.0 Spec.expand kind
chord (lp) y ≥ s·p + (y₀ − s·x₀) at piece k Δx·y ≥ Δy·(p − xₖ₊₁) + yₖ₊₁·Δx at breakpoint k+1 key + scale
domain_lo / domain_hi p ≥ min x, p ≤ max x over (snapshot, gen) same row, standing on the first / last breakpoint key
delta, order_binary pieces 0…n−1 lower breakpoint same
delta_bound δ ≤ z, 24 rows 20 rows vacuous
active_bound δ ≤ status, and δ ≤ 1 where no status — 24 rows δ ≤ status only — 8 rows vacuous
fill_order δₖ₊₁ ≤ δₖ at k (earlier), 16 rows δₖ ≤ δₖ₋₁ at k (later), 12 rows key + vacuous
binary_order zₖ₊₁ ≤ δₖ at k, 16 rows zₖ ≤ δₖ₋₁ at k, 12 rows key + vacuous
link / output_link one row per (snapshot, gen), x₀·active gated + ungated blocks, on the first breakpoint key + split
Why each, with the rows

1. Key

  • delta/order already sit at the lower breakpoint (_increments docstring: "a segment sits at its lower breakpoint") — they join linopy's pieces
  • chord, fill_order and binary_order sit at the later one: written as shift(…, offset=1) (the breakpoint before), so the first breakpoint has none
    • the same rows written at the lower breakpoint — shift(δ, along=breakpoint, offset=-1) <= δ — would key exactly as linopy's
    • so within mathspec one segment has two keys: lower for its columns, upper for its rows
  • domain and link rows read the value at an edge (x₀, x_max), and the language reads it by standing the row on that breakpoint; linopy reduces it away (min/max over breakpoints, isel(0, drop=True))
    • rows are identical; the extra breakpoint coordinate is why no dual joins

2. Scale (rung 81, curve 0→20→40→80 MW, cost 0→240→640→2440)

  • linopy: y ≥ 12p, y ≥ 20p − 160, y ≥ 45p − 1160
  • expansion: 20y ≥ 240p, 20y ≥ 400p − 3200, 40y ≥ 1800p − 46400 — the same lines × Δx (_segment_lines: "multiplied through by the run rather than dividing")
  • duals, solved on both sides: ours[bp k+1] = linopy[piece k] / Δx
    • 00:00: 2.0 → 0.1 (÷20); 01:00: 0.6 → 0.015 (÷40); 03:00: 3.0 → 0.075 (÷40)

3. Vacuous rows (rung 82: curve82 3 segments, block82 2, 4 snapshots)

  • linopy builds pieces on a grid padded to the longest curve and keeps a row whose variable is masked:
    • delta_bound at block82 piece 2: 0 ≤ 0 (4 rows)
    • active_bound there: −status ≤ 0 (4)
    • fill_order / binary_order at block82 piece 1: −δ₁ ≤ 0 (4 + 4)
  • PyPSA passes active = 1 for a generator that is not committable, so active_bound writes δ ≤ 1 for curve82 (12), which the variable's upper bound already states
  • the expansion's rows exist only where their variables do

4. Split

What would retire the entries

Also

  • the agent's fill_order / binary_order text said linopy's extra row is "at the last segment of each curve"; it is only at a curve shorter than the longest — fixed in 8d253a4 (text only, in deviations.yaml and the files that copy it)

Method

  • dumped n.optimize.create_model() rows and Spec.expand() blocks for rungs 81 and 82 at corpus 5f1464c3, PyPSA 3625c816, linopy 0.10.0
  • rung 86 not dumped
  • pixi run check passes on 8d253a4; parity CI pending

Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants