S1/S2 snap to the nearest swept λ point: in 5 of 42 areas both scenarios land on the SAME point

[MaartenHilferink]

Found while measuring choice-set pressure for the deck (p5).

The defect

S1 (same facility count as baseline) and S2 (same travel cost as baseline) are each located by a λ bracket, and the exported scenario is then read off the nearest swept grid point inside that bracket. When the two brackets share an endpoint, both scenarios snap to it and become identical.

area	S1 bracket	S2 bracket	both snap to
SE2	(0.2, 0.5)	(0.5, 1.0)	w = 0.5
FRC, FRH, FRJ, FRK	—	—	same pattern

5 of 42 areas. For SE2 the consequence is concrete: its delivered S1 has 309 open facilities, not the baseline 452 — it is really its S2 point wearing the S1 label. The scenario is defined as same count as today, and this file does not satisfy that.

What is and is not affected

• Unaffected: the interpolated λ tables (deck p57–58) and the crossings CSV (Provide frontier projection data beschikbaar maken. #48). Those interpolate between bracket endpoints and never snap.
• Affected: the exported S1/S2 open-location Arrow files (Share S1 and S2 results for some regions with Chris #45, and Chris's services_allocated reads them), and the stranding/choice-set counts read from the S1 assignment (deck p5 uses SE2's S2 for that reason).
• Also affected in principle: the planned exact soft-coverage p-median MIP, if it seeds from these files.

Fix

Replace the snap with a refinement step inside the bracket: bisect (or a few extra λ solves) until the S1 point's facility count is within a tolerance of the baseline count, and likewise S2's travel cost — then export that solution. The sweep already has a bisection routine for S2 (S2 bisection over w in … appears in the logs); S1 needs the same, and the export must take the refined point rather than the grid neighbour.

Until then, treat the S1 file as unreliable for FRC, FRH, FRJ, FRK and SE2; S2 is fine in all five (it is the point both snapped to).

[MaartenHilferink]

Measured, and a wider defect than the five areas — fixed in 29fcf7f, rerun in progress

How far off the exported S1/S2 files were

Over the 74 area×function sweeps (42 areas, LINEAR + LOGISTIC; deck_data.json scen rows, which are exactly what the sweep exported to the S1/S2 folders):

	S1 file: facility count vs today	S2 file: travel vs today
median error	5.9 %	0.31 %
off by > 5 %	41 of 74	14 of 74 (all LINEAR)
off by > 10 %	22 of 74	10 of 74
worst	Norway LOGISTIC −27.5 %, SE2 LINEAR −31.6 %, SE3 LINEAR +26.4 %	FRF LINEAR +16.9 %, FRK −15.3 %, FRH −14.5 %, FRD +13.9 %, FRJ −13.5 %, FRC −12.0 %
S1 = S2, same point	FRC, FRH, FRJ, FRK, SE2 (LINEAR) — as reported

So the issue's framing was too narrow in two ways. S1 is off by more than 10 % in 22 sweeps, not only in the five where it coincides with S2: the 1-2-5 grid is a factor 2–2.5 in λ per step and the count moves ~30 % per step, so "nearest grid point" is a coarse read anywhere. And "S2 is fine in all five" was wrong: S2 is 5–15 % off in those five and in nine more LINEAR areas (FRD, FRF, FRG, PL4, PL6, PL7, PL8, PL9, Portugal).

Why the existing S2 bisection did not help

The sweep already had a bisection for S2. It ran after the coarse sweep had climbed to w_max, and there it timed out at its first point in 14 of 42 LINEAR areas. The logs make the mechanism unambiguous:

• FRC LINEAR: w = 0.1 solved in 75 s as a grid step (warm-started from w = 0.05). The fine sweep then re-solved the same LP at w = 0.1 after the sweep had been through 1.0, 2.0 and 5.0 — each of which had timed out at 1 h — and that re-solve timed out at 1 h. Same LP, different starting basis.
• ITC LINEAR: the bisection's first point took 2,695 s from that far basis; the next five, each a near step, took 120 / 61 / 46 / 40 / 37 s.

Warm-started dual simplex is cheap for a near step and hopeless from the basis HiGHS is left with after a 1 h abort. The interior-point comment in lp_run.jl from July describes the same cliff.

The fix (29fcf7f)

• Both S1 and S2 are bisected the moment their bracket closes, inside the coarse loop, right after the grid point that closes it — every bisection solve is a near step. S1 bisects on sum_y (the LP count, monotone in λ) to within max(1, 0.2 %) of today's count; S2 on the multistart travel to within 0.2 % of today's travel. Up to 8 solves each.
• After the detour the bracket's upper endpoint is re-solved for its basis only (resolve_for_basis!), so the grid continues as if uninterrupted.
• Once both are pinned, the first tail time-out ends the sweep: the later points would start from the aborted basis and time out too (FRC lost 3 h per function that way).
• The fine sweep no longer re-solves grid points it already has (its default multipliers are the coarse grid; those re-solves were the far-basis failures in the LOGISTIC logs at 0.01/0.02/0.05).
• run_resweep_batch.ps1 -RefineOnly (SWEEP_STOP_AFTER_REFINE=1): walk the grid from 1e-4, pin both scenarios, stop — no high-λ tail. Writes logs/refine_<area>_<FUNC>.log; build_deck_data.py takes the S1/S2 summary from it when newer than the sweep log, so the frontier rows stay those of the full sweep.

Luxembourg LINEAR, refine-only, 35 s: S1 now 87 of 86 cells (was 69, −20 %), S2 within +0.3 % of today's travel (was read off a grid point 61 facilities wide).

Running now

Three refine-only workers over the 37 areas whose data did not change, in parallel with the #53 rebuild; the six areas whose pharmacy set changed (ITC, ITF, ITG, ITH, ITI, Hungary) get full sweeps afterwards. Results and the regenerated S1/S2 location files (#45) follow in a second comment.

What this does and does not change in the deck

The interpolated tables (p60–61) and the #48 crossings never used the snapped points, so they do not move. What moves: the S1/S2 markers on the region slides (p16–58), the LINEAR-vs-LOGISTIC table (p62), the exported S1/S2 location files (#45) and the stranding counts read from the S1 assignment (p9).

[MaartenHilferink]

Done — S1/S2 pinned in 87 of 88 sweeps (216ac77)

Every area was either fully re-swept (the six whose pharmacy set changed under #53, plus Finland) or refined in place (the 37 unchanged ones, -RefineOnly). Acceptance check doc/check_s1s2.py over deck_data.json:

	before	after
S1 file: count vs today, median	5.9 %	0.13 %
S1 off by > 10 %	22 of 74	0
S1 off by > 1 % (or > 1 facility)	—	1 of 88
S2 file: travel vs today, median	0.31 %	0.10 %
S2 off by > 1 %	14 of 74	0 (max 0.74 %)
S1 = S2 same point	5 areas	0

SE2 LINEAR, the headline case: S1 now 451 of 452 cells (was 309), S2 at 278 open within 0.05 % of today's travel.

The one exception

Country-level Poland LINEAR keeps its grid point (S1 +1.6 %, S2 −0.15 %). It is the largest LP in the study (274 MB OD), and near S1 its warm-started dual simplex needs 3–15 h per solve — the old full sweep spent 4.7 h on w = 0.1 and 15.5 h on w = 0.2 — so even the climb with a 4 h limit did not get there (w = 0.065: 79 min, 0.0845: 2.9 h, 0.11: time-out). Its seven NUTS-1 areas are all pinned and are what the aggregate and the ranked lists use; country-level Poland only appears as a region slide, a table row and a #45 file. Pinning it is a job for the IPM solver path or the exact MIP, not for another night of simplex.

What the refine-only walk turned out to need (all in the commits since 29fcf7f)

1. LP-only walk up to the bracket — the three roundings cost 50–100 s per grid point against a 7 s LP at low λ (e25416b).
2. Climb to S2 from the pinned S1 point in ×1.3 steps instead of taking the next grid step: that step's upper endpoint was the most expensive solve of the walk (Denmark: 1,883 s for w = 1.0 to bracket an S2 at 0.557; on SE2 it timed out) (6aaab27).
3. Climb to S1 likewise once the LP count is within ×2 of today's (92ab4e4); no basis restore after the last scenario (82d84b4); a timed-out climb step ends the walk instead of being retried (86e242e).

Two runs that had started before the climb landed went wrong and were redone: Austria LOGISTIC (time-out cascade after S1) and Denmark LOGISTIC (a fallback bisection across a failed point reported S2 "pinned" 9.5 % off — the summary line looked complete). That is why the acceptance check exists.

Wall time: full sweeps of the Italian areas 1.5–3 h each instead of 9–14 h (the tail stop); refine-only 5 min (Luxembourg) to 4 h (Austria) per area and function under 6-way contention.

Delivered on it

• Share S1 and S2 results for some regions with Chris #45: s1s2_open_locations_20260912.zip — 176 files, 44 areas, all on the pinned points.
• Provide frontier projection data beschikbaar maken. #48: issue48_frontier_data_20260912.zip.
• Deck p9 records the fix; the roadmap's exact-MIP item is now about the integrality gap, not the snap.