diff --git a/research/p104-loan-snapshot-engine-on7621/EXPECTED.json b/research/p104-loan-snapshot-engine-on7621/EXPECTED.json new file mode 100644 index 0000000000..49c8835bd7 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/EXPECTED.json @@ -0,0 +1 @@ +{"kappa_prefix": "0.0036624027915539"} diff --git a/research/p104-loan-snapshot-engine-on7621/NOTICE b/research/p104-loan-snapshot-engine-on7621/NOTICE new file mode 100644 index 0000000000..11536913e9 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/NOTICE @@ -0,0 +1,23 @@ +Apache-2.0. DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. + +The device is SYLforge's: the stationary-lifetime construction of #7616 and #7620 (a register drops a middle stop +while a partner inserts one, the dropped work rides a loan through a carrier, and stationary owned snapshots carry +the rest). #7620 is SYLforge's composition of the 167-CLEAR stationary primal/dual bypasses on #7616 (candidate work +with Claude Opus 5.5 and Antigravity, validation with OpenAI assistance) and builds on Ali Romman's #7605, our #7611 +stay drops, Clear1nsight's #7612 contact rechart, Hao Yan/FanPUYen's #7604, SYLforge's #7603, ToikY's #7546/#7544 +and Rohan Arun's signed #7113 caller. The base word is our #7621 (courier + helper re-stop on #7620), which builds +on our #7615 and #7617 (courier lineage; #7617 supplies the control word) and on the crowd's courier pattern +(Clear1nsight's #7244/#7274/#7314/#7334/#7481, rohanarun's #7480 and #7277, CubeJerry's #7577, petalas's #7326). + +Vendored data (unmodified, data only): data/base-7621-word.json.gz and data/invoice-7621.json are our #7621's +data/word.json.gz and data/invoice-new.json; data/affected-pairs-7620.json is the 837-pair census pinned in #7620's +native runner; engine/data/word-7620.json.gz is #7620's word and engine/data/word-7617.json.gz is #7617's word +(controls only); engine/data/7113_CALLER.json is Rohan Arun's signed #7113 caller (used to price kappa). + +engine/ is our own code except three helpers written earlier in this project and vendored with only their path lines +changed: pricekit.py (the exact pricer), fastx.py (randomised-exact replay) and jl.py (kappa under the #7113 +caller). No other author's code was run; their words are read as data. check.py adapts the replay, census, +crossing and column-test sections of our #7621 package; its column tests are written from the description of +the affected-pair tests in #7620's README and native runner (512 forward and 512 adjoint Gaussian columns, signed +Pauli action, absolute phase, wrong-phase rejection), following the repository's frozen interpreter and compiler +documented in main. diff --git a/research/p104-loan-snapshot-engine-on7621/README.md b/research/p104-loan-snapshot-engine-on7621/README.md new file mode 100644 index 0000000000..60ba333f91 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/README.md @@ -0,0 +1,127 @@ +# CONDITIONAL κ = 0.0036624027915 — 14 loan + stationary-snapshot instances on #7621, with the engine that finds them (premises: exactly #7620's, over 180 CLEARs) + +**κ = 0.00366240279155398…, ×1.00025679 (+0.025679%) over our +[#7621](https://github.com/CrocSwap/integer-mult-bounds/pull/7621)** (κ 0.0036614625547). The official +`evaluate_recursive_power` bracket (256 bits) is [0.0036624027915, 0.0036624027916]; the claim 0.0036624027915 is +below the exact crossing 0.00366240279155398. + +This PR ships the word **and the engine that produced it** (`engine/`), so anyone can run these incremental +passes on any P104 native word. The device is [SYLforge](https://github.com/CrocSwap/integer-mult-bounds/pull/7620)'s +(#7616, #7620); the engine finds, solves and exactly checks instances of it automatically. + +> **Premises (inherited; one reading widens).** Conditional on exactly #7620's premises: its five framework +> assumptions (`native_all_columns`, `owned_erasure`, `compiler`, `precision_tape`, `analytic_recovery`) and its +> fixed allocation of 17·68 = 1,156 owned banks. Its CLEAR reading now covers **180 CLEARs** (74 original + 106 +> stationary), not 167: the 13 new CLEARs belong to 13 new stationary owned lifetimes of the commonest shape among #7620's own +> (INIT, one write, one read, dirty CLEAR, no MOVE; 82 of #7620's lifetimes have it), each with an exact CLEAR certificate +> under the rule of #7620's `verify_certificates` (data/clear-certificates.json). The word's rows, end charts, 47 owned MOVEs, +> peak 50, 68 labels, all 1,018 old owned observations and the old 167 CLEAR contents are #7621's. If #7620's premises +> are refused, this word is void with #7620 and #7621. + +## Run it yourself + +```sh +python3 engine/run_on.py WORD.json.gz [OUT_PREFIX] # census -> greedy -> re-census, about 1 minute, one process, under 1 GB +python3 engine/run_on.py data/base-7621-word.json.gz /tmp/out # reproduces this PR: /tmp/out_14.json.gz == data/word.json.gz +python3 -I -B check.py engine # the same, as a test (JSON-equal word and equal instance log) +python3 engine/control.py # controls on #7617 with #7620's instances as data +``` +Python 3.12 and the standard library only; `pricekit.py`, `fastx.py`, `jl.py` and the #7113 caller are vendored in +`engine/`, and every path is relative to the script. Full notes are in [engine/ENGINE.md](engine/ENGINE.md). + +**The move.** A register r drops an interior stop X from its path A→X→B. A partner p takes one extra stop: on side A +(all 14 here) p is the reader of r at X and inserts A before X. The work r would have done at X is carried by (i) a +loan through a carrier c, a public register whose path runs A→X→B consecutively, (ii) stationary owned snapshots +(INIT at X, one write, one read, dirty CLEAR) when r's X-writes must reach p, and (iii) a signed role swap between r and p, with +a settle at B. Natively each instance swaps one 3|1 hop merge for one 1+2 split. + +**How the engine works.** `lb.py` indexes a word (register stays, ops, owned live intervals, projected-state checkpoints) +and solves a candidate exactly: the template coefficients that are unknown enter linearly, so a windowed replay (mod 2^61−1, 4-dim random +projection) is affine in them, is solved, rationalised and re-verified, and must reproduce the window-end state, every pre-existing +owned observation and every CLEAR content. `tpl.py` holds the templates and the legality-keeping slides. `run7621.py` takes an exact census of +both shapes ordered by the convexity gain of x ln x, tries instances greedily, accepts one only if `check.full` passes, the owned +peak stays ≤ 50 and the PRICEKIT κ rises, then re-censuses until a pass lands nothing. `control.py` rebuilds #7620's own instances +on #7617 (read as data): 39 of 44 shape instances rebuilt, 258@829 and 464@526 identical to SYLforge's (same carriers, same coefficients). + +**Known blockers on this word (85-place re-census after the 14 landed).** + +| reason | places | meaning | +|---|---|---| +| no solution | 20 | contamination reaches registers that never meet r at B | +| no free label (peak 50 binds) | 17 | owned peak 50 is full over the snapshot interval | +| no carrier | 17 | no register runs A→X→B | +| A window | 15 | the carrier leaves A before r or p arrive | +| slides | 10 | a slide partner is touched in between | +| write after read / read before p at X | 2 + 2 | side-A write-after-read; r read before p arrives at X | + +**Next levers.** A second carrier, so a contaminated register can be settled where it meets a register holding ρ rather than r; +shared snapshots or a reorder of busy owned lifetimes to free peak-50 events; carriers that are not consecutive with in-transit +settles; census shapes with several sources or readers. + +## The edits + +73 deletions and 154 insertions on #7621's word (77,196 → 77,277 events; ADD +55, INIT +13, CLEAR +13; +[data/edits.json](data/edits.json), instances in [data/instances.json](data/instances.json)). Fourteen instances, all side A: + +| r | X | p | carrier c | +|---|---|---|---| +| 382 | 438 | 553 | 1144 | +| 422 | 492 | 577 | 1182 | +| 392 | 455 | 562 | 1154 | +| 391 | 451 | 558 | 1153 | +| 390 | 453 | 560 | 1152 | +| 386 | 452 | 559 | 1148 | +| 374 | 433 | 548 | 1136 | +| 366 | 420 | 541 | 1128 | +| 364 | 419 | 540 | 1126 | +| 360 | 417 | 538 | 1122 | +| 359 | 413 | 534 | 1121 | +| 358 | 415 | 536 | 1120 | +| 356 | 416 | 537 | 1118 | +| 348 | 399 | 528 | 1110 | + +In each, r's route loses X and p's route gains the chart before X; exactly these 28 registers change chains. Thirteen instances add a +snapshot (348@399 reroutes r's X-writes straight into p). 24 new MOVE pairs appear, all strictly nested; 14 leave the MOVE set. + +## Price + +Native paid delta per call {1:+1, 2:+13, 3:−28, 4:+13, 5:+1}; over 17 calls {1:+17, 2:+221, 3:−476, 4:+221, 5:+17}. +Paid positions 110,145 and rank surplus S = 1,424 are unchanged. Official tools (`price.py`): `compare_recursive_invoices` against #7621 gives +`uniformly_better`, strictly negative for r^(1−C); `evaluate_recursive_power` (256 bits) gives `new_cost_lower` at C = 0.0036624027915, 0.0036614625547 (#7621) +and 0.0036611026181 (#7620), `new_cost_higher` at C = 0.0036624027916. + +Ordinary bill (17 calls, #7000 formula) 785,022,672 → 785,921,632 < 10^9; precision bit bounds 303 / 299 / 299 unchanged; common grid +2^25·1125 with 40 numerator bits unchanged; non-dyadic literal events 5,903 → 5,903. An independent read-only audit found the word sound relative to +#7620/#7621 (exact replay; official #7000 native engine with bad_inputs 0; strictification gate passed, γ 0.941467, δ 0.908157; +22/22 rejecting controls). + +## Checks + +```sh +python3.12 -B scripts/research_package.py --package research/p104-loan-snapshot-engine-on7621 +``` +`VERIFY.json` registers six commands: `core` (words, edit list, exact replay of both, owned data, observations, certificates, structure, invoice, κ), +`controls` (22 rejecting controls, 2 accepting), `columns1` and `columns2` (the affected-pair column tests), `price.py` and `engine` (the engine rerun on #7621's word). + +**Affected pairs.** #7621's census is 846 pairs (the 837 pinned by #7620's runner, minus (719,127), plus 10). This word changes 28 more routes; the census +becomes **964 pairs** (+118: the old and new route pairs of the registers not already counted; none dropped; every new MOVE pair is inside). +`check.py` runs 512 forward and 512 adjoint Gaussian columns, signed Pauli action, absolute phase, exact ascent rank and a wrong-phase rejection +on all 964 (964 × 1,024 = 987,136 columns, in two halves of 482 pairs). Census files: `data/affected-pairs-7621.json`, `data/affected-pairs-loan.json`. + +## Conditional premises and what is not rerun + +The claim carries exactly #7620's premises (above), now over 180 CLEARs, and the signed #7113 caller and BIT premise of the base. This package +does not rerun #7620's full native/signed 17-stage application, catalogue, outer predicates or precision/tape counts (header, rows, grid and owned data +are unchanged); the audit ran the official #7000 native engines on the word. No upstream maintainer has reviewed the package. + +## Credit + +The device is SYLforge's ([#7616](https://github.com/CrocSwap/integer-mult-bounds/pull/7616), [#7620](https://github.com/CrocSwap/integer-mult-bounds/pull/7620)); +the base is our [#7621](https://github.com/CrocSwap/integer-mult-bounds/pull/7621), on [#7617](https://github.com/CrocSwap/integer-mult-bounds/pull/7617) +and [#7615](https://github.com/CrocSwap/integer-mult-bounds/pull/7615) (courier lineage). Signed caller: Rohan Arun's #7113. See [NOTICE](NOTICE). +DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. Apache-2.0. + +## Open next questions + +**The premises.** A ruling on #7620's inherited assumptions and its CLEAR reading (`owned_erasure`, clear-form-gap Q1), now with 180 CLEARs, decides +#7620, #7621 and this word together. See [TASK.json](TASK.json). diff --git a/research/p104-loan-snapshot-engine-on7621/SOURCES.json b/research/p104-loan-snapshot-engine-on7621/SOURCES.json new file mode 100644 index 0000000000..c484f02b5a --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/SOURCES.json @@ -0,0 +1,30 @@ +{ + "schema_version": 1, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "75d77b328ea02ded535e7f23aee5b1e55d0bb1a8", + "sha256": "154c3d30455d778fd3343b6b4851f0fa4665f0f7625b0adb3675e2bafbbf6bf1" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec", + "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81" + }, + { + "path": "scripts/compare_recursive_invoices.py", + "usage": "input", + "git_blob": "fda2a852e9937e42fe40df0707694465ca7c1b48", + "sha256": "3930c22c11da7bc6042cfeb6e459ecb06e4478430037bee3469d034353855fc6" + }, + { + "path": "scripts/evaluate_recursive_power.py", + "usage": "input", + "git_blob": "ca50f8021fc3238a58fd420383f4619a8d008ace", + "sha256": "52b20f41fd0bb55fa88dccd6015af2ce358917817b9506720f3e60ce681cc3ad" + } + ] +} diff --git a/research/p104-loan-snapshot-engine-on7621/TASK.json b/research/p104-loan-snapshot-engine-on7621/TASK.json new file mode 100644 index 0000000000..52ea6733d2 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/TASK.json @@ -0,0 +1,117 @@ +{ + "schema_version": 1, + "task_id": "p104-side-correction", + "priority_revision": 8, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "original_card": { + "id": "p104-side-correction", + "question": "Can joint helper bases and a legal causal split improve the complete P104 productive/paid invoice?", + "starting_points": [ + "docs/research/current-priorities.md", + "docs/research/research-family-status.md", + "certificates/selected-result.json", + "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "research/p104-helper-subspace-splits/README.md", + "research/dyadic-helper-channel-capacity/README.md" + ], + "methods": [ + "Construct and independently check one source-bound instance of the stated first experiment.", + "Seek an exact counterexample or target-specific complete-budget obstruction to that same proposed change." + ], + "acceptance_gate": "An exact arbitrary-input identity, source/target frame availability and restoration at the chosen cut, plus the complete channel+correction+idle/hosting bill or an explicit favorable bound. The native channel alone is not a supplier. A scoped local result may be retained without a complete supplier. Expanding it into further construction rounds requires the family-level budget and obligation gate in research-family-status.md; an optimistic passing relaxation alone is not acceptance.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_selected_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Read revision 8, the selected review and live exact-source handoffs. Then follow this card's bounded first_experiment.", + "scope": "Source-bound orientation only, not a supplier replay or theorem proof." + }, + "first_experiment": "Start with the selected #5599 emitted word and review pending #5718 distinct input/output bases plus #5717/#5719 coverage before claiming work. Pick one uncovered source word, helper group and cut region. Prove the arbitrary-input relation and legal entry/exit corrections, then compare full target and paid profiles. Reject if completion, owned lifetimes or consumer costs repay the gain. A static pairing rank alone is not a physical split.", + "bottleneck_link": "Judge the actual current supplier contract and complete signed target/paid bill; frozen-profile ceilings require an applicability proof.", + "allowed_variations": [ + "One exact source head, helper/cut region or quantified proposition and changed hypothesis; check live handoffs first. Different seeds are not different scopes." + ], + "scope_change_triggers": [ + "Changing source head, helper/cut domain, operation alphabet or caller boundary without recording the new scope.", + "Treating an archived proposal or local numerical pass as complete supplier admission." + ], + "overlap_terms": [ + "P104", + "helper basis", + "helper split", + "5718", + "dyadic capacity" + ], + "existing_checks": [ + { + "scope": "PR #4133: literal-slot defect", + "evidence": "research/p104-literal-slot-defect/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "88666f1dee93fbd85b31fd5d6f91a219e111b5f4", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4136: uniform commutation gap", + "evidence": "research/p104-uniform-commutation-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "281ce601a8be9db470de630759179c23e1dfaa88", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4140: uniform encoding gap", + "evidence": "research/p104-uniform-encoding-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "de25c6525aa866962309df5d107b32d55e6e5628", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4153: source-chart shortcut exhausts its optimistic allowance", + "evidence": "research/p104-source-chart-shortcut/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "9123241fea53df77e51ffe30d9aca1625d4bf3a8", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4154: fiber-tag seam mismatch", + "evidence": "research/p104-fiber-tag-seam/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "4ffcdaee5d6255a9cea1fe53a5424435068089d0", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "Selected PR5599: complete helper-subspace construction and independent caller replay; five framework assumptions remain.", + "evidence": "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "remaining": "The next experiment must identify uncovered source-specific work. Successor archive intake does not transfer this admission.", + "status": "reviewed", + "source_commit": "09dd0965711ca7eeb169057a420abab1c9052137", + "selected_record_sha256": "2a0220e866417774606e0f991e41b9c1d26b1f43b97857238cf478dbc7d330b5" + } + ] + }, + "scope_status": "refined", + "actual_scope": "Exact #7621 word (our courier + helper re-stop on SYLforge's #7620 stationary primal/dual composition): 14 instances of SYLforge's loan + stationary-snapshot device (#7616/#7620), all side A. In each, register r drops an interior stop X and a partner p gains the chart A before X; the work r would have done at X rides a loan through a carrier register that passes A, X, B, and p picks up r's X-writes directly or from a stationary owned snapshot (INIT, one write, one read, dirty CLEAR, no MOVE), with a signed role swap between r and p and a settle at B. 13 new stationary owned lifetimes, one per snapshot (instance 348@399 needs none). Rows, end charts, owned data (47 owned MOVEs, peak 50, 68 labels, 1,018 old owned observations) and all 167 old CLEAR contents are #7621's; 24 new strictly nested chart pairs, 14 leave. Packaged with the engine that produced it (engine/): rerunning engine/run_on.py on #7621's word reproduces this word. Conditional on exactly #7620's premises (its five inherited assumptions, its fixed 17*68 bank allocation) with its CLEAR reading now covering 180 CLEARs (74 + 106 stationary), not 167.", + "scope_change_reason": "Applies SYLforge's loan + stationary-snapshot device (#7616, #7620) to #7621's word by an exact builder that finds the instances (census), solves the template coefficients exactly, and accepts a hit only after exact replay, owned-observation equality and a kappa gain; the builder is published with the word so others can run the passes themselves.", + "bottleneck_link": "Reduces the complete signed paid invoice by {1:+17, 2:+221, 3:-476, 4:+221, 5:+17} over the 17 native calls (paid positions unchanged at 110145, rank-weighted S 1424 unchanged) without changing the productive profile.", + "budget_minutes": 120, + "stop_rule": "One literal word with exact arbitrary-input row equality, legality, identical old CLEAR content, certificates for the new CLEARs, all affected-pair column tests (964 pairs), complete invoice, a certified crossing, and an engine rerun that reproduces the word.", + "outcome": "completed", + "evidence": "README.md, engine/ (ENGINE.md, run_on.py), data/edits.json, data/instances.json, check.py, price.py, VERIFY.json.", + "author": "DreamingOfClouds", + "ai_assistance": "Claude Opus 5.5 (Anthropic), via Claude Code.", + "open_questions": [ + { + "question": "Do #7620's five inherited assumptions (native_all_columns, owned_erasure, compiler, precision_tape, analytic_recovery), its reading of the CLEARs (now 180: 74 original + 106 stationary) and its fixed 17*68 bank allocation hold?", + "why_it_matters": "The claimed kappa holds only if #7620's does; this word adds 13 CLEARs of the same kind as #7620's 82 single-write single-read stationary lifetimes, and no new premise.", + "first_step": "Run check.py core and read the CLEAR lines.", + "blocked_by": "A ruling on #7620's premises (owned_erasure, clear-form-gap Q1)." + } + ] +} diff --git a/research/p104-loan-snapshot-engine-on7621/VERIFY.json b/research/p104-loan-snapshot-engine-on7621/VERIFY.json new file mode 100644 index 0000000000..51506a7f1e --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/VERIFY.json @@ -0,0 +1,77 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "scripts/compare_recursive_invoices.py", + "scripts/evaluate_recursive_power.py" + ], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py", + "core" + ], + "timeout_seconds": 200, + "checks": "Vendored #7621 word bytes and new word hash; word = #7621 + edit list (73 deletions, 154 insertions); exact arbitrary-input replay and legality of both words; header identical; all 1,569 rows and end charts equal #7621's; helpers identity, owned rows empty; owned MOVEs (47), peak 50 and 68 labels unchanged; 167 old lifetimes kept, 13 new stationary lifetimes (INIT, one write, one read, dirty CLEAR, no MOVE); all 1,018 old owned observations identical and in order; old 167 CLEAR contents identical and in order (180 CLEARs in all); exact CLEAR certificates for the 13 new CLEARs (#7620's verify_certificates rule); 14 side-A instances, exactly 28 changed chains, 24 new strictly nested MOVE pairs; 5-smooth coefficients, non-dyadic events 5903 -> 5903, grid unchanged; paid delta {1:+1, 2:+13, 3:-28, 4:+13, 5:+1}; S 1424; invoice and kappa 0.0036624027915539 recomputed; #7621 crossing reproduced. Prints PASS_CORE_LOAN_ON_7621." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py", + "controls" + ], + "timeout_seconds": 420, + "checks": "22 negative controls (loan, settle, swap, snapshot write/pickup/CLEAR, ordering, wrong chart, stop removed, flag flip, rerouted-write variants, bare stop drop on #7621) all rejected by exact replay, row/end-chart or CLEAR-content checks; the hit and #7621 words accepted. Prints PASS_CONTROLS_LOAN." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py", + "columns1" + ], + "timeout_seconds": 120, + "checks": "Gaussian column tests (512 forward + 512 adjoint, signed Pauli action, absolute phase, exact rank, wrong-phase rejection) on the even-indexed half of the 964-pair affected census (#7621's 846 plus 118 pairs on the old or new routes of the 28 changed registers; none dropped), with rejecting controls. Prints PASS_COLUMNS_ON_7621_1." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py", + "columns2" + ], + "timeout_seconds": 120, + "checks": "Same column tests on the odd-indexed half of the 964-pair census. Prints PASS_COLUMNS_ON_7621_2." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/price.py" + ], + "timeout_seconds": 60, + "checks": "Repository tools: uniformly_better against #7621; new_cost_lower at C=0.0036624027915, 0.0036614625547 and 0.0036611026181; new_cost_higher at C=0.0036624027916. Prints PASS_PRICE." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py", + "engine" + ], + "timeout_seconds": 180, + "checks": "Engine reproduction: engine/run_on.py on #7621's word (census, greedy loan + snapshot passes, re-census) lands 14 instances; the output word is JSON-equal to data/word.json.gz and the instance log equals data/instances.json. Prints PASS_ENGINE_REPRODUCES_HIT_LOAN_14." + } + ] +} diff --git a/research/p104-loan-snapshot-engine-on7621/check.py b/research/p104-loan-snapshot-engine-on7621/check.py new file mode 100644 index 0000000000..299f9921d6 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/check.py @@ -0,0 +1,1073 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Exact replay of 14 loan + stationary-snapshot instances applied to #7621's word. + +Standard library only. Written for this package; it imports no other package and none of the other PRs' code was +run (their words, invoices and the pinned affected-pair census are vendored or copied as data). + +Modes (each is a separate VERIFY command): core, controls, columns1, columns2, engine; all runs core, controls and both +column halves (not engine). + +core + 1. The vendored #7621 word (data/base-7621-word.json.gz, sha256 ff8cc4ac...) plus data/edits.json gives data/word.json.gz + event for event (77,196 -> 77,277 events; sha256 97391721...). + 2. Both words are replayed over the rationals from arbitrary inputs, with full legality checks. Header identical; all 1,569 rows + and end charts equal #7621's; helpers end at identity, owned rows end empty; owned MOVEs (47), owned peak (50) and the 68 + owned labels unchanged. + 3. Owned data. All 167 old lifetimes survive; there are exactly 13 new lifetimes, each stationary (INIT chart = CLEAR chart, + one write of coefficient 1, one read of coefficient 1, dirty CLEAR, no MOVE). Every old owned observation (operation, + source-row and destination-row content at every owned-touching event) reappears identically and in order once the + new lifetimes' own events are removed. The old 167 CLEAR contents are identical and in order; the word has 180 CLEARs. + 4. Certificates: each of the 13 new CLEARs erases a row that is an exact rational combination of other live registers' rows + (the acceptance rule of #7620's verify_certificates: keys live, not the cleared field, exact sum), from data/clear-certificates.json. + 5. Structure: 14 instances (data/instances.json); in each, register r drops its interior stop X and a partner p gains the chart A + that precedes X on r's route, immediately before X in p's route (side A); exactly these 28 registers change chains, endpoints kept; + every new MOVE pair is strictly nested. + 6. Coefficients: 5-smooth denominators, grid unchanged, non-dyadic literal events 5,903 -> 5,903; event delta per native call + ADD +55, INIT +13, CLEAR +13; native paid delta {1:+1, 2:+13, 3:-28, 4:+13, 5:+1}; S 1,424; paid positions 110,145. + 7. Census, complete invoice (17 native calls) and kappa recomputed; #7621's crossing reproduced. +controls 22 negative controls (rejected) and 2 positives (accepted). +columns1/2 affected-pair column tests on the 964-pair census (even / odd sorted index): 512 forward + 512 adjoint exact Gaussian + columns, signed Pauli action, absolute phase, exact ascent rank, wrong-phase rejection; rejecting controls. +engine reruns engine/run_on.py on #7621's word and requires the output word and instance log to equal data/word.json.gz + and data/instances.json. +""" +import collections +import multiprocessing +import copy +import decimal +import gzip +import hashlib +import json +import sys +from fractions import Fraction as Q +from pathlib import Path + +HERE = Path(__file__).resolve().parent +DATA = HERE / 'data' +BASE_SHA256 = 'ff8cc4ac45dc89c6dfca54d1954b21edefe599c14d31b2ead7ceeafe3bed1ec8' +WORD_SHA256 = '97391721ab7d30f8dd84c8d3c29a3c3ed2df2d65ca55e7ff8329bd09877681d2' +CALLS = 17 + + +def require(cond, label): + if not cond: + raise AssertionError(label) + + +def load_gz(path): + return json.loads(gzip.decompress(path.read_bytes())) + + +def span(vectors): + s = {0} + for v in vectors: + require(type(v) is int and 0 < v < 512 and v not in s, 'frame vectors independent in F2^9') + s |= {x ^ v for x in s} + return frozenset(s) + + +def mains(W): + live = set(W['live']) + return live | {r + 112 for r in live} | set(W['hosted']['X2']) | set(W['hosted']['Y2']) + + +def apply_edits(program, E): + require(len(program) == E['base_events'], 'edit list base length') + dele = {} + for i, ev in E['delete']: + require(program[i] == ev, 'deleted event %d matches' % i) + dele[i] = ev + ins = collections.defaultdict(list) + for i, evs in E['insert_before']: + ins[i].extend(evs) + out = [] + for i, ev in enumerate(program): + out.extend(ins.get(i, [])) + if i not in dele: + out.append(ev) + out.extend(ins.get(len(program), [])) + return out + E['append'] + + +def replay(W): + """Exact replay; returns rows, end charts, MOVE chains and owned data. Raises on any illegality.""" + n, N = W['public'], W['public'] + W['owned_fields'] + sp = [span(f) for f in W['frames']] + rows = [{r: Q(1)} if r < n else {} for r in range(N)] + cur = list(W['initial']) + [None] * W['owned_fields'] + alive, chains, owned_moves = set(), collections.defaultdict(list), collections.Counter() + lifetimes, peak = collections.Counter(), 0 + init_chart = {} + stats = dict(max_den_exp=0, odd_den=1, max_num_bits=0, smooth=True) + + def note(q): + d = q.denominator + e = (d & -d).bit_length() - 1 + stats['max_den_exp'] = max(stats['max_den_exp'], e) + odd = d >> e + o = odd + for p in (3, 5): + while o % p == 0: + o //= p + if o != 1: + stats['smooth'] = False + if odd > 1 and stats['odd_den'] % odd: + stats['odd_den'] = stats['odd_den'] * odd // __import__('math').gcd(stats['odd_den'], odd) + stats['max_num_bits'] = max(stats['max_num_bits'], abs(q.numerator).bit_length()) + + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + require(type(r) is int and 0 <= r < N, '%d register index' % i) + if op == 'MOVE': + a, b, flag = e[2], e[3], e[4] + require(type(flag) is bool and flag == (r >= n), '%d MOVE ownership flag' % i) + require(cur[r] == a, '%d MOVE from the current chart' % i) + if r >= n: + require(r in alive, '%d owned MOVE on a live field' % i) + require(sp[a] < sp[b] or sp[b] < sp[a], '%d owned MOVE nested' % i) + owned_moves[(a, b)] += 1 + else: + require(sp[a] < sp[b], '%d public MOVE strictly ascends' % i) + cur[r] = b + chains[r].append((a, b)) + elif op == 'INIT': + require(r >= n and r not in alive and len(e) == 3, '%d INIT of a dead owned field' % i) + alive.add(r); rows[r] = {}; cur[r] = e[2]; init_chart[r] = e[2] + peak = max(peak, len(alive)) + elif op == 'CLEAR': + require(r >= n and r in alive and cur[r] == e[2], '%d CLEAR at the current chart' % i) + lifetimes[(r, init_chart.pop(r), e[2])] += 1 + alive.discard(r); rows[r] = {}; cur[r] = None + elif op == 'SCALE': + q = Q(e[2]) + require(q != 0, '%d SCALE by zero' % i) + require(cur[r] == e[3] and (r < n or r in alive), '%d SCALE chart' % i) + rows[r] = {k: v * q for k, v in rows[r].items()} + for v in rows[r].values(): + note(v) + elif op in ('ADD', 'COPY'): + s, q, f = e[2], Q(e[3]), e[4] + require(type(s) is int and 0 <= s < N and s != r, '%d operand' % i) + require(q != 0, '%d zero coefficient' % i) + require(cur[r] == f and cur[s] == f, '%d ADD/COPY operands at chart %d' % (i, f)) + for x in (r, s): + require(x < n or x in alive, '%d owned field live' % i) + if op == 'COPY': + require(not rows[r], '%d COPY into a zero field' % i) + R = rows[r] + for k, v in rows[s].items(): + nv = R.get(k, 0) + q * v + if nv: + R[k] = nv + note(nv) + else: + R.pop(k, None) + else: + raise AssertionError('%d unknown op %r' % (i, op)) + require(not alive, 'every owned field cleared') + require(cur[:n] == list(W['final']), 'public fields end at final charts') + return dict(rows=rows, cur=cur, chains=chains, owned_moves=owned_moves, + lifetimes=lifetimes, peak=peak, sp=sp, stats=stats) + + +def census(W, R): + """Per-native paid MOVE ranks and helper targets (the #7546 accounting).""" + dim = [len(f) for f in W['frames']] + paid = collections.Counter(abs(dim[e[3]] - dim[e[2]]) for e in W['program'] if e[0] == 'MOVE') + M, n = mains(W), W['public'] + helpers = collections.Counter(dim[W['final'][r]] - dim[W['initial'][r]] for r in range(n) if r not in M) + rises = collections.Counter(dim[e[3]] - dim[e[2]] for e in W['program'] + if e[0] == 'MOVE' and e[1] >= n and dim[e[3]] > dim[e[2]]) + hist = lambda c: {str(k): v for k, v in sorted(c.items()) if v} + return dict(events=len(W['program']), public=n, helpers=hist(helpers), paid=hist(paid), + lifetimes=sum(R['lifetimes'].values()), owned_peak=R['peak'], rises=hist(rises)) + + +def crossing(T, P, digits=40): + """Root of F(C) = sum (T_r - P_r) r^(1-C) on [0.003, 0.0045] by bisection in 80-digit decimals.""" + D = decimal.Decimal + with decimal.localcontext(decimal.Context(prec=digits + 40)) as ctx: + ranks = sorted({int(r) for r in T} | {int(r) for r in P}) + terms = [(D(T.get(str(r), 0) - P.get(str(r), 0)) * r, D(r).ln()) for r in ranks] + + def F(C): + return sum((c * (-C * l).exp() for c, l in terms), D(0)) + lo, hi = D('0.003'), D('0.0045') + require(F(lo) > 0 > F(hi), 'crossing bracketed') + for _ in range(160): + mid = (lo + hi) / 2 + if F(mid) > 0: + lo = mid + else: + hi = mid + return decimal.Context(prec=digits).create_decimal(lo) + + +def invoice_json(P, T): + return {'schema': 'integer-mult-recursive-invoice/v1', + 'contract': {'model': 'mixed-target-unnormalized-power-v1', + 'rank_semantics': 'H_r costs r^(1-C); debit surrendered productive fields', + 'ambient_rank': 145, + 'boundary': 'complete signed #7113 caller application with 17 native calls'}, + 'paid': {k: P[k] for k in sorted(P, key=int)}, + 'targets': {k: T[k] for k in sorted(T, key=int)}} + + +def clears(W): + """Row content erased at every owned CLEAR, in program order.""" + n = W['public'] + rows = [{r: Q(1)} if r < n else {} for r in range(n + W['owned_fields'])] + out = [] + for e in W['program']: + op, r = e[0], e[1] + if op == 'INIT': + rows[r] = {} + elif op == 'CLEAR': + out.append((r, tuple(sorted(rows[r].items())))) + rows[r] = {} + elif op == 'SCALE': + rows[r] = {k: v * Q(e[2]) for k, v in rows[r].items()} + elif op in ('ADD', 'COPY'): + R = rows[r] + for k, v in rows[e[2]].items(): + nv = R.get(k, 0) + Q(e[3]) * v + if nv: + R[k] = nv + else: + R.pop(k, None) + return out + + + +def literal_coeffs(program): + return [Q(e[2]) for e in program if e[0] == 'SCALE'] + [Q(e[3]) for e in program if e[0] in ('ADD', 'COPY')] + + +def nondyadic(program): + return sum(1 for q in literal_coeffs(program) if q.denominator & (q.denominator - 1)) + + + +# --------------------------------------------------------------------------- +# Column tests for every affected public chart pair (own implementation). +# A chart pair (a, b) is the operator F_a^dagger F_b of two nine-bit stabilizer +# frames (Gaussian-dyadic Clifford maps). For each pair this section (i) builds +# the circuit of F_a^dagger F_b directly from the frame definition, (ii) compiles +# it to the child recipe monomial . Fourier^r . monomial with r = dim b - dim a, +# (iii) requires the recipe and the direct circuit to agree in signed Pauli action +# and exact absolute phase, and (iv) compares all 512 forward and 512 adjoint +# Gaussian columns, then checks that a recipe with its absolute phase shifted is +# rejected. Gates: P (bit permutation), D (diagonal Z4 quadratic form), +# C (one Fourier line, (1+i)I/2 + (1-i)X/2), X (offset), phase (i^k). +# --------------------------------------------------------------------------- +NB = 9 +FULL = 1 << NB + + +def par(x): + return x.bit_count() & 1 + + +def rank2(rows): + piv = {} + for x in rows: + while x: + b = x.bit_length() - 1 + if b not in piv: + piv[b] = x + break + x ^= piv[b] + return len(piv) + + +def tr(rows, n=NB): + return tuple(sum(((rows[j] >> i) & 1) << j for j in range(n)) for i in range(n)) + + +def mvec(rows, x): + return sum(par(r & x) << i for i, r in enumerate(rows)) + + +def minv(rows, n=NB): + a = [r | (1 << (n + i)) for i, r in enumerate(rows)] + for j in range(n): + k = next(i for i in range(j, n) if a[i] >> j & 1) + a[j], a[k] = a[k], a[j] + for i in range(n): + if i != j and a[i] >> j & 1: + a[i] ^= a[j] + require([r & ((1 << n) - 1) for r in a] == [1 << i for i in range(n)], 'binary inverse') + return tuple(r >> n for r in a) + + +def extend(vecs, n=NB): + out = [] + for x in list(vecs) + [1 << i for i in range(n)]: + if rank2(out + [x]) > len(out): + out.append(x) + require(len(out) == n, 'basis extension') + return tuple(out) + + +def qform(Qr, al, x): + return (sum(al[i] for i in range(NB) if x >> i & 1) + + 2 * sum(par((Qr[i] >> (i + 1)) & (x >> (i + 1))) for i in range(NB) if x >> i & 1)) % 4 + + +def dagger(circ): + out = [] + for g in reversed(circ): + t = g[0] + if t == 'P': + out.append(('P', minv(g[1]))) + elif t == 'D': + out.append(('D', g[1], tuple(-a % 4 for a in g[2]))) + elif t == 'C': + out.append(('C', g[1], -g[2])) + elif t == 'phase': + out.append(('phase', -g[1] % 4)) + else: + out.append(g) + return tuple(out) + + +def fourier(r): + return tuple(('C', 1 << i, 1) for i in range(r)) + + +def frame_circuit(vecs): + r = len(vecs) + if r == 0: + return () + if r == NB: + return fourier(NB) + G = tr(extend(vecs)) + gi = tr(minv(G)) + Qr = tuple((1 << i) ^ sum(par(a & b) << j for j, b in enumerate(gi)) for i, a in enumerate(gi)) + chart = (('P', G), ('D', Qr, tuple((row >> i) & 1 for i, row in enumerate(Qr)))) + return dagger(chart) + fourier(r) + chart + + +def tableau(circ): + rows = [(1 << i, 0, 0) for i in range(NB)] + [(0, 1 << i, 0) for i in range(NB)] + for g in circ: + t = g[0] + out = [] + it = tr(minv(g[1])) if t == 'P' else None + for x, z, p in rows: + if t == 'P': + x, z = mvec(g[1], x), mvec(it, z) + elif t == 'X': + p += 2 * par(z & g[1]) + elif t == 'C': + if par(z & g[1]): + x ^= g[1] + p += 3 * g[2] + elif t == 'D': + p += sum(g[2][i] for i in range(NB) if x >> i & 1) + p += 2 * sum(par((g[1][i] >> (i + 1)) & (x >> (i + 1))) for i in range(NB) if x >> i & 1) + z ^= mvec(g[1], x) + else: + require(t == 'phase', 'gate type') + out.append((x, z, p % 4)) + rows = out + return tuple(rows) + + +_ACT = {} + + +def dact(g): + if g not in _ACT: + _ACT[g] = tuple(qform(g[1], g[2], x) for x in range(FULL)) + return _ACT[g] + + +def rot(pr, k): + a, b = pr + return ((a, b), (-b, a), (-a, -b), (b, -a))[k % 4] + + +def column(circ, basis=0): + """Exact Gaussian-dyadic column: ({address: (re, im)}, denominator exponent), value = num / 2^(den/2) ... .""" + val, den = {basis: (1, 0)}, 0 + for g in circ: + t = g[0] + res = {} + if t == 'C': + d, s = g[1], g[2] + for ad, (a, b) in val.items(): + for tg, pr in ((ad, (a - s * b, b + s * a)), (ad ^ d, (a + s * b, b - s * a))): + o = res.get(tg, (0, 0)) + res[tg] = (o[0] + pr[0], o[1] + pr[1]) + den += 1 + else: + for ad, pr in val.items(): + if t == 'P': + tg = mvec(g[1], ad) + elif t == 'X': + tg = ad ^ g[1] + elif t == 'D': + tg = ad + pr = rot(pr, dact(g)[ad]) + else: + require(t == 'phase', 'gate type') + tg = ad + pr = rot(pr, g[1]) + require(tg not in res, 'monomial map injective') + res[tg] = pr + val = {ad: pr for ad, pr in res.items() if pr != (0, 0)} + while den and all((a | b) % 2 == 0 for a, b in val.values()): + den -= 1 + val = {ad: (a // 2, b // 2) for ad, (a, b) in val.items()} + return val, den + + +def monomial_circuit(rec): + M, Qr, al = tuple(rec['matrix_rows']), tuple(rec['quadratic_rows']), tuple(rec['linear_Z4']) + require(tr(Qr) == Qr and all(al[i] % 2 == (Qr[i] >> i) & 1 for i in range(NB)), 'diagonal form') + minv(M) + return (('D', Qr, al), ('P', M), ('X', rec['offset']), ('phase', rec['phase'])) + + +def recipe_circuit(rec): + if rec['rank'] == 0: + return monomial_circuit(rec['left']) + return monomial_circuit(rec['right']) + fourier(rec['rank']) + monomial_circuit(rec['left']) + + +# compiler: symplectic tableau algebra used to produce the recipe +def pmul(a, b): + x, z, p = a + y, w, q = b + return x ^ y, z ^ w, (p + q + 2 * par(z & y)) % 4 + + +def tapply(T, pa): + x, z, ph = pa + out = (0, 0, ph) + for i in range(NB): + if x >> i & 1: + out = pmul(out, T[i]) + for i in range(NB): + if z >> i & 1: + out = pmul(out, T[NB + i]) + return out + + +def tcompose(A, B): + return tuple(tapply(A, p) for p in B) + + +def tidentity(): + return tuple([(1 << i, 0, 0) for i in range(NB)] + [(0, 1 << i, 0) for i in range(NB)]) + + +def tinverse(T): + cols = [x | (z << NB) for x, z, p in T] + ci = tr(minv(tr(cols, 2 * NB), 2 * NB), 2 * NB) + out = [] + for j, w in enumerate(ci): + x, z = w & (FULL - 1), w >> NB + u, v, p = tapply(T, (x, z, 0)) + require((u, v) == tidentity()[j][:2], 'tableau inverse') + out.append((x, z, -p % 4)) + return tuple(out) + + +class Op: + def __init__(s, tab, circ): + s.tab, s.circ = tab, tuple(circ) + + def then(s, o): + return Op(tcompose(o.tab, s.tab), s.circ + o.circ) + + def dag(s): + return Op(tinverse(s.tab), dagger(s.circ)) + + +def op_perm(G): + cols, gi = tr(G), minv(G) + return Op(tuple([(a, 0, 0) for a in cols] + [(0, a, 0) for a in gi]), [('P', tuple(G))]) + + +def op_diag(Qr, al=None): + Qr = tuple(Qr) + al = tuple(al) if al is not None else tuple((Qr[i] >> i) & 1 for i in range(NB)) + return Op(tuple([(1 << i, Qr[i], al[i]) for i in range(NB)] + [(0, 1 << i, 0) for i in range(NB)]), [('D', Qr, al)]) + + +def op_c(q, s=1): + return Op(tuple([(1 << i, 0, 0) for i in range(NB)] + + [(q if q >> i & 1 else 0, 1 << i, (3 * s) % 4 if q >> i & 1 else 0) for i in range(NB)]), [('C', q, s)]) + + +def op_x(b): + return Op(tuple([(1 << i, 0, 0) for i in range(NB)] + [(0, 1 << i, 2 * ((b >> i) & 1)) for i in range(NB)]), [('X', b)]) + + +def op_id(): + return Op(tidentity(), []) + + +def op_four(axes): + o = op_id() + for i in axes: + o = o.then(op_c(1 << i)) + return o + + +def op_had(axes): + S = op_diag(tuple(1 << i if i in axes else 0 for i in range(NB))) + return S.then(op_four(axes)).then(S) + + +def frame_op(vecs): + r = len(vecs) + if r == 0: + return op_id() + if r == NB: + return op_four(range(NB)) + G = tr(extend(vecs)) + gi = tr(minv(G)) + Qr = tuple((1 << i) ^ sum(par(a & b) << j for j, b in enumerate(gi)) for i, a in enumerate(gi)) + K = op_perm(G).then(op_diag(Qr)) + return K.dag().then(op_four(range(r))).then(K) + + +def split_monomial(op): + T = op.tab + require(all(x == 0 and p % 2 == 0 for x, z, p in T[NB:]), 'monomial tableau') + G = tr([x for x, z, p in T[:NB]]) + b = mvec(G, sum((p // 2) << i for i, (x, z, p) in enumerate(T[NB:]))) + cols = [mvec(tr(G), z) for x, z, p in T[:NB]] + Qr = tr(cols) + al = [(p - 2 * par(b & z)) % 4 for x, z, p in T[:NB]] + out = op_diag(Qr, al).then(op_perm(G)).then(op_x(b)) + require(out.tab == T, 'monomial rebuilt') + val, den = column(op.circ) + require(den == 0 and all(v == (0, 0) for a, v in val.items() if a != b), 'monomial column') + ph = [(1, 0), (0, 1), (-1, 0), (0, -1)].index(val[b]) + out = out.then(Op(tidentity(), [('phase', ph)])) + return out, dict(matrix_rows=G, offset=b, quadratic_rows=Qr, linear_Z4=al, phase=ph) + + +def lower(op): + Bc = [x for x, z, p in op.tab[NB:]] + r = rank2(Bc) + if not r: + mon, rec = split_monomial(op) + return dict(rank=0, left=rec, right=None) + G = tr(extend(Bc)) + A = minv(G) + PA = op_perm(A) + part = op.then(PA) + Bc = [x for x, z, p in part.tab[NB:]] + Dc = [z for x, z, p in part.tab[NB:]] + picks, sel = [], [] + for j, x in enumerate(Bc): + if rank2(sel + [x]) > len(sel): + picks.append(j) + sel.append(x) + require(len(picks) == r and all(x >> r == 0 for x in Bc), 'rank selection') + Br = tr(sel, r) + Bi = minv(Br, r) + Dr = tr([Dc[j] & ((1 << r) - 1) for j in picks], r) + Qs = tuple(sum(par(row & col) << j for j, col in enumerate(tr(Bi, r))) for row in Dr) + DQ = op_diag(Qs + (0,) * (NB - r)) + V = part.then(DQ).then(op_had(range(r))) + require(all(x == 0 for x, z, p in V.tab[NB:]), 'V monomial') + Vm, _ = split_monomial(V) + S = op_diag(tuple(1 << i if i < r else 0 for i in range(NB))) + left = S.dag().then(DQ.dag()).then(PA.dag()) + right = Vm.then(S.dag()).then(op_x((1 << r) - 1)) + lm, lrec = split_monomial(left) + rm, rrec = split_monomial(right) + comp = rm.then(op_four(range(r))).then(lm) + require(comp.tab == op.tab and column(comp.circ) == column(op.circ), 'recipe reproduces the pair operator') + return dict(rank=r, left=lrec, right=rrec) + + +def pair_columns(frames, a, b, wrong_phase=True): + """Full column test of the chart pair (a, b); returns its rank.""" + U, V = frames[a], frames[b] + rec = lower(frame_op(U).dag().then(frame_op(V))) + require(rec['rank'] == len(V) - len(U), 'exact public ascent rank of pair %s' % ((a, b),)) + expected = dagger(frame_circuit(U)) + frame_circuit(V) + actual = recipe_circuit(rec) + require(tableau(actual) == tableau(expected), 'signed Pauli action of pair %s' % ((a, b),)) + for x, y in ((actual, expected), (dagger(actual), dagger(expected))): + for c in range(FULL): + require(column(x, c) == column(y, c), 'Gaussian column %d of pair %s' % (c, (a, b))) + if wrong_phase: + bad = actual + (('phase', 1),) + require(tableau(bad) == tableau(actual) and column(bad) != column(expected), 'wrong absolute phase rejected') + return rec['rank'] + + +MODE = sys.argv[1] if len(sys.argv) > 1 else 'all' + + +def load_gz(path): + return json.loads(gzip.decompress(Path(path).read_bytes())) + + +def observations(W): + """Per owned-touching event the observation (event, source row, destination row); plus per-lifetime records.""" + n = W['public'] + rows = [{r: Q(1)} if r < n else {} for r in range(n + W['owned_fields'])] + out, lt, open_ = [], [], {} + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + if op == 'INIT': + rows[r] = {} + open_[r] = dict(init=i, chart=e[2], w=[], rd=[], mv=0, field=r) + out.append((i, r, tuple(e))) + elif op == 'CLEAR': + x = open_.pop(r) + x['clear'] = i + x['content'] = dict(rows[r]) + lt.append(x) + out.append((i, r, (tuple(e), tuple(sorted(rows[r].items()))))) + rows[r] = {} + elif op == 'MOVE': + if r >= n: + open_[r]['mv'] += 1 + out.append((i, r, (tuple(e), tuple(sorted(rows[r].items()))))) + elif op == 'SCALE': + if r >= n: + open_[r]['w'].append(('SCALE', e[2])) + out.append((i, r, (tuple(e), tuple(sorted(rows[r].items()))))) + rows[r] = {k: v * Q(e[2]) for k, v in rows[r].items()} + else: + s = e[2] + if r >= n: + open_[r]['w'].append((s, e[3])) + if s >= n: + open_[s]['rd'].append((r, e[3])) + if r >= n or s >= n: + out.append((i, r if r >= n else s, (tuple(e), tuple(sorted(rows[s].items())), tuple(sorted(rows[r].items()))))) + R = rows[r] + for k, v in rows[s].items(): + nv = R.get(k, 0) + Q(e[3]) * v + if nv: + R[k] = nv + else: + R.pop(k, None) + return out, lt + + +def new_lifetimes(ltb, ltw): + """Lifetimes of W not matched by a lifetime of B on the same (field, chart), in order.""" + old = collections.Counter((x['field'], x['chart']) for x in ltb) + used, new = collections.Counter(), [] + for x in ltw: + k = (x['field'], x['chart']) + if used[k] < old[k]: + used[k] += 1 + else: + new.append(x) + return new + + +def ordered_subsequence(small, big): + it = iter(big) + return all(any(c == d for d in it) for c in small) + + +def compare(B, RB, W): + """Everything that must hold between #7621's word B and a candidate W, except observations and certificates.""" + require(all(B[k] == W[k] for k in B if k != 'program') and B.keys() == W.keys(), 'header identical') + R = replay(W) + n = W['public'] + bad = [r for r in range(len(R['rows'])) if R['rows'][r] != RB['rows'][r]] + require(not bad, 'all rows equal #7621 (first differing %s)' % bad[:3]) + require(R['cur'] == RB['cur'], 'all end charts equal #7621') + M = mains(W) + require(all(R['rows'][r] == {r: Q(1)} for r in range(n) if r not in M), 'helpers end at identity') + require(all(not R['rows'][r] for r in range(n, len(R['rows']))), 'owned rows end empty') + require(R['owned_moves'] == RB['owned_moves'], 'owned MOVE multiset unchanged') + require(R['peak'] == RB['peak'], 'owned peak unchanged') + lb, lw = RB['lifetimes'], R['lifetimes'] + require(all(lw[k] >= v for k, v in lb.items()), 'every old lifetime kept') + new = lw - lb + require(sum(new.values()) == sum(lw.values()) - sum(lb.values()) and all(k[1] == k[2] for k in new), + 'new lifetimes are stationary (INIT chart = CLEAR chart)') + cb, cw = clears(B), clears(W) + require(ordered_subsequence(cb, cw), 'old CLEAR contents identical and in order') + return R, sum(new.values()), len(cb), len(cw) + + +def chain_routes(RX, r): + ch = RX['chains'][r] + return [ch[0][0]] + [b for a, b in ch] + + +def structure(B, RB, W, R, I): + """The 14 instances: r drops X; p gains the chart A before X (side A). Exactly these 28 registers change.""" + changed = sorted(r for r in set(R['chains']) | set(RB['chains']) if R['chains'][r] != RB['chains'][r]) + want = set() + for x in I: + r, X, p = x['r'], x['X'], x['p'] + require(x['side'] == 'A', 'instance %s is side A' % ((r, X, p),)) + ro, rn, po, pn = chain_routes(RB, r), chain_routes(R, r), chain_routes(RB, p), chain_routes(R, p) + require(ro.count(X) == 1 and 0 < ro.index(X) < len(ro) - 1, 'X is an interior stop of r') + k = ro.index(X) + require(rn == ro[:k] + ro[k + 1:], 'r %d drops X %d (route %s -> %s)' % (r, X, ro, rn)) + A = ro[k - 1] + require(X in po and po.count(X) == 1, 'p visits X') + j = po.index(X) + require(pn == po[:j] + [A] + po[j:] and A not in po, 'p %d gains chart %d immediately before X (route %s -> %s)' % (p, A, po, pn)) + want |= {r, p} + require(len(I) == 14 and len(want) == 28 and changed == sorted(want), 'exactly the 28 registers of the 14 instances change chains') + mp = lambda X: {(e[2], e[3]) for e in X['program'] if e[0] == 'MOVE'} + newp, gone = sorted(mp(W) - mp(B)), sorted(mp(B) - mp(W)) + require((len(newp), len(gone)) == (24, 14), 'MOVE pairs: 24 new, 14 leave (got %d, %d)' % (len(newp), len(gone))) + for a, b in newp: + require(R['sp'][a] < R['sp'][b], 'new pair %s strictly nested' % ((a, b),)) + return changed, newp, gone + + +def verify_certificates(W, certs, newidx): + """Exact CLEAR certificates for the new CLEARs: erased row == sum of cert coefficients times live registers' rows.""" + n = W['public'] + require(set(certs) == {str(i) for i in newidx}, 'a certificate for every new CLEAR and no other') + rows = [{r: Q(1)} if r < n else {} for r in range(n + W['owned_fields'])] + live = set(range(n)) + sizes = [] + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + if op == 'INIT': + live.add(r) + rows[r] = {} + elif op == 'CLEAR': + if str(i) in certs: + cert = {int(k): Q(v) for k, v in certs[str(i)].items()} + require(all(k in live and k != r for k in cert), 'certificate keys live and not the cleared field') + s = {} + for k, c in cert.items(): + for t, v in rows[k].items(): + s[t] = s.get(t, 0) + c * v + require({t: v for t, v in s.items() if v} == rows[r] and rows[r], 'exact nonzero certificate for CLEAR %d' % i) + sizes.append(len(cert)) + rows[r] = {} + live.discard(r) + elif op == 'SCALE': + rows[r] = {k: v * Q(e[2]) for k, v in rows[r].items()} + elif op != 'MOVE': + R = rows[r] + for k, v in rows[e[2]].items(): + nv = R.get(k, 0) + Q(e[3]) * v + if nv: + R[k] = nv + else: + R.pop(k, None) + return sorted(sizes) + + +def routes_of(X): + out = collections.defaultdict(list) + for e in X['program']: + if e[0] == 'MOVE' and e[1] < X['public']: + out[e[1]].append((e[2], e[3])) + return out + + +def affected_pairs(B, W): + """#7621's 846-pair census (the 837 pairs pinned by #7620's runner, minus (719,127), plus 10) extended to the loan word: + every old- or new-route pair of every register whose chain changed (28 registers) joins; none leaves.""" + pin = {tuple(p) for p in json.loads((DATA / 'affected-pairs-7620.json').read_text())} + require(len(pin) == 837, 'pinned census has 837 pairs') + ten = {(3, 755), (3, 757), (260, 906), (755, 757), (755, 924), (757, 923), (757, 924), (906, 127), (923, 127), (924, 127)} + p846 = {tuple(p) for p in json.loads((DATA / 'affected-pairs-7621.json').read_text())} + require(p846 == (pin - {(719, 127)}) | ten and len(p846) == 846, "#7621's census is 846 pairs: 837 - (719,127) + 10") + rb, rw = routes_of(B), routes_of(W) + changed = [r for r in range(W['public']) if rb[r] != rw[r]] + add = {q for r in changed for q in rb[r] + rw[r]} + allp = p846 | add + filep = {tuple(p) for p in json.loads((DATA / 'affected-pairs-loan.json').read_text())} + require(allp == filep and len(allp) == 964, 'loan census is 964 pairs (846 + 118)') + require(len(allp - p846) == 118 and not p846 - allp, '118 pairs added, none dropped') + mp = lambda X: {(e[2], e[3]) for e in X['program'] if e[0] == 'MOVE'} + require(mp(W) - mp(B) <= allp, 'every new MOVE pair is tested') + return p846, allp + + +MODE = sys.argv[1] if len(sys.argv) > 1 else 'all' +require(MODE in ('all', 'core', 'controls', 'columns1', 'columns2', 'engine'), 'mode is all, core, controls, columns1, columns2 or engine') + + +def _pair_job(args): + frames, a, b = args + return (a, b, pair_columns(frames, a, b)) + + +def columns_section(B, W, part): + p846, allp = affected_pairs(B, W) + frames = [tuple(f) for f in W['frames']] + ordered = sorted(allp) + mine = ordered[part::2] + print('affected pairs: 846 (#7621) + 118 = %d; half %d of 2 tests %d pairs' % (len(allp), part + 1, len(mine))) + jobs = [(frames, a, b) for a, b in mine] + with multiprocessing.Pool(min(8, multiprocessing.cpu_count())) as pool: + done = pool.map(_pair_job, jobs, chunksize=4) + require(len(done) == len(mine), 'all pairs of this half tested') + print('PASS columns half %d: %d chart pairs x (512 forward + 512 adjoint) exact Gaussian columns = %d, signed Pauli action, absolute phase,' + ' exact public ascent rank, wrong-phase rejection each' % (part + 1, len(done), len(done) * 1024)) + if part: + return + + def col_rejected(label, mutate): + try: + mutate() + except AssertionError as exc: + print('PASS control rejected: %s (%s)' % (label, str(exc)[:70])) + return + raise AssertionError('control not rejected: ' + label) + + def non_nested(): + pair_columns(frames, 260, 128) + require(not (span(frames[260]) < span(frames[128]) or span(frames[128]) < span(frames[260])), 'control pair is not nested') + col_rejected('column test rejects the non-nested pair (260,128)', non_nested) + exp_of = lambda a, b: dagger(frame_circuit(frames[a])) + frame_circuit(frames[b]) + mp = lambda X: {(e[2], e[3]) for e in X['program'] if e[0] == 'MOVE'} + newp = sorted(mp(W) - mp(B)) + for a, b in (newp[0], newp[-1]): + def tampered_recipe(a=a, b=b): + rec = lower(frame_op(frames[a]).dag().then(frame_op(frames[b]))) + rec['left']['offset'] ^= 1 + exp = exp_of(a, b) + require(tableau(recipe_circuit(rec)) == tableau(exp) and column(recipe_circuit(rec), 0) == column(exp, 0), 'recipe column 0') + col_rejected('column test rejects a (%d,%d) recipe with a shifted offset' % (a, b), tampered_recipe) + + def phase_only(a=a, b=b): + rec = lower(frame_op(frames[a]).dag().then(frame_op(frames[b]))) + require(column(recipe_circuit(rec) + (('phase', 2),), 0) == column(exp_of(a, b), 0), 'recipe column 0 phase') + col_rejected('column test rejects a (%d,%d) recipe with a global phase error' % (a, b), phase_only) + + +def load_words(): + base_raw = (DATA / 'base-7621-word.json.gz').read_bytes() + require(hashlib.sha256(base_raw).hexdigest() == BASE_SHA256, 'vendored #7621 word bytes') + word_raw = (DATA / 'word.json.gz').read_bytes() + require(hashlib.sha256(word_raw).hexdigest() == WORD_SHA256, 'new word bytes') + return json.loads(gzip.decompress(base_raw)), json.loads(gzip.decompress(word_raw)) + + +def run_controls(B, W): + RB = replay(B) + cB = clears(B) + P0 = W['program'] + res = [] + + def outcome(P, X0): + X = dict(X0) + X['program'] = P + try: + R = replay(X) + except AssertionError as ex: + return 'illegal: ' + str(ex) + if R['rows'] != RB['rows'] or R['cur'] != RB['cur']: + return 'rows/end charts differ (%d rows)' % sum(a != b for a, b in zip(R['rows'], RB['rows'])) + if not ordered_subsequence(cB, clears(X)): + return 'old CLEAR contents differ' + return None + + def test(label, P, X0=W, expect_reject=True): + o = outcome(P, X0) + require((o is not None) == expect_reject, 'control %s: %s' % (label, 'ACCEPTED' if o is None else 'REJECTED ' + o)) + if expect_reject: + res.append(1) + print('PASS control %s: %s %s' % ('rejected' if expect_reject else 'accepted', label, '(%s)' % o[:80] if o else ''), flush=True) + + def ix(ev, P=P0): + return P.index(ev) + + def mod(f, base=P0): + P = copy.deepcopy(base) + f(P) + return P + + def setv(old, new): + return lambda P: P.__setitem__(ix(old, P), new) + + def mvb(ev, before): + return lambda P: (P.remove(ev), P.insert(ix(before, P), ev)) + # instance r=392 X=455 p=562 carrier c=1154 snapshot 1525 + INI = ['INIT', 1525, 455] + SW = ['ADD', 1525, 394, '1', 455] + MP1 = ['MOVE', 562, 3, 445, False] + RD = ['ADD', 562, 392, '1', 445] + SWP = ['ADD', 392, 562, '-1', 445] + LOAN = ['ADD', 392, 1154, '1', 445] + MP2 = ['MOVE', 562, 445, 455, False] + MR = ['MOVE', 392, 445, 848, False] + PICK = ['ADD', 562, 1525, '1', 455] + CLR = ['CLEAR', 1525, 455] + SET = ['ADD', 392, 1154, '-1', 848] + # instance r=382 X=438 p=553 and r=348 X=399 p=528 (no snapshot: X-writes rerouted straight into p) + W724 = ['ADD', 553, 724, '2', 438] + W93 = ['ADD', 528, 93, '1', 399] + W714 = ['ADD', 528, 714, '-2', 399] + L348 = ['ADD', 348, 1110, '1', 390] + S348 = ['ADD', 348, 1110, '-1', 837] + for ev in (INI, SW, MP1, RD, SWP, LOAN, MP2, MR, PICK, CLR, SET, W724, W93, W714, L348, S348): + require(P0.count(ev) == 1, 'control event present once: %r' % (ev,)) + test('hit word (positive)', P0, expect_reject=False) + test('#7621 word itself (positive)', B['program'], B, expect_reject=False) + test('1 loan 392+=1154 @445 dropped', mod(lambda P: P.remove(LOAN))) + test('2 settle 392+=-1154 @848 dropped', mod(lambda P: P.remove(SET))) + test('3 loan and settle both dropped', mod(lambda P: (P.remove(LOAN), P.remove(SET)))) + test('4 loan coefficient +1 -> -1', mod(setv(LOAN, ['ADD', 392, 1154, '-1', 445]))) + test('5 swap 392+=-562 coefficient -1 -> +1', mod(setv(SWP, ['ADD', 392, 562, '1', 445]))) + test('6 snapshot pickup 562+=1525 @455 dropped', mod(lambda P: P.remove(PICK))) + test('7 snapshot write 1525+=394 @455 dropped', mod(lambda P: P.remove(SW))) + test('8 snapshot CLEAR before pickup', mod(mvb(CLR, PICK))) + test('9 snapshot never cleared', mod(lambda P: P.remove(CLR))) + test('10 snapshot write before its INIT', mod(mvb(SW, INI))) + test('11 MOVE 562 445->455 before 562+=392 @445', mod(mvb(MP2, RD))) + test('12 settle before MOVE 392 445->848', mod(mvb(SET, MR))) + test('13 snapshot pickup before 562 arrives at 455', mod(mvb(PICK, MP2))) + test('14 snapshot INIT on chart 445 (wrong chart)', mod(lambda P: (setv(INI, ['INIT', 1525, 445])(P), setv(CLR, ['CLEAR', 1525, 445])(P)))) + test('15 inserted stop dropped: MOVE 562 3->455 direct, ops at 445 removed', + mod(lambda P: (P.remove(MP2), P.remove(SWP), P.remove(LOAN), P.remove(RD), setv(MP1, ['MOVE', 562, 3, 455, False])(P)))) + test('16 MOVE 392 445->848 ownership flag True', mod(setv(MR, MR[:4] + [True]))) + test('17 p=553 X-write 2*724 @438 sent back to r=382 (old form)', mod(setv(W724, ['ADD', 382, 724, '2', 438]))) + test('18 p=528 X-write 93 @399 dropped', mod(lambda P: P.remove(W93))) + test('19 p=528 X-write -2*714 coefficient -> +2', mod(setv(W714, ['ADD', 528, 714, '2', 399]))) + test('20 348 loan dropped', mod(lambda P: P.remove(L348))) + test('21 348 settle coefficient -1 -> -2', mod(setv(S348, ['ADD', 348, 1110, '-2', 837]))) + Bp = B['program'] + test('22 bare stop drop on #7621: 392 goes 445->848 direct, no loan or snapshot', + mod(lambda P: (P.remove(['MOVE', 392, 445, 455, False]), + P.__setitem__(ix(['MOVE', 392, 455, 848, False], P), ['MOVE', 392, 445, 848, False])), Bp), B) + require(len(res) == 22, '22 negative controls rejected') + print('PASS_CONTROLS_LOAN: 22/22 negative controls rejected, 2 positives accepted') + + +def run_engine(B, W): + import subprocess + import tempfile + with tempfile.TemporaryDirectory(prefix='loan-engine-') as tmp: + out = str(Path(tmp) / 'out') + r = subprocess.run([sys.executable, '-I', '-B', str(HERE / 'engine' / 'run_on.py'), str(DATA / 'base-7621-word.json.gz'), out], + capture_output=True, text=True, timeout=1800) + require(r.returncode == 0, 'engine run exited 0: ' + r.stderr[-300:]) + print(r.stdout[-700:]) + new = load_gz(out + '_14.json.gz') + require(new == W, 'engine output on #7621 equals data/word.json.gz (JSON-equal)') + log = json.loads(Path(out + '_log.json').read_text()) + require(log == json.loads((DATA / 'instances.json').read_text()), 'engine instance log equals data/instances.json') + print('PASS_ENGINE_REPRODUCES_HIT_LOAN_14: run_on.py on #7621 lands 14 instances; word JSON-equal, log equal') + + +def main(): + B, W = load_words() + if MODE == 'engine': + run_engine(B, W) + return + if MODE in ('columns1', 'columns2'): + columns_section(B, W, 0 if MODE == 'columns1' else 1) + print('PASS_COLUMNS_ON_7621_' + MODE[-1]) + return + if MODE == 'controls': + run_controls(B, W) + return + E = json.loads((DATA / 'edits.json').read_text()) + require(apply_edits(B['program'], E) == W['program'], 'word = #7621 word + edit list') + require(len(B['program']) == 77196 and len(W['program']) == 77277, 'event census 77,196 -> 77,277') + print('PASS word = #7621 word (%d events) + %d deletions, %d insertions = %d events; word sha256 %s' + % (len(B['program']), len(E['delete']), sum(len(x[1]) for x in E['insert_before']), len(W['program']), WORD_SHA256[:12])) + RB = replay(B) + R, nnew, ncb, ncw = compare(B, RB, W) + require((sum(RB['lifetimes'].values()), sum(R['lifetimes'].values()), RB['peak'], R['peak']) == (167, 180, 50, 50), 'lifetimes 167 -> 180, peak 50') + require((ncb, ncw, nnew) == (167, 180, 13), 'CLEARs 167 -> 180 (74 + 106 stationary); 13 new') + owned_labels = {e[1] for e in W['program'] if e[0] == 'INIT'} + require(len(owned_labels) == 68 and len(owned_labels) == W['owned_fields'], '68 owned labels') + print('PASS exact replay: legal, header identical, all %d rows and end charts equal #7621, helpers identity, owned empty;' + ' owned MOVEs (%d) unchanged, peak %d, labels 68; lifetimes 167 -> 180 (13 new, all stationary)' + % (len(R['rows']), sum(R['owned_moves'].values()), R['peak'])) + ob, ltb = observations(B) + ow, ltw = observations(W) + new = new_lifetimes(ltb, ltw) + require(len(new) == 13, '13 new lifetimes') + for x in new: + require(x['init'] < x['clear'] and x['chart'] == W['program'][x['clear']][2] and x['mv'] == 0 and len(x['w']) == 1 and len(x['rd']) == 1 + and Q(x['w'][0][1]) == 1 and Q(x['rd'][0][1]) == 1 and x['content'], 'new lifetime at %d is stationary: INIT, one write, one read (coefficient 1), dirty CLEAR, no MOVE' % x['init']) + excl = {i for x in new for i, f, o in ow if f == x['field'] and x['init'] <= i <= x['clear']} + ow2 = [o for i, f, o in ow if i not in excl] + require(ow2 == [o for i, f, o in ob], 'every old owned observation reappears identically and in order') + print('PASS owned observations: all %d old observations identical and in order once the %d events of the 13 new lifetimes are removed' + % (len(ob), len(excl))) + certs = json.loads((DATA / 'clear-certificates.json').read_text())['clear_event_index'] + sizes = verify_certificates(W, certs, [x['clear'] for x in new]) + print('PASS CLEAR certificates: the 13 new CLEARs each erase an exact combination of %d..%d other live registers (#7620 verify_certificates rule)' % (sizes[0], sizes[-1])) + nz = sum(1 for _, c in clears(W) if c) + print('PASS CLEAR reading: %d CLEARs in all (74 + 106 stationary: 167 old identical to #7621 + 13 new), %d with nonzero content; premises = #7620\'s, now over 180 CLEARs' % (ncw, nz)) + + I = json.loads((DATA / 'instances.json').read_text()) + changed, newp, gone = structure(B, RB, W, R, I) + dim = [len(f) for f in W['frames']] + print('PASS structure: 14 side-A instances; r drops X, p gains the chart before X; exactly the %d registers change chains; %d new MOVE pairs, all strictly nested, %d leave' + % (len(changed), len(newp), len(gone))) + ops = lambda P: collections.Counter(e[0] for e in P) + d = ops(W['program']) + d.subtract(ops(B['program'])) + require({k: v for k, v in d.items() if v} == {'ADD': 55, 'INIT': 13, 'CLEAR': 13}, 'event delta ADD +55, INIT +13, CLEAR +13') + print('PASS event delta', {k: v for k, v in sorted(d.items()) if v}) + sb, sw = RB['stats'], R['stats'] + require(sw['smooth'] and sb['smooth'], 'replayed coefficients have 5-smooth denominators') + for q in literal_coeffs(W['program']): + o = q.denominator + while o % 2 == 0: o //= 2 + while o % 3 == 0: o //= 3 + while o % 5 == 0: o //= 5 + require(o == 1, 'literal coefficient has a 5-smooth denominator') + nb, nw = nondyadic(B['program']), nondyadic(W['program']) + require((nb, nw) == (5903, 5903), 'non-dyadic literal events 5903 -> 5903') + require(sw['max_den_exp'] <= sb['max_den_exp'] and sw['odd_den'] == sb['odd_den'] and sw['max_num_bits'] <= sb['max_num_bits'], 'grid within #7621') + print('PASS coefficients: 5-smooth; non-dyadic literal events %d -> %d; grid equal to #7621 (2-adic exponent %d, odd denominator lcm %d, numerator bits %d)' + % (nb, nw, sw['max_den_exp'], sw['odd_den'], sw['max_num_bits'])) + cb, cw = census(B, RB), census(W, R) + require(cw['helpers'] == cb['helpers'] and cw['rises'] == cb['rises'], 'helper targets and owned rises unchanged') + dpaid = collections.Counter({int(k): v for k, v in cw['paid'].items()}) + dpaid.subtract({int(k): v for k, v in cb['paid'].items()}) + dpaid = {k: v for k, v in sorted(dpaid.items()) if v} + require(dpaid == {1: 1, 2: 13, 3: -28, 4: 13, 5: 1}, 'native paid delta {1:+1, 2:+13, 3:-28, 4:+13, 5:+1}') + print('PASS native paid delta per call', dpaid) + inv0 = json.loads((DATA / 'invoice-7621.json').read_text()) + P0 = {k: inv0['paid'][k] for k in sorted(inv0['paid'], key=int)} + T = {k: inv0['targets'][k] for k in sorted(inv0['targets'], key=int)} + P = collections.Counter({int(k): v for k, v in P0.items()}) + S0 = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in P0.items()) + paid0 = sum(P.values()) + for k, v in dpaid.items(): + P[k] += CALLS * v + Pn = {str(k): v for k, v in sorted(P.items()) if v} + S = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in Pn.items()) + require(S == S0 == 1424, 'rank surplus 1424 unchanged') + require(json.loads((DATA / 'invoice-new.json').read_text()) == invoice_json(Pn, T), 'invoice-new.json matches') + require(json.loads((DATA / 'invoice-zero.json').read_text()) == invoice_json({}, {}), 'zero invoice') + require(inv0 == invoice_json(P0, T), 'invoice-7621.json is #7621 invoice in the official schema') + paid1 = sum(Pn.values()) + require((paid0, paid1) == (110145, 110145), 'paid positions 110145 unchanged') + print('PASS complete invoice: paid positions %d -> %d, productive %d, S = %d' % (paid0, paid1, sum(T.values()), S)) + k0, k1 = crossing(T, P0), crossing(T, Pn) + require(decimal.Decimal('0.0036614625547') <= k0 <= decimal.Decimal('0.0036614625548'), '#7621 crossing reproduced') + print('PASS kappa #7621 %s' % k0) + print('PASS kappa new %s (ratio over #7621 %s)' % (k1, +decimal.Context(prec=12).divide(k1, k0))) + expected = json.loads((HERE / 'EXPECTED.json').read_text()) + require(str(k1)[:len(expected['kappa_prefix'])] == expected['kappa_prefix'], 'kappa digits as recorded') + require(k1 > decimal.Decimal('0.0036624027915'), 'claim 0.0036624027915 is below the exact crossing') + if MODE == 'core': + print('PASS_CORE_LOAN_ON_7621') + return + run_controls(B, W) + columns_section(B, W, 0) + columns_section(B, W, 1) + print('PASS_LOAN_SNAPSHOT_ON_7621_EXACT (conditional on exactly #7620\'s premises, 180 CLEARs)') + + +if __name__ == '__main__': + main() diff --git a/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7620.json b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7620.json new file mode 100644 index 0000000000..b230682cf7 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7620.json @@ -0,0 +1 @@ +[[1,389],[1,390],[2,299],[3,1],[3,2],[3,9],[3,10],[3,11],[3,18],[3,21],[3,23],[3,24],[3,25],[3,27],[3,29],[3,30],[3,31],[3,35],[3,39],[3,40],[3,46],[3,50],[3,51],[3,52],[3,58],[3,60],[3,61],[3,66],[3,68],[3,71],[3,79],[3,83],[3,84],[3,85],[3,86],[3,90],[3,95],[3,96],[3,97],[3,100],[3,101],[3,102],[3,257],[3,259],[3,261],[3,278],[3,280],[3,281],[3,282],[3,284],[3,285],[3,298],[3,299],[3,300],[3,301],[3,313],[3,315],[3,319],[3,320],[3,321],[3,322],[3,323],[3,324],[3,327],[3,331],[3,332],[3,333],[3,334],[3,337],[3,338],[3,339],[3,340],[3,351],[3,353],[3,359],[3,360],[3,361],[3,369],[3,371],[3,372],[3,373],[3,375],[3,376],[3,377],[3,378],[3,381],[3,382],[3,386],[3,389],[3,390],[3,393],[3,394],[3,395],[3,397],[3,398],[3,400],[3,401],[3,404],[3,405],[3,411],[3,413],[3,414],[3,419],[3,420],[3,429],[3,431],[3,432],[3,435],[3,436],[3,438],[3,441],[3,451],[3,452],[3,455],[3,456],[3,475],[3,488],[3,492],[3,493],[3,495],[3,506],[3,521],[3,545],[3,546],[3,551],[3,561],[3,562],[3,574],[3,575],[3,576],[3,594],[3,595],[3,597],[3,602],[3,603],[3,652],[3,661],[3,744],[3,745],[3,746],[3,817],[3,1053],[3,1083],[7,251],[9,635],[9,665],[10,270],[10,277],[11,275],[18,334],[21,370],[23,301],[24,429],[25,255],[27,411],[29,315],[30,351],[31,371],[31,372],[35,316],[39,484],[40,487],[42,501],[46,369],[50,331],[51,333],[51,334],[52,313],[58,274],[60,446],[61,428],[66,272],[68,405],[70,386],[71,314],[76,386],[78,404],[79,353],[83,295],[83,296],[84,274],[85,276],[85,277],[86,260],[90,441],[90,448],[93,310],[95,332],[95,344],[96,313],[97,485],[100,356],[101,369],[101,373],[101,376],[102,372],[128,127],[129,127],[131,127],[133,127],[134,127],[139,127],[140,127],[141,127],[142,127],[149,127],[152,127],[153,127],[155,127],[156,127],[158,127],[165,127],[167,127],[172,127],[174,127],[176,127],[178,127],[181,127],[183,127],[184,127],[186,127],[188,127],[190,127],[192,127],[194,127],[196,127],[200,127],[201,127],[202,127],[204,127],[206,127],[207,127],[209,127],[210,127],[212,127],[215,127],[218,127],[221,127],[225,127],[227,127],[229,127],[238,127],[245,127],[246,127],[251,264],[255,261],[257,553],[257,682],[259,127],[259,723],[260,719],[261,730],[261,736],[264,743],[270,285],[272,281],[274,283],[274,284],[275,287],[276,282],[277,280],[277,286],[278,564],[278,633],[280,826],[280,838],[280,951],[280,965],[281,827],[281,839],[281,952],[281,966],[282,537],[282,846],[282,959],[282,973],[283,525],[283,834],[283,960],[283,974],[284,850],[284,979],[284,993],[284,1074],[285,851],[285,866],[285,980],[285,994],[286,857],[286,873],[286,985],[286,999],[287,858],[287,874],[287,986],[287,1000],[295,300],[296,298],[298,526],[298,828],[298,953],[298,975],[299,829],[299,847],[299,954],[299,976],[300,538],[300,840],[300,961],[300,967],[301,835],[301,841],[301,962],[301,968],[310,716],[313,322],[313,323],[313,554],[314,326],[315,321],[315,822],[316,325],[319,838],[320,839],[321,526],[321,822],[322,554],[322,847],[323,853],[323,881],[323,981],[323,1007],[324,1008],[324,1075],[325,863],[325,889],[325,985],[325,1013],[326,864],[326,890],[326,986],[326,1014],[327,783],[327,907],[331,338],[331,817],[332,340],[332,555],[333,339],[333,823],[334,337],[334,343],[334,816],[337,816],[337,846],[338,525],[338,817],[339,823],[339,840],[340,555],[340,841],[343,855],[343,979],[344,856],[344,892],[344,980],[344,1016],[351,360],[351,555],[353,359],[353,823],[356,727],[359,538],[359,823],[360,555],[360,835],[361,885],[361,1009],[362,1010],[362,1076],[369,378],[369,379],[369,554],[370,382],[371,377],[371,822],[372,375],[372,381],[372,816],[372,871],[373,618],[373,733],[373,913],[375,537],[375,816],[376,817],[376,834],[377,822],[377,828],[378,554],[378,829],[379,877],[379,1001],[379,1007],[379,1077],[380,888],[380,1008],[381,871],[381,893],[382,872],[382,894],[382,994],[382,1016],[386,574],[386,740],[386,909],[389,395],[389,557],[390,393],[390,818],[393,818],[393,1074],[394,866],[395,557],[395,875],[397,881],[398,1075],[400,892],[401,922],[401,1062],[404,521],[404,913],[405,414],[405,821],[411,575],[411,912],[413,873],[414,821],[414,874],[419,889],[420,890],[428,437],[429,576],[429,770],[431,850],[432,851],[435,885],[436,1076],[437,528],[437,973],[438,894],[441,456],[441,527],[446,458],[448,457],[451,857],[452,858],[455,1077],[456,527],[456,888],[457,844],[457,965],[458,845],[458,966],[475,877],[484,494],[485,497],[487,496],[488,548],[488,660],[492,855],[493,856],[494,528],[495,872],[496,766],[497,767],[501,513],[506,651],[506,664],[513,127],[513,914],[521,913],[521,1029],[525,127],[525,974],[526,127],[526,975],[527,976],[528,127],[528,973],[537,127],[537,959],[538,127],[538,961],[545,665],[546,548],[548,127],[548,660],[551,553],[553,127],[553,682],[554,946],[555,948],[557,945],[561,657],[562,564],[564,127],[564,633],[574,909],[574,1035],[575,912],[575,1036],[576,770],[576,1033],[594,602],[594,723],[594,760],[594,764],[594,765],[595,127],[595,603],[595,618],[595,736],[597,731],[597,732],[602,636],[602,638],[602,640],[602,641],[602,684],[602,685],[602,686],[602,687],[602,688],[602,689],[603,127],[618,733],[618,789],[618,913],[631,127],[631,152],[633,127],[633,212],[635,127],[635,210],[636,631],[636,647],[638,645],[638,651],[640,647],[641,645],[645,127],[645,651],[647,127],[647,631],[649,127],[649,176],[651,127],[651,140],[651,174],[652,649],[652,657],[657,127],[657,649],[660,127],[660,231],[661,664],[664,127],[664,651],[665,127],[665,635],[674,127],[674,142],[682,127],[682,134],[682,180],[684,682],[684,695],[685,694],[685,696],[686,674],[686,693],[687,694],[688,695],[689,693],[693,127],[693,674],[694,127],[694,696],[695,127],[695,682],[696,127],[696,165],[716,783],[716,807],[719,127],[719,907],[723,127],[723,902],[727,729],[729,789],[729,808],[730,127],[730,913],[731,826],[732,827],[733,238],[736,127],[736,909],[740,1062],[740,1071],[743,245],[743,922],[744,746],[744,916],[745,747],[746,207],[746,916],[747,245],[747,917],[760,853],[764,863],[765,864],[766,844],[767,845],[770,901],[783,807],[783,907],[789,808],[789,913],[807,932],[808,933],[816,939],[817,940],[818,939],[821,938],[822,945],[823,947],[826,127],[826,951],[827,127],[827,952],[828,127],[828,953],[829,127],[829,954],[834,127],[834,960],[835,127],[835,962],[838,127],[838,965],[839,127],[839,966],[840,127],[840,967],[841,127],[841,968],[844,127],[844,965],[845,127],[845,966],[846,127],[846,973],[847,127],[847,976],[850,127],[850,979],[851,127],[851,980],[853,127],[853,981],[855,127],[855,979],[856,127],[856,980],[857,127],[857,985],[858,127],[858,986],[863,127],[863,985],[864,127],[864,986],[866,127],[866,994],[871,993],[872,127],[872,994],[873,127],[873,999],[874,127],[874,1000],[875,127],[877,127],[877,1001],[881,127],[881,1007],[885,127],[885,1009],[888,127],[888,1008],[889,127],[889,1013],[890,127],[890,1014],[892,127],[892,1016],[893,1015],[894,127],[894,1016],[901,246],[902,127],[907,127],[907,177],[909,127],[912,127],[913,127],[913,147],[914,245],[916,127],[917,245],[922,195],[922,245],[932,128],[933,152],[938,184],[939,184],[939,212],[940,192],[945,200],[945,210],[946,194],[947,210],[948,194],[951,127],[951,156],[952,127],[952,139],[953,127],[953,156],[954,127],[954,139],[959,127],[959,178],[960,127],[960,221],[961,127],[961,178],[962,127],[962,221],[965,127],[965,149],[965,206],[966,127],[966,218],[966,227],[967,127],[967,206],[968,127],[968,218],[973,127],[973,158],[973,229],[974,127],[974,172],[975,127],[975,158],[976,127],[976,172],[976,229],[979,127],[979,129],[979,153],[980,127],[980,133],[980,181],[981,127],[981,181],[985,127],[985,183],[985,196],[986,127],[986,155],[986,204],[993,127],[993,188],[993,209],[994,127],[994,201],[994,215],[999,127],[999,131],[1000,127],[1000,190],[1001,127],[1001,186],[1007,127],[1007,176],[1007,225],[1008,127],[1008,141],[1008,165],[1009,127],[1009,165],[1010,127],[1010,225],[1013,127],[1013,174],[1014,127],[1014,140],[1015,142],[1016,127],[1016,140],[1016,167],[1029,1079],[1033,1041],[1035,1042],[1036,1043],[1041,246],[1042,127],[1043,127],[1053,1083],[1062,922],[1062,1071],[1071,1082],[1074,127],[1074,993],[1075,127],[1075,1008],[1076,127],[1076,1010],[1077,127],[1077,1007],[1079,127],[1082,134],[1083,202]] \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7621.json b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7621.json new file mode 100644 index 0000000000..da93c17e80 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-7621.json @@ -0,0 +1 @@ +[[1,389],[1,390],[2,299],[3,1],[3,2],[3,9],[3,10],[3,11],[3,18],[3,21],[3,23],[3,24],[3,25],[3,27],[3,29],[3,30],[3,31],[3,35],[3,39],[3,40],[3,46],[3,50],[3,51],[3,52],[3,58],[3,60],[3,61],[3,66],[3,68],[3,71],[3,79],[3,83],[3,84],[3,85],[3,86],[3,90],[3,95],[3,96],[3,97],[3,100],[3,101],[3,102],[3,257],[3,259],[3,261],[3,278],[3,280],[3,281],[3,282],[3,284],[3,285],[3,298],[3,299],[3,300],[3,301],[3,313],[3,315],[3,319],[3,320],[3,321],[3,322],[3,323],[3,324],[3,327],[3,331],[3,332],[3,333],[3,334],[3,337],[3,338],[3,339],[3,340],[3,351],[3,353],[3,359],[3,360],[3,361],[3,369],[3,371],[3,372],[3,373],[3,375],[3,376],[3,377],[3,378],[3,381],[3,382],[3,386],[3,389],[3,390],[3,393],[3,394],[3,395],[3,397],[3,398],[3,400],[3,401],[3,404],[3,405],[3,411],[3,413],[3,414],[3,419],[3,420],[3,429],[3,431],[3,432],[3,435],[3,436],[3,438],[3,441],[3,451],[3,452],[3,455],[3,456],[3,475],[3,488],[3,492],[3,493],[3,495],[3,506],[3,521],[3,545],[3,546],[3,551],[3,561],[3,562],[3,574],[3,575],[3,576],[3,594],[3,595],[3,597],[3,602],[3,603],[3,652],[3,661],[3,744],[3,745],[3,746],[3,755],[3,757],[3,817],[3,1053],[3,1083],[7,251],[9,635],[9,665],[10,270],[10,277],[11,275],[18,334],[21,370],[23,301],[24,429],[25,255],[27,411],[29,315],[30,351],[31,371],[31,372],[35,316],[39,484],[40,487],[42,501],[46,369],[50,331],[51,333],[51,334],[52,313],[58,274],[60,446],[61,428],[66,272],[68,405],[70,386],[71,314],[76,386],[78,404],[79,353],[83,295],[83,296],[84,274],[85,276],[85,277],[86,260],[90,441],[90,448],[93,310],[95,332],[95,344],[96,313],[97,485],[100,356],[101,369],[101,373],[101,376],[102,372],[128,127],[129,127],[131,127],[133,127],[134,127],[139,127],[140,127],[141,127],[142,127],[149,127],[152,127],[153,127],[155,127],[156,127],[158,127],[165,127],[167,127],[172,127],[174,127],[176,127],[178,127],[181,127],[183,127],[184,127],[186,127],[188,127],[190,127],[192,127],[194,127],[196,127],[200,127],[201,127],[202,127],[204,127],[206,127],[207,127],[209,127],[210,127],[212,127],[215,127],[218,127],[221,127],[225,127],[227,127],[229,127],[238,127],[245,127],[246,127],[251,264],[255,261],[257,553],[257,682],[259,127],[259,723],[260,719],[260,906],[261,730],[261,736],[264,743],[270,285],[272,281],[274,283],[274,284],[275,287],[276,282],[277,280],[277,286],[278,564],[278,633],[280,826],[280,838],[280,951],[280,965],[281,827],[281,839],[281,952],[281,966],[282,537],[282,846],[282,959],[282,973],[283,525],[283,834],[283,960],[283,974],[284,850],[284,979],[284,993],[284,1074],[285,851],[285,866],[285,980],[285,994],[286,857],[286,873],[286,985],[286,999],[287,858],[287,874],[287,986],[287,1000],[295,300],[296,298],[298,526],[298,828],[298,953],[298,975],[299,829],[299,847],[299,954],[299,976],[300,538],[300,840],[300,961],[300,967],[301,835],[301,841],[301,962],[301,968],[310,716],[313,322],[313,323],[313,554],[314,326],[315,321],[315,822],[316,325],[319,838],[320,839],[321,526],[321,822],[322,554],[322,847],[323,853],[323,881],[323,981],[323,1007],[324,1008],[324,1075],[325,863],[325,889],[325,985],[325,1013],[326,864],[326,890],[326,986],[326,1014],[327,783],[327,907],[331,338],[331,817],[332,340],[332,555],[333,339],[333,823],[334,337],[334,343],[334,816],[337,816],[337,846],[338,525],[338,817],[339,823],[339,840],[340,555],[340,841],[343,855],[343,979],[344,856],[344,892],[344,980],[344,1016],[351,360],[351,555],[353,359],[353,823],[356,727],[359,538],[359,823],[360,555],[360,835],[361,885],[361,1009],[362,1010],[362,1076],[369,378],[369,379],[369,554],[370,382],[371,377],[371,822],[372,375],[372,381],[372,816],[372,871],[373,618],[373,733],[373,913],[375,537],[375,816],[376,817],[376,834],[377,822],[377,828],[378,554],[378,829],[379,877],[379,1001],[379,1007],[379,1077],[380,888],[380,1008],[381,871],[381,893],[382,872],[382,894],[382,994],[382,1016],[386,574],[386,740],[386,909],[389,395],[389,557],[390,393],[390,818],[393,818],[393,1074],[394,866],[395,557],[395,875],[397,881],[398,1075],[400,892],[401,922],[401,1062],[404,521],[404,913],[405,414],[405,821],[411,575],[411,912],[413,873],[414,821],[414,874],[419,889],[420,890],[428,437],[429,576],[429,770],[431,850],[432,851],[435,885],[436,1076],[437,528],[437,973],[438,894],[441,456],[441,527],[446,458],[448,457],[451,857],[452,858],[455,1077],[456,527],[456,888],[457,844],[457,965],[458,845],[458,966],[475,877],[484,494],[485,497],[487,496],[488,548],[488,660],[492,855],[493,856],[494,528],[495,872],[496,766],[497,767],[501,513],[506,651],[506,664],[513,127],[513,914],[521,913],[521,1029],[525,127],[525,974],[526,127],[526,975],[527,976],[528,127],[528,973],[537,127],[537,959],[538,127],[538,961],[545,665],[546,548],[548,127],[548,660],[551,553],[553,127],[553,682],[554,946],[555,948],[557,945],[561,657],[562,564],[564,127],[564,633],[574,909],[574,1035],[575,912],[575,1036],[576,770],[576,1033],[594,602],[594,723],[594,760],[594,764],[594,765],[595,127],[595,603],[595,618],[595,736],[597,731],[597,732],[602,636],[602,638],[602,640],[602,641],[602,684],[602,685],[602,686],[602,687],[602,688],[602,689],[603,127],[618,733],[618,789],[618,913],[631,127],[631,152],[633,127],[633,212],[635,127],[635,210],[636,631],[636,647],[638,645],[638,651],[640,647],[641,645],[645,127],[645,651],[647,127],[647,631],[649,127],[649,176],[651,127],[651,140],[651,174],[652,649],[652,657],[657,127],[657,649],[660,127],[660,231],[661,664],[664,127],[664,651],[665,127],[665,635],[674,127],[674,142],[682,127],[682,134],[682,180],[684,682],[684,695],[685,694],[685,696],[686,674],[686,693],[687,694],[688,695],[689,693],[693,127],[693,674],[694,127],[694,696],[695,127],[695,682],[696,127],[696,165],[716,783],[716,807],[719,907],[723,127],[723,902],[727,729],[729,789],[729,808],[730,127],[730,913],[731,826],[732,827],[733,238],[736,127],[736,909],[740,1062],[740,1071],[743,245],[743,922],[744,746],[744,916],[745,747],[746,207],[746,916],[747,245],[747,917],[755,757],[755,924],[757,923],[757,924],[760,853],[764,863],[765,864],[766,844],[767,845],[770,901],[783,807],[783,907],[789,808],[789,913],[807,932],[808,933],[816,939],[817,940],[818,939],[821,938],[822,945],[823,947],[826,127],[826,951],[827,127],[827,952],[828,127],[828,953],[829,127],[829,954],[834,127],[834,960],[835,127],[835,962],[838,127],[838,965],[839,127],[839,966],[840,127],[840,967],[841,127],[841,968],[844,127],[844,965],[845,127],[845,966],[846,127],[846,973],[847,127],[847,976],[850,127],[850,979],[851,127],[851,980],[853,127],[853,981],[855,127],[855,979],[856,127],[856,980],[857,127],[857,985],[858,127],[858,986],[863,127],[863,985],[864,127],[864,986],[866,127],[866,994],[871,993],[872,127],[872,994],[873,127],[873,999],[874,127],[874,1000],[875,127],[877,127],[877,1001],[881,127],[881,1007],[885,127],[885,1009],[888,127],[888,1008],[889,127],[889,1013],[890,127],[890,1014],[892,127],[892,1016],[893,1015],[894,127],[894,1016],[901,246],[902,127],[906,127],[907,127],[907,177],[909,127],[912,127],[913,127],[913,147],[914,245],[916,127],[917,245],[922,195],[922,245],[923,127],[924,127],[932,128],[933,152],[938,184],[939,184],[939,212],[940,192],[945,200],[945,210],[946,194],[947,210],[948,194],[951,127],[951,156],[952,127],[952,139],[953,127],[953,156],[954,127],[954,139],[959,127],[959,178],[960,127],[960,221],[961,127],[961,178],[962,127],[962,221],[965,127],[965,149],[965,206],[966,127],[966,218],[966,227],[967,127],[967,206],[968,127],[968,218],[973,127],[973,158],[973,229],[974,127],[974,172],[975,127],[975,158],[976,127],[976,172],[976,229],[979,127],[979,129],[979,153],[980,127],[980,133],[980,181],[981,127],[981,181],[985,127],[985,183],[985,196],[986,127],[986,155],[986,204],[993,127],[993,188],[993,209],[994,127],[994,201],[994,215],[999,127],[999,131],[1000,127],[1000,190],[1001,127],[1001,186],[1007,127],[1007,176],[1007,225],[1008,127],[1008,141],[1008,165],[1009,127],[1009,165],[1010,127],[1010,225],[1013,127],[1013,174],[1014,127],[1014,140],[1015,142],[1016,127],[1016,140],[1016,167],[1029,1079],[1033,1041],[1035,1042],[1036,1043],[1041,246],[1042,127],[1043,127],[1053,1083],[1062,922],[1062,1071],[1071,1082],[1074,127],[1074,993],[1075,127],[1075,1008],[1076,127],[1076,1010],[1077,127],[1077,1007],[1079,127],[1082,134],[1083,202]] \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-loan.json b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-loan.json new file mode 100644 index 0000000000..89c535e4bf --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/affected-pairs-loan.json @@ -0,0 +1 @@ +[[1,389],[1,390],[2,299],[3,1],[3,2],[3,4],[3,5],[3,9],[3,10],[3,11],[3,18],[3,21],[3,23],[3,24],[3,25],[3,26],[3,27],[3,29],[3,30],[3,31],[3,35],[3,39],[3,40],[3,46],[3,47],[3,50],[3,51],[3,52],[3,55],[3,56],[3,58],[3,60],[3,61],[3,66],[3,68],[3,69],[3,71],[3,78],[3,79],[3,82],[3,83],[3,84],[3,85],[3,86],[3,89],[3,90],[3,95],[3,96],[3,97],[3,100],[3,101],[3,102],[3,257],[3,259],[3,261],[3,278],[3,280],[3,281],[3,282],[3,284],[3,285],[3,298],[3,299],[3,300],[3,301],[3,313],[3,315],[3,319],[3,320],[3,321],[3,322],[3,323],[3,324],[3,327],[3,331],[3,332],[3,333],[3,334],[3,337],[3,338],[3,339],[3,340],[3,351],[3,353],[3,359],[3,360],[3,361],[3,369],[3,371],[3,372],[3,373],[3,375],[3,376],[3,377],[3,378],[3,381],[3,382],[3,386],[3,389],[3,390],[3,393],[3,394],[3,395],[3,397],[3,398],[3,399],[3,400],[3,401],[3,404],[3,405],[3,407],[3,408],[3,409],[3,410],[3,411],[3,413],[3,414],[3,415],[3,416],[3,417],[3,419],[3,420],[3,427],[3,429],[3,431],[3,432],[3,433],[3,435],[3,436],[3,438],[3,441],[3,443],[3,445],[3,447],[3,448],[3,451],[3,452],[3,453],[3,455],[3,456],[3,475],[3,486],[3,488],[3,492],[3,493],[3,495],[3,506],[3,521],[3,545],[3,546],[3,551],[3,561],[3,562],[3,574],[3,575],[3,576],[3,594],[3,595],[3,597],[3,602],[3,603],[3,652],[3,661],[3,744],[3,745],[3,746],[3,755],[3,757],[3,817],[3,1053],[3,1083],[4,447],[4,448],[5,443],[7,251],[9,635],[9,665],[10,270],[10,277],[11,275],[18,334],[21,370],[23,301],[24,429],[24,438],[24,849],[25,255],[26,390],[27,408],[27,411],[29,315],[30,351],[31,371],[31,372],[35,316],[39,484],[40,487],[42,501],[46,369],[47,486],[50,331],[51,333],[51,334],[52,313],[55,407],[56,410],[58,274],[60,446],[61,428],[66,272],[68,405],[69,409],[69,410],[70,386],[71,314],[76,386],[78,404],[78,407],[79,353],[82,427],[83,295],[83,296],[84,274],[85,276],[85,277],[86,260],[89,445],[90,441],[90,448],[93,310],[95,332],[95,344],[96,313],[97,485],[100,356],[101,369],[101,373],[101,376],[102,372],[128,127],[129,127],[131,127],[133,127],[134,127],[137,127],[139,127],[140,127],[141,127],[142,127],[143,127],[145,127],[147,127],[149,127],[152,127],[153,127],[155,127],[156,127],[158,127],[162,127],[163,127],[165,127],[167,127],[172,127],[174,127],[176,127],[178,127],[181,127],[183,127],[184,127],[186,127],[188,127],[190,127],[192,127],[194,127],[196,127],[200,127],[201,127],[202,127],[204,127],[206,127],[207,127],[209,127],[210,127],[212,127],[214,127],[215,127],[218,127],[221,127],[225,127],[227,127],[229,127],[238,127],[245,127],[246,127],[251,264],[255,261],[257,553],[257,682],[259,127],[259,723],[260,719],[260,906],[261,730],[261,736],[264,743],[270,285],[272,281],[274,283],[274,284],[275,287],[276,282],[277,280],[277,286],[278,564],[278,633],[280,826],[280,838],[280,951],[280,965],[281,827],[281,839],[281,952],[281,966],[282,537],[282,846],[282,959],[282,973],[283,525],[283,834],[283,960],[283,974],[284,850],[284,979],[284,993],[284,1074],[285,851],[285,866],[285,980],[285,994],[286,857],[286,873],[286,985],[286,999],[287,858],[287,874],[287,986],[287,1000],[295,300],[296,298],[298,526],[298,828],[298,953],[298,975],[299,829],[299,847],[299,954],[299,976],[300,538],[300,840],[300,961],[300,967],[301,835],[301,841],[301,962],[301,968],[310,716],[313,322],[313,323],[313,554],[314,326],[315,321],[315,822],[316,325],[319,838],[320,839],[321,526],[321,822],[322,554],[322,847],[323,853],[323,881],[323,981],[323,1007],[324,1008],[324,1075],[325,863],[325,889],[325,985],[325,1013],[326,864],[326,890],[326,986],[326,1014],[327,783],[327,907],[331,338],[331,817],[332,340],[332,555],[333,339],[333,823],[334,337],[334,343],[334,816],[337,816],[337,846],[338,525],[338,817],[339,823],[339,840],[340,555],[340,841],[343,855],[343,979],[344,856],[344,892],[344,980],[344,1016],[351,360],[351,555],[353,359],[353,823],[356,727],[359,538],[359,823],[360,555],[360,835],[361,885],[361,1009],[362,1010],[362,1076],[369,378],[369,379],[369,554],[370,382],[371,377],[371,822],[372,375],[372,381],[372,816],[372,871],[373,618],[373,733],[373,913],[375,537],[375,816],[376,817],[376,834],[377,822],[377,828],[378,554],[378,829],[379,877],[379,1001],[379,1007],[379,1077],[380,888],[380,1008],[381,871],[381,893],[382,872],[382,894],[382,994],[382,1016],[386,574],[386,740],[386,909],[389,395],[389,557],[390,393],[390,399],[390,818],[390,837],[393,818],[393,1074],[394,866],[395,557],[395,875],[397,881],[398,1075],[399,837],[399,891],[400,892],[401,922],[401,1062],[404,521],[404,913],[405,414],[405,821],[407,416],[407,417],[407,824],[407,836],[408,420],[408,833],[409,415],[409,556],[410,413],[410,419],[410,820],[410,832],[411,575],[411,912],[413,820],[413,873],[414,821],[414,874],[415,556],[415,867],[416,824],[416,868],[417,836],[417,883],[419,832],[419,889],[420,833],[420,890],[427,433],[427,556],[428,437],[429,576],[429,770],[431,850],[432,851],[433,556],[433,859],[435,885],[436,1076],[437,528],[437,973],[438,849],[438,894],[441,456],[441,527],[443,452],[443,821],[445,455],[445,848],[446,458],[447,453],[447,557],[448,451],[448,457],[448,820],[451,820],[451,857],[452,821],[452,858],[453,557],[453,612],[455,848],[455,1077],[456,527],[456,888],[457,844],[457,965],[458,845],[458,966],[475,877],[484,494],[485,497],[486,492],[486,837],[487,496],[488,548],[488,660],[492,837],[492,855],[493,856],[494,528],[495,872],[496,766],[497,767],[501,513],[506,651],[506,664],[513,127],[513,914],[521,913],[521,1029],[525,127],[525,974],[526,127],[526,975],[527,976],[528,127],[528,973],[537,127],[537,959],[538,127],[538,961],[545,665],[546,548],[548,127],[548,660],[551,553],[553,127],[553,682],[554,946],[555,948],[556,947],[557,945],[561,657],[562,564],[564,127],[564,633],[574,909],[574,1035],[575,912],[575,1036],[576,770],[576,1033],[594,602],[594,723],[594,760],[594,764],[594,765],[595,127],[595,603],[595,618],[595,736],[597,731],[597,732],[602,636],[602,638],[602,640],[602,641],[602,684],[602,685],[602,686],[602,687],[602,688],[602,689],[603,127],[612,127],[618,733],[618,789],[618,913],[631,127],[631,152],[633,127],[633,212],[635,127],[635,210],[636,631],[636,647],[638,645],[638,651],[640,647],[641,645],[645,127],[645,651],[647,127],[647,631],[649,127],[649,176],[651,127],[651,140],[651,174],[652,649],[652,657],[657,127],[657,649],[660,127],[660,231],[661,664],[664,127],[664,651],[665,127],[665,635],[674,127],[674,142],[682,127],[682,134],[682,180],[684,682],[684,695],[685,694],[685,696],[686,674],[686,693],[687,694],[688,695],[689,693],[693,127],[693,674],[694,127],[694,696],[695,127],[695,682],[696,127],[696,165],[716,783],[716,807],[719,907],[723,127],[723,902],[727,729],[729,789],[729,808],[730,127],[730,913],[731,826],[732,827],[733,238],[736,127],[736,909],[740,1062],[740,1071],[743,245],[743,922],[744,746],[744,916],[745,747],[746,207],[746,916],[747,245],[747,917],[755,757],[755,924],[757,923],[757,924],[760,853],[764,863],[765,864],[766,844],[767,845],[770,901],[783,807],[783,907],[789,808],[789,913],[807,932],[808,933],[816,939],[817,940],[818,939],[820,937],[821,938],[822,945],[823,947],[824,948],[826,127],[826,951],[827,127],[827,952],[828,127],[828,953],[829,127],[829,954],[832,951],[833,952],[834,127],[834,960],[835,127],[835,962],[836,961],[837,959],[838,127],[838,965],[839,127],[839,966],[840,127],[840,967],[841,127],[841,968],[844,127],[844,965],[845,127],[845,966],[846,127],[846,973],[847,127],[847,976],[848,975],[849,974],[850,127],[850,979],[851,127],[851,980],[853,127],[853,981],[855,127],[855,979],[856,127],[856,980],[857,127],[857,985],[858,127],[858,986],[859,127],[863,127],[863,985],[864,127],[864,986],[866,127],[866,994],[867,127],[868,127],[871,993],[872,127],[872,994],[873,127],[873,999],[874,127],[874,1000],[875,127],[877,127],[877,1001],[881,127],[881,1007],[883,127],[885,127],[885,1009],[888,127],[888,1008],[889,127],[889,1013],[890,127],[890,1014],[891,127],[892,127],[892,1016],[893,1015],[894,127],[894,1016],[901,246],[902,127],[906,127],[907,127],[907,177],[909,127],[912,127],[913,127],[913,147],[914,245],[916,127],[917,245],[922,195],[922,245],[923,127],[924,127],[932,128],[933,152],[937,137],[938,184],[939,184],[939,212],[940,192],[945,200],[945,210],[946,194],[947,210],[947,214],[948,194],[948,200],[951,127],[951,156],[951,162],[952,127],[952,139],[952,143],[953,127],[953,156],[954,127],[954,139],[959,127],[959,145],[959,178],[960,127],[960,221],[961,127],[961,163],[961,178],[962,127],[962,221],[965,127],[965,149],[965,206],[966,127],[966,218],[966,227],[967,127],[967,206],[968,127],[968,218],[973,127],[973,158],[973,229],[974,127],[974,147],[974,172],[975,127],[975,147],[975,158],[976,127],[976,172],[976,229],[979,127],[979,129],[979,153],[980,127],[980,133],[980,181],[981,127],[981,181],[985,127],[985,183],[985,196],[986,127],[986,155],[986,204],[993,127],[993,188],[993,209],[994,127],[994,201],[994,215],[999,127],[999,131],[1000,127],[1000,190],[1001,127],[1001,186],[1007,127],[1007,176],[1007,225],[1008,127],[1008,141],[1008,165],[1009,127],[1009,165],[1010,127],[1010,225],[1013,127],[1013,174],[1014,127],[1014,140],[1015,142],[1016,127],[1016,140],[1016,167],[1029,1079],[1033,1041],[1035,1042],[1036,1043],[1041,246],[1042,127],[1043,127],[1053,1083],[1062,922],[1062,1071],[1071,1082],[1074,127],[1074,993],[1075,127],[1075,1008],[1076,127],[1076,1010],[1077,127],[1077,1007],[1079,127],[1082,134],[1083,202]] \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/base-7621-word.json.gz b/research/p104-loan-snapshot-engine-on7621/data/base-7621-word.json.gz new file mode 100644 index 0000000000..3cef99e591 Binary files /dev/null and b/research/p104-loan-snapshot-engine-on7621/data/base-7621-word.json.gz differ diff --git a/research/p104-loan-snapshot-engine-on7621/data/clear-certificates.json b/research/p104-loan-snapshot-engine-on7621/data/clear-certificates.json new file mode 100644 index 0000000000..16ad32e344 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/clear-certificates.json @@ -0,0 +1 @@ +{"clear_event_index":{"33948":{"385":"-1/4","387":"-1/2","389":"-1/4","394":"-1/2","64":"1/8"},"34297":{"385":"1/2","387":"-1","389":"-1/2","394":"-1","64":"1/4","96":"1"},"34379":{"385":"-1/2","387":"-1","389":"1/2","394":"-1","64":"-1/4","65":"-1/2","96":"1/2"},"34382":{"29":"-1/2","30":"-1/4","353":"-1/2","355":"-1","357":"1/2","362":"-1","59":"1/2"},"35077":{"384":"-1/2","385":"-1/2","387":"-1","389":"1/2","394":"1","64":"-3/4"},"35149":{"30":"-1/4","352":"-1/2","353":"-1/2","355":"-1","357":"1/2","362":"1","59":"1/2"},"35152":{"26":"1/4","368":"-3/4","369":"-1/2","371":"-1","373":"1/2","378":"1","8":"1/4"},"35562":{"104":"1/2","416":"-1/2","417":"1","421":"1/2","426":"2"},"35722":{"26":"1/8","368":"-1/8","369":"-1/4","371":"1/2","373":"-1/4","378":"-1/2","8":"1/8","95":"-1/4"},"37912":{"30":"1/4","353":"1/2","355":"-1","357":"-1/2","362":"1"},"37923":{"30":"1/8","353":"-1/4","355":"-1/2","357":"-1/4","362":"-1/2"},"37937":{"30":"1/8","352":"1/4","353":"1/4","355":"1/2","357":"1/4","362":"-1/2","59":"-1/4"},"37948":{"30":"1/8","353":"-1/4","355":"1/2","357":"-1/4","362":"-1/2"}}} \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/edits.json b/research/p104-loan-snapshot-engine-on7621/data/edits.json new file mode 100644 index 0000000000..caea604ee1 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/edits.json @@ -0,0 +1 @@ +{"base_events":77196,"delete":[[28767,["MOVE",386,443,452,false]],[28769,["ADD",386,384,"1",452]],[28815,["MOVE",359,410,413,false]],[28817,["ADD",359,357,"1",413]],[28849,["MOVE",391,448,451,false]],[28851,["ADD",391,389,"1",451]],[28895,["MOVE",358,409,415,false]],[28897,["ADD",358,355,"1",415]],[29322,["MOVE",356,407,416,false]],[29324,["ADD",356,353,"1",416]],[29327,["MOVE",360,407,417,false]],[29329,["ADD",360,362,"1",417]],[29333,["MOVE",364,410,419,false]],[29335,["ADD",364,59,"1",419]],[29337,["MOVE",366,408,420,false]],[29339,["ADD",366,30,"1",420]],[29583,["MOVE",390,447,453,false]],[29585,["ADD",390,387,"1",453]],[30504,["MOVE",392,445,455,false]],[30506,["ADD",392,394,"1",455]],[33941,["MOVE",562,3,455,false]],[33942,["ADD",562,392,"1",455]],[33952,["MOVE",392,455,848,false]],[34284,["MOVE",559,3,452,false]],[34285,["ADD",559,386,"1",452]],[34292,["MOVE",386,452,821,false]],[34354,["MOVE",558,3,451,false]],[34355,["ADD",558,391,"1",451]],[34357,["MOVE",534,3,413,false]],[34358,["ADD",534,359,"1",413]],[34363,["MOVE",391,451,820,false]],[34365,["MOVE",359,413,820,false]],[34443,["MOVE",374,427,433,false]],[34445,["ADD",374,371,"1",433]],[34593,["MOVE",422,486,492,false]],[34595,["ADD",422,419,"1",492]],[34692,["MOVE",382,24,438,false]],[34694,["ADD",382,95,"1",438]],[35046,["MOVE",560,3,453,false]],[35047,["ADD",560,390,"1",453]],[35054,["MOVE",390,453,557,false]],[35088,["ADD",1028,548,"-2",3]],[35106,["MOVE",536,3,415,false]],[35107,["ADD",536,358,"1",415]],[35109,["MOVE",548,3,433,false]],[35110,["ADD",548,374,"1",433]],[35115,["MOVE",358,415,556,false]],[35116,["MOVE",374,433,556,false]],[35810,["ADD",382,724,"2",438]],[36549,["MOVE",348,390,399,false]],[36551,["ADD",348,93,"1",399]],[36702,["ADD",348,714,"-2",399]],[41740,["MOVE",537,3,416,false]],[41741,["ADD",537,356,"1",416]],[41758,["MOVE",538,3,417,false]],[41759,["ADD",538,360,"1",417]],[41766,["MOVE",540,3,419,false]],[41767,["ADD",540,364,"1",419]],[41939,["MOVE",528,3,399,false]],[41940,["ADD",528,348,"1",399]],[42060,["MOVE",577,3,492,false]],[42061,["ADD",577,422,"1",492]],[42169,["MOVE",553,3,438,false]],[42170,["ADD",553,382,"1",438]],[42563,["MOVE",422,492,837,false]],[42565,["MOVE",348,399,837,false]],[42634,["MOVE",356,416,824,false]],[42637,["MOVE",541,3,420,false]],[42638,["ADD",541,366,"1",420]],[42766,["MOVE",382,438,849,false]],[42810,["MOVE",360,417,836,false]],[42813,["MOVE",364,419,832,false]],[44488,["MOVE",366,420,833,false]]],"insert_before":[[28769,[["INIT",1533,452],["ADD",1533,384,"1",452]]],[28817,[["INIT",1534,413],["ADD",1534,357,"1",413]]],[28851,[["INIT",1527,451],["ADD",1527,389,"1",451]]],[28897,[["INIT",1538,415],["ADD",1538,355,"1",415]]],[29324,[["INIT",1551,416],["ADD",1551,353,"1",416]]],[29329,[["INIT",1550,417],["ADD",1550,362,"1",417]]],[29335,[["INIT",1549,419],["ADD",1549,59,"1",419]]],[29339,[["INIT",1548,420],["ADD",1548,30,"1",420]]],[29585,[["INIT",1532,453],["ADD",1532,387,"1",453]]],[30506,[["INIT",1525,455],["ADD",1525,394,"1",455]]],[33939,[["MOVE",562,3,445,false],["ADD",562,392,"1",445],["ADD",392,562,"-1",445],["ADD",392,1154,"1",445],["MOVE",562,445,455,false],["MOVE",392,445,848,false]]],[33941,[["ADD",562,1525,"1",455],["CLEAR",1525,455]]],[33949,[["ADD",392,1154,"-1",848]]],[34269,[["MOVE",559,3,443,false],["ADD",559,386,"1",443],["ADD",386,559,"-1",443],["ADD",386,1148,"1",443],["MOVE",559,443,452,false],["MOVE",386,443,821,false]]],[34284,[["ADD",559,1533,"1",452],["CLEAR",1533,452]]],[34288,[["ADD",386,1148,"-1",821]]],[34335,[["MOVE",534,3,410,false],["ADD",534,359,"1",410],["ADD",359,534,"-1",410],["ADD",359,1121,"1",410],["MOVE",534,410,413,false],["MOVE",359,410,820,false]]],[34352,[["MOVE",558,3,448,false],["ADD",558,391,"1",448],["ADD",391,558,"-1",448],["ADD",391,1153,"1",448],["MOVE",558,448,451,false],["MOVE",391,448,820,false]]],[34354,[["ADD",558,1527,"1",451],["CLEAR",1527,451]]],[34357,[["ADD",534,1534,"1",413],["CLEAR",1534,413]]],[34361,[["ADD",391,1153,"-1",820]]],[34362,[["ADD",359,1121,"-1",820]]],[34445,[["INIT",1533,433],["ADD",1533,371,"1",433]]],[34595,[["INIT",1527,492],["ADD",1527,419,"1",492]]],[34694,[["INIT",1526,438],["ADD",1526,95,"1",438]]],[35035,[["MOVE",560,3,447,false],["ADD",560,390,"1",447],["ADD",390,560,"-1",447],["ADD",390,1152,"1",447],["MOVE",560,447,453,false],["MOVE",390,447,557,false]]],[35046,[["ADD",560,1532,"1",453],["CLEAR",1532,453]]],[35050,[["ADD",390,1152,"-1",557]]],[35084,[["ADD",1028,548,"-2",3]]],[35085,[["MOVE",548,3,427,false],["ADD",548,374,"1",427],["ADD",374,548,"-1",427],["ADD",374,1136,"1",427],["MOVE",548,427,433,false],["MOVE",374,427,556,false]]],[35104,[["MOVE",536,3,409,false],["ADD",536,358,"1",409],["ADD",358,536,"-1",409],["ADD",358,1120,"1",409],["MOVE",536,409,415,false],["MOVE",358,409,556,false]]],[35106,[["ADD",536,1538,"1",415],["CLEAR",1538,415]]],[35109,[["ADD",548,1533,"1",433],["CLEAR",1533,433]]],[35113,[["ADD",374,1136,"-1",556]]],[35114,[["ADD",358,1120,"-1",556]]],[35514,[["MOVE",577,3,486,false],["ADD",577,422,"1",486],["ADD",422,577,"-1",486],["ADD",422,1182,"1",486],["MOVE",577,486,492,false],["ADD",577,1527,"1",492],["CLEAR",1527,492],["MOVE",422,486,837,false]]],[35666,[["MOVE",553,3,24,false],["ADD",553,382,"1",24],["ADD",382,553,"-1",24],["ADD",382,1144,"1",24],["MOVE",553,24,438,false],["ADD",553,1526,"1",438],["CLEAR",1526,438],["MOVE",382,24,849,false]]],[35810,[["ADD",553,724,"2",438]]],[36418,[["MOVE",528,3,390,false],["ADD",528,348,"1",390],["ADD",348,528,"-1",390],["ADD",348,1110,"1",390],["MOVE",528,390,399,false],["MOVE",348,390,837,false]]],[36551,[["ADD",528,93,"1",399]]],[36702,[["ADD",528,714,"-2",399]]],[37843,[["MOVE",537,3,407,false],["ADD",537,356,"1",407],["ADD",356,537,"-1",407],["ADD",356,1118,"1",407],["MOVE",537,407,416,false],["ADD",537,1551,"1",416],["CLEAR",1551,416],["MOVE",356,407,824,false]]],[37846,[["MOVE",538,3,407,false],["ADD",538,360,"1",407],["ADD",360,538,"-1",407],["ADD",360,1122,"1",407],["MOVE",538,407,417,false],["ADD",538,1550,"1",417],["CLEAR",1550,417],["MOVE",360,407,836,false]]],[37852,[["MOVE",540,3,410,false],["ADD",540,364,"1",410],["ADD",364,540,"-1",410],["ADD",364,1126,"1",410],["MOVE",540,410,419,false],["ADD",540,1549,"1",419],["CLEAR",1549,419],["MOVE",364,410,832,false]]],[37855,[["MOVE",541,3,408,false],["ADD",541,366,"1",408],["ADD",366,541,"-1",408],["ADD",366,1128,"1",408],["MOVE",541,408,420,false],["ADD",541,1548,"1",420],["CLEAR",1548,420],["MOVE",366,408,833,false]]],[42064,[["ADD",422,1182,"-1",837]]],[42065,[["ADD",348,1110,"-1",837]]],[42174,[["ADD",382,1144,"-1",849]]],[42281,[["ADD",356,1118,"-1",824]]],[42382,[["ADD",360,1122,"-1",836]]],[42590,[["ADD",364,1126,"-1",832]]],[42645,[["ADD",366,1128,"-1",833]]]],"append":[]} \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/instances.json b/research/p104-loan-snapshot-engine-on7621/data/instances.json new file mode 100644 index 0000000000..cb4535740d --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/instances.json @@ -0,0 +1,212 @@ +[ +{ +"r": 382, +"X": 438, +"p": 553, +"side": "A", +"net": 1.4555158301618432, +"c": 1144, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366169560413416642460404341532726624267223003757034999108150", +"new_lifetimes": 1 +}, +{ +"r": 422, +"X": 492, +"p": 577, +"side": "A", +"net": 0.3397980735907946, +"c": 1182, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366174999347074518248183319077360063890301145349054010025291", +"new_lifetimes": 1 +}, +{ +"r": 392, +"X": 455, +"p": 562, +"side": "A", +"net": 0.3397980735907946, +"c": 1154, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366180438442301239565793971583069706789928580435619078978226", +"new_lifetimes": 1 +}, +{ +"r": 391, +"X": 451, +"p": 558, +"side": "A", +"net": 0.3397980735907946, +"c": 1153, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366185877699104005343683382581242392314113562778082254468500", +"new_lifetimes": 1 +}, +{ +"r": 390, +"X": 453, +"p": 560, +"side": "A", +"net": 0.3397980735907946, +"c": 1152, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366191317117490014939945400484375286898772350874028545970884", +"new_lifetimes": 1 +}, +{ +"r": 386, +"X": 452, +"p": 559, +"side": "A", +"net": 0.3397980735907946, +"c": 1148, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366196756697466468140352390760052493411295249936757557722434", +"new_lifetimes": 1 +}, +{ +"r": 374, +"X": 433, +"p": 548, +"side": "A", +"net": 0.3397980735907946, +"c": 1136, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366202196439040565158386990922732813943566822805459557701662", +"new_lifetimes": 1 +}, +{ +"r": 366, +"X": 420, +"p": 541, +"side": "A", +"net": 0.3397980735907946, +"c": 1128, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366207636342219506635273868369854392469866266447920061591350", +"new_lifetimes": 1 +}, +{ +"r": 364, +"X": 419, +"p": 540, +"side": "A", +"net": 0.3397980735907946, +"c": 1126, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366213076407010493640011481030070712436630672394954406261360", +"new_lifetimes": 1 +}, +{ +"r": 360, +"X": 417, +"p": 538, +"side": "A", +"net": 0.3397980735907946, +"c": 1122, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366218516633420727669403840857696740389628881100358986791860", +"new_lifetimes": 1 +}, +{ +"r": 359, +"X": 413, +"p": 534, +"side": "A", +"net": 0.3397980735907946, +"c": 1121, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366223957021457410648092280143704045602235886342898162982076", +"new_lifetimes": 1 +}, +{ +"r": 358, +"X": 415, +"p": 536, +"side": "A", +"net": 0.3397980735907946, +"c": 1120, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366229397571127744928587220665984089775172929666184618024972", +"new_lifetimes": 1 +}, +{ +"r": 356, +"X": 416, +"p": 537, +"side": "A", +"net": 0.3397980735907946, +"c": 1118, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366234838282438933291299945664364646151646667636877363746426", +"new_lifetimes": 1 +}, +{ +"r": 348, +"X": 399, +"p": 528, +"side": "A", +"net": 0.3397980735907946, +"c": 1110, +"sol": { +"$b": "-1", +"$be": "1", +"$ep": "-1" +}, +"kappa": "0.00366240279155398178944574374649845678910539553586138135192474", +"new_lifetimes": 0 +} +] \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/invoice-7621.json b/research/p104-loan-snapshot-engine-on7621/data/invoice-7621.json new file mode 100644 index 0000000000..a53a37912f --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/invoice-7621.json @@ -0,0 +1,47 @@ +{ + "schema": "integer-mult-recursive-invoice/v1", + "contract": { + "model": "mixed-target-unnormalized-power-v1", + "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", + "ambient_rank": 145, + "boundary": "complete signed #7113 caller application with 17 native calls" + }, + "paid": { + "1": 68112, + "2": 15725, + "3": 11509, + "4": 4743, + "5": 5984, + "6": 391, + "7": 187, + "8": 758, + "9": 32, + "16": 416, + "24": 416, + "32": 416, + "40": 416, + "48": 416, + "56": 416, + "64": 208 + }, + "targets": { + "1": 306, + "2": 357, + "3": 799, + "4": 3825, + "5": 1360, + "6": 731, + "7": 323, + "8": 544, + "9": 10888, + "17": 416, + "25": 416, + "33": 416, + "41": 416, + "49": 416, + "57": 416, + "65": 416, + "73": 208, + "137": 208 + } +} \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/invoice-new.json b/research/p104-loan-snapshot-engine-on7621/data/invoice-new.json new file mode 100644 index 0000000000..638ad110b8 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/invoice-new.json @@ -0,0 +1 @@ +{"schema": "integer-mult-recursive-invoice/v1", "contract": {"model": "mixed-target-unnormalized-power-v1", "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", "ambient_rank": 145, "boundary": "complete signed #7113 caller application with 17 native calls"}, "paid": {"1": 68129, "2": 15946, "3": 11033, "4": 4964, "5": 6001, "6": 391, "7": 187, "8": 758, "9": 32, "16": 416, "24": 416, "32": 416, "40": 416, "48": 416, "56": 416, "64": 208}, "targets": {"1": 306, "2": 357, "3": 799, "4": 3825, "5": 1360, "6": 731, "7": 323, "8": 544, "9": 10888, "17": 416, "25": 416, "33": 416, "41": 416, "49": 416, "57": 416, "65": 416, "73": 208, "137": 208}} \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/invoice-zero.json b/research/p104-loan-snapshot-engine-on7621/data/invoice-zero.json new file mode 100644 index 0000000000..6b9ea16bcf --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/data/invoice-zero.json @@ -0,0 +1 @@ +{"schema": "integer-mult-recursive-invoice/v1", "contract": {"model": "mixed-target-unnormalized-power-v1", "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", "ambient_rank": 145, "boundary": "complete signed #7113 caller application with 17 native calls"}, "paid": {}, "targets": {}} \ No newline at end of file diff --git a/research/p104-loan-snapshot-engine-on7621/data/word.json.gz b/research/p104-loan-snapshot-engine-on7621/data/word.json.gz new file mode 100644 index 0000000000..d4528141a4 Binary files /dev/null and b/research/p104-loan-snapshot-engine-on7621/data/word.json.gz differ diff --git a/research/p104-loan-snapshot-engine-on7621/engine/ENGINE.md b/research/p104-loan-snapshot-engine-on7621/engine/ENGINE.md new file mode 100644 index 0000000000..fe9e463465 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/ENGINE.md @@ -0,0 +1,85 @@ +# LOANBUILD engine: how to run it + +LOANBUILD is an exact builder for SYLforge's stationary-lifetime device (#7616/#7620). It works on P104 native words (schema p104-accumulated-block-composition/2). All code is ours; SYLforge's words were read only as data. Credit SYLforge for the device. + +## The move +1. Register r drops an interior stop X from its path A→X→B, so it now goes A→B. +2. A partner p takes one extra stop: + - side B: p is the source of r's write at X, and p inserts B after X; + - side A: p is the reader of r at X, and p inserts A before X. +3. r's work at X is carried by: + - a loan through a carrier c, a public register whose path runs A→X→B consecutively; + - stationary owned snapshots (INIT at X, one write, one read, dirty CLEAR); + - on side A, a signed role swap between r and p. +4. Native κ cost: each instance swaps one 3|1 hop merge against one 1+2 split, about +0.0015% κ. + +## Files +- `lb.py` is the core. + - `Word(W)` builds indices: register stays (`seg`), ops per register, owned live intervals, the owned-live profile, and projected-state checkpoints every 500 events. + - `Cand` holds one candidate: original program minus deletions plus keyed insertions. Coefficients written as `'$name'` are unknowns. + - `Cand.solve()` replays only the affected window (mod 2^61−1, 4-dim random projection), once with all unknowns at 0 and once per unknown at 1. + - It solves the affine system so the window-end state equals the original. It also requires every pre-existing owned observation (source and destination content at each owned-touching op) and every CLEAR content to match. + - It rationalises the solution and re-checks it. + - `instantiate()` writes the word and drops empty INIT/CLEAR pairs. + - `free_label` picks an owned label that is not live over the needed interval and keeps peak ≤ 50. +- `tpl.py` holds the templates. + - `build_B(Wd, r, X, p, carrier, beta, snap)`: + - loan c += β·r at A, placed just before c leaves A; + - c's existing X-transfer into p carries ρ; + - readers re-read p, or a snapshot of p; + - at B: unknown settles p += ζ·r and c += ε·r, then r's write relocated as r += q·p. + - β is enumerated from c's transfer coefficient; the other coefficients are solved. + - `build_A(Wd, r, X, p, carrier, snap, early)`: + - at A: p += α·r, swap r += b·p, loan r += β·c; + - at X: p picks up r's earlier writes from their sources, directly or via a snapshot; + - at B: settle r += ε·c. b, β and ε are solved. + - Slides keep the timing legal: + - `slide_earlier` moves r's late A-ops and p's late Z-ops before the cut; + - `slide_later` moves r's B-ops after the settle, cascading through later stops. + - Partners must sit on the same chart and be untouched in between. + - `try_instance(Wd, r, X, p, side)` loops over carriers × snapshot/no-snapshot (and β on side B) and returns the first exact solution, or failure reasons. +- `check.py` `full(base, new)`: + - fastx rows equal for all registers, plus structure; + - every base owned lifetime's observation trace, CLEAR content included, present unchanged; + - new lifetime count, peak, labels, non-dyadic count. +- `run7621.py` is the driver: + 1. exact census of both template shapes, gain-ordered by the convexity margin of x ln x; + 2. greedy `try_instance` over the list; + 3. each candidate is accepted only if `check.full` passes, peak ≤ 50 and PRICEKIT κ rises; + 4. after each pass it re-censuses; it stops on a pass with no hits. + - It writes `_.json.gz` (n = instances landed) and `_log.json`. +- `control.py` is the positive control. It reads (r, X, partner, side) from #7620's word as data and rebuilds those instances on #7617. + +## Run it (Python 3.12 + standard library; everything is vendored in this directory) +`pricekit.py` (the exact pricer), `fastx.py` (randomised-exact replay) and `jl.py` (kappa of a native word under the signed #7113 caller, `data/7113_CALLER.json`) sit next to the scripts, and every path is relative to the script. Run from the package root or anywhere. +``` +python3 engine/run_on.py WORD.json.gz [OUT_PREFIX] [MAX_PASSES] + # census -> greedy loan + snapshot passes -> re-census on any P104 native word (about 1 minute on #7621's word) +python3 engine/run_on.py data/base-7621-word.json.gz /tmp/out # writes /tmp/out_14.json.gz and /tmp/out_log.json = data/word.json.gz, data/instances.json +python3 engine/control.py # controls on #7617 using #7620's instances as data; expect {'rebuilt': 39, 'fail': 6, 'no partner': 1} +python3 engine/control.py 258@829 464@526 # the two named controls (same carriers and coefficients as SYLforge) +python3 -I -B check.py engine # the packaged test: rerun on #7621, require JSON-equality with data/word.json.gz +``` +Independent exact check of a result (not part of the engine): `python3 -I -B check.py core` on a package built like this one. + +## Results +- Controls: 39/44 of #7620's shape instances rebuilt (89%); 39/46 counting the 2 courier stops. 258@829 and 464@526 match SYLforge exactly. +- #7621 → data/word.json.gz (hit_loan_14): 14 instances, κ ×1.00025679. Every check passes (`check.py core`, `controls`, `columns1`, `columns2` in this package). +- A second round on hit_loan_14 with an extended template variant (not shipped) landed 0. That round added compact snapshots that start only when the source is about to change, side-A write-after-read (r += q·s moved to A or B), and extra unknown settles at B for spread contamination. `tpl.py` here is the version that reproduces hit_loan_14 exactly. + +## Remaining blockers on hit_loan_14 (85-place census) +| reason | count | meaning | +|---|---|---| +| nosol | 20 | contamination reaches registers that never meet r at B | +| no label | 17 | owned peak 50 is full over the snapshot interval | +| no carrier | 17 | no register runs A→X→B | +| A window | 15 | the carrier leaves A before r or p arrive | +| slides | 10 | a slide partner is touched in between | +| write after read | 2 | side-A write-after-read (r += q·s) | +| read before p at X | 2 | with no solution; r is read before p arrives at X | + +## Next levers +- A second carrier, so a contaminated register can be settled where it meets a register holding ρ rather than r. +- Shared snapshots, or a reorder of busy owned lifetimes, to free peak-50 events. +- Carriers that are not consecutive (A→…→X→…→B) with in-transit settles. +- Census shapes with several sources or readers. diff --git a/research/p104-loan-snapshot-engine-on7621/engine/check.py b/research/p104-loan-snapshot-engine-on7621/engine/check.py new file mode 100644 index 0000000000..d06596fcb2 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/check.py @@ -0,0 +1,39 @@ +# full checks of a built word against its base (own code) +import sys, collections +import os, sys +HERE = os.path.dirname(os.path.abspath(__file__)) +if HERE not in sys.path: sys.path.insert(0, HERE) +import fastx, pricekit as pk # vendored in this directory +def traces(W, R): + """per owned lifetime: tuple of (op, other-reg-if-public, coef, chart, src proj, dst proj) incl. CLEAR content""" + n=W['public']; NO=W['owned_fields']; K=4; P=fastx.P + M=[list(R[r]) for r in range(n)]+[[0]*K for _ in range(NO)] + cur={}; out=[]; live=0; peak=0 + for e in W['program']: + op=e[0] + if op=='MOVE': continue + if op=='INIT': cur[e[1]]=[('INIT',e[2])]; M[e[1]]=[0]*K; live+=1; peak=max(peak,live); continue + if op=='CLEAR': cur[e[1]].append(('CLEAR',e[2],tuple(M[e[1]]))); out.append(tuple(cur.pop(e[1]))); M[e[1]]=[0]*K; live-=1; continue + if op=='SCALE': + if e[1]>=n: cur[e[1]].append(('S',e[2],e[3],tuple(M[e[1]]))) + q=Q(e[2]); qq=q.numerator%P*pow(q.denominator%P,-1,P)%P; M[e[1]]=[x*qq%P for x in M[e[1]]]; continue + d,s=e[1],e[2] + if d>=n or s>=n: + rec=('A', d if d=n: cur[x].append(rec) + q=Q(e[3]); qq=q.numerator%P*pow(q.denominator%P,-1,P)%P + if op=='COPY': M[d]=[0]*K + M[d]=[(x+qq*y)%P for x,y in zip(M[d],M[s])] + return out, peak +from fractions import Fraction as Q +def full(B, W2, label=''): + n=B['public']; R=fastx.proj_matrix(n,4,1) + fB=fastx.FastReplay(B,R,4); fW=fastx.FastReplay(W2,R,4) + ok=fW.ok and all(fB.proj[q]==fW.proj[q] for q in range(n+B['owned_fields'])) + tb,pb=traces(B,R); tw,pw=traces(W2,R) + cb=collections.Counter(tb); cw=collections.Counter(tw) + miss=cb-cw; extra=cw-cb + labels=len({e[1] for e in W2['program'] if e[0]=='INIT'}) + nd=sum(1 for e in W2['program'] if e[0] in('ADD','COPY','SCALE') and (Q(e[3] if e[0]!='SCALE' else e[2]).denominator & (Q(e[3] if e[0]!='SCALE' else e[2]).denominator-1))) + return dict(rows_ok=ok, problems=fW.bad[:3], base_lifetimes_kept=not miss, missing=sum(miss.values()), new_lifetimes=sum(extra.values()), peak=pw, labels=labels, nondyadic=nd, events=len(W2['program'])) diff --git a/research/p104-loan-snapshot-engine-on7621/engine/control.py b/research/p104-loan-snapshot-engine-on7621/engine/control.py new file mode 100644 index 0000000000..3663b708ff --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/control.py @@ -0,0 +1,46 @@ +# CONTROLS: rebuild #7620's realised instances on #7617 (best7616) with our engine; (r,X,p,side) read from #7620 as data. +import sys, time, collections, pickle +import os, sys +HERE = os.path.dirname(os.path.abspath(__file__)) +if HERE not in sys.path: sys.path.insert(0, HERE) +import lb, tpl, check +import pricekit as pk +DATA=os.path.join(HERE,'data') +B=pk.load(DATA+'/word-7617.json.gz'); D=pk.load(DATA+'/word-7620.json.gz') +n=B['public']; dim=[len(b) for b in B['frames']] +def paths(W): + p={r:[W['initial'][r]] for r in range(n)} + for e in W['program']: + if e[0]=='MOVE' and e[1] .proj (N,2,k), .bad (structural problems, same kinds as pricekit.replay), .ok, .nproblems + proj_matrix(n, N, k, seed) -> R for a word with n publics and N-n owned (owned rows start at 0) + exact_rows(W, regs) -> {reg: Fraction row} by exact replay of the backward slice only + slice_events(W, regs) -> sorted event indices that can influence the final rows of `regs` +""" +import random, collections +from fractions import Fraction as Q +P = 2 ** 61 - 1 + +def proj_matrix(n, k=4, seed=1): + """R for publics 0..n-1: list of n random k-vectors mod 2^61-1.""" + rng = random.Random(seed) + return [[rng.randrange(1, P) for _ in range(k)] for _ in range(n)] + +class FastReplay: + def __init__(self, W, R=None, k=4, seed=1): + n = W['public']; NO = W['owned_fields']; N = n + NO; self.n = n; self.N = N; self.k = k + if R is None: R = proj_matrix(n, k, seed) + assert len(R) >= n + zero = [0] * k + M = [list(R[r]) for r in range(n)] + [list(zero) for _ in range(NO)] + cur = list(W['initial']) + [None] * NO; bad = []; owned = set() + cache = {} + def qm(s): + c = cache.get(s) + if c is None: + q = Q(s) + if q.denominator % P == 0: bad.append(('denominator-divisible-by-modulus', s)) + c = cache[s] = q.numerator % P * pow(q.denominator % P, -1, P) % P + return c + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + if op == 'MOVE': + if cur[r] != e[2]: bad.append(('move-from', i)) + cur[r] = e[3] + elif op == 'INIT': owned.add(r); M[r] = list(zero); cur[r] = e[2] + elif op == 'CLEAR': owned.discard(r); M[r] = list(zero); cur[r] = None + elif op == 'SCALE': + if cur[r] != e[3]: bad.append(('scale-frame', i)) + q = qm(e[2]); M[r] = [x * q % P for x in M[r]] + elif op in ('ADD', 'COPY'): + s = e[2]; f = e[4] + if cur[r] != f or cur[s] != f: bad.append(('add-frame', i)) + if op == 'COPY': + if any(M[r]): bad.append(('copy-nonzero', i)) + M[r] = list(zero) + q = qm(e[3]); M[r] = [(x + q * y) % P for x, y in zip(M[r], M[s])] + else: bad.append(('op', i, op)) + if owned: bad.append(('owned-alive', len(owned))) + if cur[:n] != list(W['final']): bad.append(('exit-frames',)) + self.proj = M; self.bad = bad; self.ok = not bad; self.nproblems = len(bad); self.cur = cur + self.W = W + def is_zero(self, r): return not any(self.proj[r]) + +def slice_events(W, regs): + """Backward slice: indices of events that can influence the final rows of regs.""" + need = set(regs); keep = [] + P = W['program'] + for i in range(len(P) - 1, -1, -1): + e = P[i]; op = e[0] + if op == 'MOVE': continue + r = e[1] + if r not in need: continue + keep.append(i) + if op in ('INIT', 'CLEAR'): need.discard(r) # content before is dropped + elif op == 'COPY': need.discard(r); need.add(e[2]) + elif op == 'ADD': need.add(e[2]) + return sorted(keep) + +def exact_rows(W, regs): + n = W['public']; N = n + W['owned_fields']; regs = list(regs) + ev = slice_events(W, regs); P = W['program'] + rows = {} + def row(r): + if r not in rows: rows[r] = {r: Q(1)} if r < n else {} + return rows[r] + for i in ev: + e = P[i]; op = e[0]; r = e[1] + if op in ('INIT', 'CLEAR'): rows[r] = {}; continue + if op == 'SCALE': + q = Q(e[2]); rows[r] = {k: v * q for k, v in row(r).items()}; continue + s = e[2]; q = Q(e[3]); R = row(r) + if op == 'COPY': R.clear() + for k, v in list(row(s).items()): + nv = R.get(k, 0) + q * v + if nv: R[k] = nv + else: R.pop(k, None) + return {r: row(r) for r in regs} + +def replay(W, R=None, k=4, seed=1): + """settle.replay-shaped API: returns (rows, cur, bad) like REBUILDKIT/settle.replay(W), BUT rows[r] is the k-vector PROJECTION of row r + (list of k ints mod 2^61-1), not a Fraction dict. Use proj_matrix(...) once and pass R to compare two words' rows + (equal projections <=> equal rows except with probability 2^-244 per row). bad entries are structural, same shapes as settle.replay.""" + F = FastReplay(W, R, k, seed) + return F.proj, F.cur, F.bad + +def unit_projection(W, R, r): + """projection of the identity row e_r (what 'helper ends identity' compares against).""" + return list(R[r]) + +def rows_equal(rowsA, rowsB, mapA=lambda r: r): + """indices r (of A) whose projections differ between A and B (B indexed through mapA).""" + return [r for r in range(len(rowsA)) if rowsA[r] != rowsB[mapA(r)]] diff --git a/research/p104-loan-snapshot-engine-on7621/engine/jl.py b/research/p104-loan-snapshot-engine-on7621/engine/jl.py new file mode 100644 index 0000000000..f53c4b0e64 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/jl.py @@ -0,0 +1,140 @@ +# JOINTEDIT library: tag-sequence model of a word (non-MOVE events in order, each with a chart tag). +# A public register's path = clean([initial] + tags of its ops in order + [final]); MOVEs are regenerated from it. +# Joint edit = slide ops inside exact commutation windows + retag, so the main skips a stop; partners pay hop changes. +import sys, gzip, json, math, bisect, os +import os, sys +HERE = os.path.dirname(os.path.abspath(__file__)) +if HERE not in sys.path: sys.path.insert(0, HERE) +from collections import defaultdict +import pricekit as pk # vendored in this directory +KAPPA = 0.0036576 +def f(r): return r ** (1 - KAPPA) if r > 0 else 0.0 +def spanmask(b): + S = {0} + for v in b: S |= {x ^ v for x in S} + m = 0 + for x in S: m |= 1 << x + return m +C_PATH = os.path.join(HERE, 'data', '7113_CALLER.json') # Rohan Arun's signed #7113 caller, vendored as data +_CI = None +def kappa(W): + global _CI + if _CI is None: _CI = pk.caller_invoice(pk.load(C_PATH)) + pr = pk.price(pk.invoice(_CI, pk.native_from_word(W)), 30) + return pr +class Word: + def __init__(s, W): + s.W = W; n = W['public']; s.n = n + s.sp = [spanmask(b) for b in W['frames']]; s.dim = [len(b) for b in W['frames']] + live = set(W['live']); H = W['hosted'] + s.mains = live | {r + 112 for r in live} | set(H['X2']) | set(H['Y2']) + s.ev = [e for e in W['program'] if e[0] != 'MOVE'] # E-order + s.key = [float(i) for i in range(len(s.ev))] + s.tag = [None] * len(s.ev) + s.ops = defaultdict(list) # public reg -> E idx (ADD/COPY/SCALE) + s.touch = defaultdict(list) # any reg -> E idx incl INIT/CLEAR + for i, e in enumerate(s.ev): + op = e[0] + if op in ('ADD', 'COPY'): + s.tag[i] = e[4]; regs = (e[1], e[2]) + elif op == 'SCALE': + s.tag[i] = e[3]; regs = (e[1],) + else: + s.tag[i] = e[2]; regs = (e[1],) + for r in regs: + s.touch[r].append(i) + if r < n and op != 'INIT' and op != 'CLEAR': s.ops[r].append(i) + def legal(s, a, b): return (s.sp[a] & ~s.sp[b]) == 0 and s.sp[a] != s.sp[b] + def path(s, r, override=None): + """override: dict E idx -> (key, tag) for moved/retagged ops (may include ops of other regs).""" + L = s.ops[r] + if override: + items = [] + for i in L: + k, t = override.get(i, (s.key[i], s.tag[i])); items.append((k, t)) + items.sort() + tags = [t for _, t in items] + else: + tags = [s.tag[i] for i in L] + q = [s.W['initial'][r]] + tags + [s.W['final'][r]] + o = [q[0]] + for c in q[1:]: + if c != o[-1]: o.append(c) + return o + def cost(s, o): + t = 0.0 + for a, b in zip(o, o[1:]): + if not s.legal(a, b): return None + t += f(abs(s.dim[b] - s.dim[a])) + return t + def conflicts(s, i, x): + """does event x fail to commute with ADD event i?""" + e = s.ev[i]; d, src = e[1], e[2]; y = s.ev[x]; op = y[0] + if op in ('INIT', 'CLEAR', 'SCALE'): return y[1] in (d, src) + # ADD/COPY y: dst y1, src y2 + if op == 'COPY' and y[1] in (d, src): return True + if y[2] == d: return True # reads d + if y[1] == src: return True # writes src + return False + def window(s, i): + e = s.ev[i] + if e[0] != 'ADD': return (i, i) + d, src = e[1], e[2] + cand = sorted(set(s.touch[d]) | set(s.touch[src])) + j = bisect.bisect_left(cand, i) + lo = -1 + for x in reversed(cand[:j]): + if s.conflicts(i, x): lo = x; break + hi = len(s.ev) + for x in cand[j + 1:]: + if s.conflicts(i, x): hi = x; break + return (lo, hi) + def rebuild(s, override): + """new word: apply override {E idx: (key, tag)}; regenerate MOVEs of affected public regs only.""" + W = s.W; n = s.n + aff = set() + for i in override: + e = s.ev[i] + for r in ((e[1], e[2]) if e[0] in ('ADD', 'COPY') else (e[1],)): + assert r < n, 'owned touched'; aff.add(r) + # new E order + keys = [override[i][0] if i in override else s.key[i] for i in range(len(s.ev))] + # attach old MOVEs (unaffected regs) to the next non-MOVE event in old order + prog = W['program']; att = defaultdict(list); tail = []; k = 0; pend = [] + for e in prog: + if e[0] == 'MOVE': + if e[1] in aff: continue + pend.append(e) + else: + if pend and k not in override: att[k].extend(pend); pend = [] + k += 1 + tail = pend + order = sorted(range(len(s.ev)), key=lambda i: (keys[i], i)) + evs = [] + for i in order: + e = list(s.ev[i]) + if i in override: + t = override[i][1] + if e[0] in ('ADD', 'COPY'): e[4] = t + elif e[0] == 'SCALE': e[3] = t + evs.append((i, e)) + # MOVE insertion for affected regs: before first op at each new chart + ins = defaultdict(list); endm = [] + for r in aff: + o = s.path(r, override) + opsr = sorted(s.ops[r], key=lambda i: (keys[i], i)) + first = {} + for i in opsr: + t = override[i][1] if i in override else s.tag[i] + first.setdefault(t, i) + for a, b in zip(o, o[1:]): + if b in first: ins[first[b]].append(['MOVE', r, a, b, False]) + else: endm.append(['MOVE', r, a, b, False]) + newp = [] + for i, e in evs: + newp.extend(att.get(i, [])) # NOTE: unaffected MOVEs keep their old successor event + newp.extend(ins.get(i, [])) + newp.append(e) + newp.extend(tail); newp.extend(endm) + W2 = dict(W); W2['program'] = newp + return W2 diff --git a/research/p104-loan-snapshot-engine-on7621/engine/lb.py b/research/p104-loan-snapshot-engine-on7621/engine/lb.py new file mode 100644 index 0000000000..3858353f04 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/lb.py @@ -0,0 +1,249 @@ +# LOANBUILD engine (own code). SYLforge's lifetime device as an exact, solver-backed builder. +# A candidate = path edit (r drops X; partner p inserts B after X [side B] or A before X [side A]) + template ops whose +# coefficients are partly enumerated and partly unknown; the unknowns enter linearly, so the window replay (mod 2^61-1, +# k-dim random projection) is affine in them and is solved exactly, then rationalised and verified by full replay. +# Equality targets: full register state at window end + every pre-existing owned observation (src/dst content at each +# owned-touching op) + every pre-existing CLEAR content inside the window. +import sys, math, bisect, collections, itertools +import os, sys +HERE = os.path.dirname(os.path.abspath(__file__)) +if HERE not in sys.path: sys.path.insert(0, HERE) +from fractions import Fraction as Q +import pricekit as pk # vendored in this directory +PM = 2 ** 61 - 1; K = 4 + +def spanmask(b): + S = {0} + for v in b: S |= {x ^ v for x in S} + m = 0 + for x in S: m |= 1 << x + return m + +_qc = {} +def qmod(s): + c = _qc.get(s) + if c is None: + q = Q(s); c = _qc[s] = q.numerator % PM * pow(q.denominator % PM, -1, PM) % PM + return c + +def ratrec(a, bound=1 << 28): + """rational reconstruction of a mod PM""" + r0, r1, s0, s1 = PM, a % PM, 0, 1 + while r1 >= bound: + qq = r0 // r1; r0, r1 = r1, r0 - qq * r1; s0, s1 = s1, s0 - qq * s1 + if s1 == 0 or abs(s1) >= bound: return None + q = Q(r1, s1) + if (q.numerator - a * q.denominator) % PM: return None + return q + +class Word: + def __init__(s, W, ck=500, seed=1): + s.W = W; s.n = n = W['public']; s.NO = W['owned_fields']; s.N = n + s.NO; P = W['program']; s.T = len(P) + s.sp = [spanmask(b) for b in W['frames']]; s.dim = [len(b) for b in W['frames']] + import random; rng = random.Random(seed) + s.R = [[rng.randrange(1, PM) for _ in range(K)] for _ in range(n)] + # residences of public regs: list of (chart, tin, tout) + s.seg = collections.defaultdict(list); cur = list(W['initial']); tin = [-1] * n + for i, e in enumerate(P): + if e[0] == 'MOVE' and e[1] < n: + r = e[1]; s.seg[r].append((cur[r], tin[r], i)); cur[r] = e[3]; tin[r] = i + for r in range(n): s.seg[r].append((cur[r], tin[r], s.T)) + # ops per register (non-MOVE, public involvement), with chart + s.ops = collections.defaultdict(list) + for i, e in enumerate(P): + if e[0] in ('ADD', 'COPY'): + s.ops[e[1]].append(i); s.ops[e[2]].append(i) + elif e[0] == 'SCALE': s.ops[e[1]].append(i) + # owned live intervals + s.olive = collections.defaultdict(list); st = {} + for i, e in enumerate(P): + if e[0] == 'INIT': st[e[1]] = i + elif e[0] == 'CLEAR': s.olive[e[1]].append((st.pop(e[1]), i)) + s.prof = [0] * (s.T + 1); live = 0 + for i, e in enumerate(P): + if e[0] == 'INIT': live += 1 + elif e[0] == 'CLEAR': live -= 1 + s.prof[i] = live + # checkpoints of the projected state + s.ck = ck; s.cks = [] + M = s.init_state() + for i, e in enumerate(P): + if i % ck == 0: s.cks.append([list(x) for x in M]) + s.step(M, e) + def init_state(s): + return [list(s.R[r]) for r in range(s.n)] + [[0] * K for _ in range(s.NO)] + @staticmethod + def step(M, e, vals=None): + op = e[0] + if op == 'MOVE': return + if op in ('INIT', 'CLEAR'): M[e[1]] = [0] * K; return + if op == 'SCALE': + q = qmod(e[2]) if not str(e[2]).startswith('$') else vals[e[2]] + M[e[1]] = [x * q % PM for x in M[e[1]]]; return + c = e[3]; q = vals[c] if str(c).startswith('$') else qmod(c) + if op == 'COPY': M[e[1]] = [0] * K + if q: M[e[1]] = [(x + q * y) % PM for x, y in zip(M[e[1]], M[e[2]])] + def state_at(s, i): + j = i // s.ck; M = [list(x) for x in s.cks[j]] + for t in range(j * s.ck, i): s.step(M, s.W['program'][t]) + return M + def where(s, r, t): + """chart of public r at original time t (between events: just before event t)""" + for c, a, b in s.seg[r]: + if a < t <= b: return c + return None + def segidx(s, r, X): + return [k for k, (c, a, b) in enumerate(s.seg[r]) if c == X] + def ops_at(s, r, lo, hi): + L = s.ops[r]; i = bisect.bisect_right(L, lo); out = [] + while i < len(L) and L[i] < hi: + out.append(L[i]); i += 1 + return out + def lt(s, a, b): return (s.sp[a] & ~s.sp[b]) == 0 and s.sp[a] != s.sp[b] + +def run_window(Wd, events, s0, vals, track_owned): + """events: list of (key, ev, oi) sorted; replay from original state at s0. Returns (state, obs).""" + M = Wd.state_at(s0); obs = {}; n = Wd.n + for key, e, oi in events: + if oi is not None and oi in track_owned: + op = e[0] + if op == 'CLEAR': obs[(oi, 'c')] = tuple(M[e[1]]) + elif op in ('ADD', 'COPY'): + obs[(oi, 'd')] = tuple(M[e[1]]); obs[(oi, 's')] = tuple(M[e[2]]) + elif op == 'SCALE': obs[(oi, 'd')] = tuple(M[e[1]]) + Wd.step(M, e, vals) + return M, obs + +def owned_track(Wd, s0, e0): + n = Wd.n; P = Wd.W['program']; out = set() + for i in range(s0, e0): + e = P[i] + if e[0] in ('CLEAR',) or (e[0] in ('ADD', 'COPY') and (e[1] >= n or e[2] >= n)) or (e[0] == 'SCALE' and e[1] >= n): + out.add(i) + return out + +def solve_mod(cols, rhs): + """least-squares-free exact solve over GF(PM): find u with sum u_k cols[k] = rhs (all equations). None if inconsistent.""" + nv = len(cols); rows = [] + keys = set(rhs) + for c in cols: keys |= set(c) + for kk in keys: + r = [c.get(kk, 0) % PM for c in cols] + [rhs.get(kk, 0) % PM] + if any(r): rows.append(r) + # gaussian elimination + piv = []; R = rows; ri = 0 + for col in range(nv): + p = None + for j in range(ri, len(R)): + if R[j][col]: p = j; break + if p is None: continue + R[ri], R[p] = R[p], R[ri] + inv = pow(R[ri][col], -1, PM); R[ri] = [x * inv % PM for x in R[ri]] + for j in range(len(R)): + if j != ri and R[j][col]: + f = R[j][col]; R[j] = [(x - f * y) % PM for x, y in zip(R[j], R[ri])] + piv.append(col); ri += 1 + if ri == len(R): break + for j in range(ri, len(R)): + if R[j][nv]: return None + u = [0] * nv + for j, col in enumerate(piv): u[col] = R[j][nv] + return u + +def flat(M, obs, regs=None): + d = {} + for r in range(len(M)): + for k in range(K): + if M[r][k]: d[('s', r, k)] = M[r][k] + for kk, v in obs.items(): + for k in range(K): + if v[k]: d[('o',) + kk + (k,)] = v[k] + return d + +def diffd(a, b): + out = {} + for k in set(a) | set(b): + v = (a.get(k, 0) - b.get(k, 0)) % PM + if v: out[k] = v + return out + +class Cand: + """new program = original with deletions (orig indices), replacements, insertions (key, ev).""" + def __init__(s, Wd): + s.Wd = Wd; s.dele = set(); s.ins = []; s.vars = [] + def add(s, key, ev): s.ins.append((key, list(ev))) + def var(s, name): s.vars.append(name); return name + def events(s, s0, e0): + P = s.Wd.W['program'] + ev = [(float(i), P[i], i) for i in range(s0, e0) if i not in s.dele] + ev += [(k, e, None) for k, e in s.ins] + ev.sort(key=lambda x: x[0]) + return ev + def window(s): + ks = [k for k, e in s.ins] + list(s.dele) + return max(0, int(math.floor(min(ks)))), min(s.Wd.T, int(math.floor(max(ks))) + 2) + def solve(s, fixed=None): + """returns dict var->Fraction or None""" + s0, e0 = s.window(); Wd = s.Wd; tr = owned_track(Wd, s0, e0) + tr -= s.dele + orig = [(float(i), Wd.W['program'][i], i) for i in range(s0, e0)] + Mt, ot = run_window(Wd, orig, s0, {}, tr); target = flat(Mt, ot) + ev = s.events(s0, e0) + vs = list(dict.fromkeys(s.vars)) + z = {v: 0 for v in vs} + M0, o0 = run_window(Wd, ev, s0, z, tr); F0 = flat(M0, o0) + rhs = diffd(target, F0) + if not rhs: return {v: Q(0) for v in vs} + cols = [] + for v in vs: + z1 = dict(z); z1[v] = 1 + M1, o1 = run_window(Wd, ev, s0, z1, tr); cols.append(diffd(flat(M1, o1), F0)) + u = solve_mod(cols, rhs) + if u is None: return None + sol = {} + for v, x in zip(vs, u): + q = ratrec(x) + if q is None: return None + sol[v] = q + # affine check + vals = {v: qmod(str(q)) for v, q in sol.items()} + M2, o2 = run_window(Wd, ev, s0, vals, tr) + if diffd(flat(M2, o2), target): return None + return sol + def instantiate(s, sol): + P = s.Wd.W['program'] + ev = [(float(i), list(P[i])) for i in range(len(P)) if i not in s.dele] + for k, e in s.ins: + e = list(e) + if e[0] in ('ADD', 'COPY') and str(e[3]).startswith('$'): + q = sol[e[3]] + if q == 0: continue + e[3] = str(q) + ev.append((k, e)) + ev.sort(key=lambda x: x[0]) + # drop snapshot lifetimes that became empty (INIT, CLEAR only) + W2 = dict(s.Wd.W); W2['program'] = prune_empty([e for k, e in ev]) + return W2 + +def prune_empty(P): + idx = collections.defaultdict(list); st = {} + for i, e in enumerate(P): + if e[0] == 'INIT': st[e[1]] = [i] + elif e[0] == 'CLEAR': st[e[1]].append(i); idx[e[1]].append(st.pop(e[1])) + elif e[0] != 'MOVE': + for r in ([e[1], e[2]] if e[0] in ('ADD', 'COPY') else [e[1]]): + if r in st: st[r].append(i) + drop = set() + for o, L in idx.items(): + for ix in L: + if len(ix) == 2: drop |= set(ix) + return [e for i, e in enumerate(P) if i not in drop] + +def free_label(Wd, lo, hi, used): + """owned label not live in [lo,hi] (orig keys) and not already used by this candidate in an overlapping way""" + n = Wd.n + if max(Wd.prof[int(lo):int(hi) + 1] + [0]) + 1 + len(used) > 50: return None + for o in range(n, n + Wd.NO): + if o in used: continue + if all(b < lo or a > hi for a, b in Wd.olive.get(o, [])): return o + return None diff --git a/research/p104-loan-snapshot-engine-on7621/engine/pricekit.py b/research/p104-loan-snapshot-engine-on7621/engine/pricekit.py new file mode 100644 index 0000000000..260a190a1c --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/pricekit.py @@ -0,0 +1,489 @@ +#!/usr/bin/env python3 +"""PRICEKIT: the one canonical exact pricer for P104 caller+native invoices. Standard library only. + +Accounting contract (README.md has the one-page version). Units: + rank r job a paid job of rank r (P[r] += 1) or a productive target of rank r (T[r] += 1). + S rank-mass surplus sum_r r (T_r - P_r) (= F(0)). + F(t) the bill sum_r (T_r - P_r) r^(1-t). + kappa the unique root of F on (0, hi]; certified by an interval enclosure (Decimal, error-bounded signs). + D0 sum (T-P) r ln r (= -F'(0); the HOSTCRIB/MAINRETIME/CALLERMAP "D", e.g. #6934 422589.15263423) + Dk sum (T-P) r^(1-kappa) ln r (= -F'(kappa); the DSPLIT/ARCHEQ dk.py/TRADELAW "D(kappa)") + +Charging rules (caller events, '#7113 format' = schema mixed-root-caller/1): + port count every caller event of type A/B is charged once per port of that type (len(types[t]['ports'])). + QUERY P[1] per port (thin unit / sign query). + IDLE_ONE s P[8|s|] per port (one idle over the stage set s). + IDLE_ODD s P[8|s|+1] per port. + LAMBDA ordinary work, 1000 units per port-use (#6913). lambda_mode='paid_rank1' charges P[1] per port + instead (counterfactual only). + ADD/SWAP/SCALE not invoiced (ordinary only). + diagonal row T[(q mod 2) + 8*sum(stage bits)] per port: a field of degree d is a target of rank 8d+1 (odd phase) + or 8d (even phase). Hubs are fields with d = #stages (r137 at 17 stages). + J (one stage) ONE native word per J event: stage = call = J event = one copy of the native word. The native word + serves all 104 ports internally, so it is NOT multiplied by ports. +Native word (schema p104-accumulated-block-composition/2): + paid P[|dim b - dim a|] per MOVE. + helpers T[dim final - dim initial] per public register that is not a main. + mains live X (r), live Y (r+112), hosted X2, Y2. A port with a hosted plane (r in hosted.Dports) has 4 + mains, otherwise 2. Mains are never in T: their climbs are paid (MOVEs) and their credit is the + caller diagonal. Ledger law: native mains per port per stage == caller slots covered per stage + (= mask rank - idle incidences); check_slots() tests it. +Parametric native (native_param): start from a word or a histogram native and + drop_mains_A/B k mains per A/B port per call removed, priced by main_unit: + 'rank1x8' (8 rank-1 jobs, the ARMS/MONOCALLER/SLOT3 unit), 'rank8' (one rank-8 job), or exact via + drop_registers() on a real word (preferred; this is not parametric). + c_A/c_B extra owned rank-1 paid jobs per A/B port per call (may be a Fraction). + dT/dP helper/paid histogram deltas per call. +Every switch is an explicit argument; nothing reads the environment. +""" +import gzip, json, math +from collections import Counter +from decimal import Decimal, localcontext +from fractions import Fraction as Q + +VERSION = 'pricekit/1 (2026-11-11)' +NPORTS = 104 + +# ------------------------------------------------------------------------------------------- io +def load(path): + op = gzip.open if str(path).endswith('.gz') else open + with op(path, 'rt') as f: + return json.load(f) + +def hist(d): + """dict with str/int keys -> Counter{int: int|Fraction}; zero entries dropped.""" + return Counter({int(k): (Q(v) if isinstance(v, str) else v) for k, v in d.items() if v}) + +def _add(dst, src, mult=1): + for r, v in src.items(): + dst[r] += mult * v + return dst + +# ------------------------------------------------------------------------------------------- native +def native_from_word(W, verify=False): + """Exact native invoice of one word (one call). verify=True also replays the word exactly (slow).""" + n = len(W['initial']); live = set(W['live']); H = W.get('hosted') or {} + X2, Y2 = list(H.get('X2', [])), list(H.get('Y2', [])) + main = live | {r + 112 for r in live} | set(X2) | set(Y2) + dim = [len(b) for b in W['frames']] + paid = Counter(); main_paid = Counter(); cls = {} + for r in live: cls[r] = 'X'; cls[r + 112] = 'Y' + for r in X2: cls[r] = 'X2' + for r in Y2: cls[r] = 'Y2' + by_class = {k: Counter() for k in ('X', 'Y', 'X2', 'Y2')} + for e in W['program']: + if e[0] == 'MOVE': + rk = abs(dim[e[3]] - dim[e[2]]); paid[rk] += 1 + if e[1] in cls: by_class[cls[e[1]]][rk] += 1 + helpers = Counter(dim[W['final'][r]] - dim[W['initial'][r]] for r in range(n) if r not in main) + dports = set(H.get('Dports', [])) + mains_per_port = {p: 2 + 2 * (p in dports and bool(X2)) for p in range(NPORTS)} + out = dict(T=+helpers, P=+paid, public=n, events=len(W['program']), mains=len(main), main_moves=by_class, + mains_per_port=mains_per_port, Dports=sorted(dports), source='word') + if verify: + out['replay'] = replay(W)['ok'] + return out + +def native_from_hist(paid, helpers, Dports=None, mains_A=4, mains_B=2): + dp = set(Dports) if Dports is not None else None + mpp = None if dp is None else {p: (mains_A if p in dp else mains_B) for p in range(NPORTS)} + return dict(T=hist(helpers), P=hist(paid), mains_per_port=mpp, Dports=sorted(dp) if dp else None, source='hist') + +def native_param(base, nA, nB, drop_mains_A=0, drop_mains_B=0, main_unit='rank1x8', c_A=0, c_B=0, dT=None, dP=None, + allow_negative=False): + """Parametric native per call. base = native dict. Returns a new native dict (source='param').""" + T = Counter(base['T']); P = Counter(base['P']) + unit = {'rank1x8': {1: 8}, 'rank8': {8: 1}}[main_unit] + for ports, k in ((nA, drop_mains_A), (nB, drop_mains_B)): + for r, v in unit.items(): + P[r] -= ports * k * v + P[1] += nA * Q(c_A) + nB * Q(c_B) + _add(T, dT or {}); _add(P, dP or {}) + mpp = base.get('mains_per_port') + if mpp is not None: + dp = set(base['Dports'] or []) + mpp = {p: m - (drop_mains_A if p in dp else drop_mains_B) for p, m in mpp.items()} + for r in list(P): # a negative native count is legal only if the caller absorbs it (invoice() re-checks) + if P[r] < 0 and not allow_negative: raise ValueError('native_param: negative paid count at rank %d' % r) + return dict(T=T, P=P, mains_per_port=mpp, Dports=base.get('Dports'), source='param', + param=dict(drop_mains_A=drop_mains_A, drop_mains_B=drop_mains_B, main_unit=main_unit, c_A=str(c_A), c_B=str(c_B))) + +# ------------------------------------------------------------------------------------------- exact word replay / pruning +def _span(b): + return frozenset(b) + +def replay(W): + """Exact replay (Fractions, sparse rows). Returns dict(ok, rows, frames, problems).""" + n = W['public']; N = n + W['owned_fields']; frames = W['frames'] + fs = [set(f) for f in frames] + rows = [{r: Q(1)} if r < n else {} for r in range(N)] + cur = list(W['initial']) + [None] * W['owned_fields']; owned = set(); bad = [] + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + if op == 'MOVE': + a, b, priv = e[2:] + if cur[r] != a: bad.append(('move-from', i)); + cur[r] = b + elif op == 'INIT': + owned.add(r); rows[r] = {}; cur[r] = e[2] + elif op == 'CLEAR': + owned.discard(r); rows[r] = {}; cur[r] = None + elif op == 'SCALE': + q = Q(e[2]) + if cur[r] != e[3]: bad.append(('scale-frame', i)) + rows[r] = {k: v * q for k, v in rows[r].items()} + elif op in ('ADD', 'COPY'): + s = e[2]; q = Q(e[3]); f = e[4] + if cur[r] != f or cur[s] != f: bad.append(('add-frame', i)) + if op == 'COPY' and rows[r]: bad.append(('copy-nonzero', i)) + R = rows[r] + for k, v in rows[s].items(): + nv = R.get(k, 0) + q * v + if nv: R[k] = nv + else: R.pop(k, None) + else: + bad.append(('op', i, op)) + if owned: bad.append(('owned-alive', len(owned))) + if cur[:n] != list(W['final']): bad.append(('exit-frames',)) + return dict(ok=not bad, rows=rows, cur=cur, problems=bad[:20], nproblems=len(bad)) + +def drop_registers(W, drop): + """Exact dead-code deletion of the public registers in `drop` (e.g. hosted X2/Y2). + Keeps every event that some kept public output depends on; removes whole owned lifetimes with no live use. + Refuses (ValueError) if a dropped register feeds a kept output. Returns (W2, report).""" + drop = set(drop); n = W['public']; N = n + W['owned_fields']; PR = W['program'] + need_mask = 0 + dem = [0] * N + for r in range(n): + if r not in drop: dem[r] = 1 << r + keep = [False] * len(PR); feeds_dropped = 0 + for i in range(len(PR) - 1, -1, -1): + e = PR[i]; op, t = e[0], e[1] + if op in ('INIT', 'CLEAR'): + dem[t] = 0; continue + if op == 'MOVE': + continue + if dem[t]: + keep[i] = True + if op in ('ADD', 'COPY'): + s = e[2] + if s in drop: feeds_dropped += 1 + dem[s] |= dem[t] + if op == 'COPY': dem[t] = 0 + if feeds_dropped: + raise ValueError('drop_registers: %d kept events read a dropped register' % feeds_dropped) + # lifetime decisions for owned registers + life_keep = {}; open_life = {} + for i, e in enumerate(PR): + op, t = e[0], e[1] + if t >= n: + if op == 'INIT': open_life[t] = i; life_keep[i] = False + elif op in ('ADD', 'COPY', 'SCALE') and keep[i]: life_keep[open_life[t]] = True + if op in ('ADD', 'COPY') and keep[i] and e[2] >= n: life_keep[open_life[e[2]]] = True + life_of = {}; open_life = {} + for i, e in enumerate(PR): + op, t = e[0], e[1] + if t >= n: + if op == 'INIT': open_life[t] = i + life_of[i] = open_life[t] + out = []; dropped_moves = Counter(); dropped_lifetimes = 0; dim = [len(b) for b in W['frames']] + for i, e in enumerate(PR): + op, t = e[0], e[1] + if t in drop: + if op == 'MOVE': dropped_moves[abs(dim[e[3]] - dim[e[2]])] += 1 + continue + if t >= n: + if not life_keep[life_of[i]]: + if op == 'MOVE': dropped_moves[abs(dim[e[3]] - dim[e[2]])] += 1 + if op == 'INIT': dropped_lifetimes += 1 + continue + if op in ('INIT', 'CLEAR', 'MOVE') or keep[i]: out.append(e) + continue + if op == 'MOVE' or keep[i]: out.append(e) + W2 = dict(W); W2['program'] = out + H = dict(W.get('hosted') or {}) + if drop >= set(H.get('X2', [])) | set(H.get('Y2', [])) and H.get('X2'): + H['X2'] = []; H['Y2'] = []; H['dropped_hosted_plane'] = True + W2['hosted'] = H + rep = dict(events_before=len(PR), events_after=len(out), dropped_moves=dict(dropped_moves), + dropped_owned_lifetimes=dropped_lifetimes) + return W2, rep + +def zero_registers(W, drop): + """Hard deletion: registers in `drop` cease to exist (rows = 0). Every event that targets OR reads them is + removed, then dead code is pruned with drop_registers(). Only valid if verify_drop() shows every kept public row + unchanged; otherwise the word genuinely uses the dropped registers and a rebuild is needed.""" + drop = set(drop) + W1 = dict(W); W1['program'] = [e for e in W['program'] if not (e[1] in drop and e[0] != 'MOVE') + and not (e[0] in ('ADD', 'COPY') and e[2] in drop)] + removed_reads = sum(1 for e in W['program'] if e[0] in ('ADD', 'COPY') and e[2] in drop and e[1] not in drop) + W2, rep = drop_registers(W1, drop); rep['removed_reads'] = removed_reads + return W2, rep + +def verify_drop(W, W2, drop): + """Replay both words exactly; every kept public row and exit frame must be unchanged.""" + a = replay(W); b = replay(W2); n = W['public'] + same = all(a['rows'][r] == b['rows'][r] for r in range(n) if r not in set(drop)) + frames = all(b['cur'][r] == W['final'][r] for r in range(n)) + return dict(original_ok=a['ok'], pruned_ok=b['ok'], pruned_problems=b['problems'], kept_rows_equal=same, exit_frames=frames) + +# ------------------------------------------------------------------------------------------- caller +def caller_ports(C): + return {t: len(v['ports']) for t, v in C['types'].items()} + +def caller_invoice(C, lambda_mode='ordinary', only=None): + """Caller part of the invoice (no native). only='A'/'B' restricts to one port type. + Returns dict(T, P, nJ, ports, lambda_uses, ops).""" + assert lambda_mode in ('ordinary', 'paid_rank1') + ports = caller_ports(C); T = Counter(); P = Counter(); nJ = 0; lam = 0; ops = Counter() + for e in C['events']: + if e[0] == 'J': + nJ += 1; continue + ty, op = e[0], e[1] + if only and ty != only: continue + k = ports[ty]; ops[(ty, op)] += 1 + if op == 'QUERY': P[1] += k + elif op == 'IDLE_ONE': P[8 * len(e[3])] += k + elif op == 'IDLE_ODD': P[8 * len(e[3]) + 1] += k + elif op == 'LAMBDA': + lam += k + if lambda_mode == 'paid_rank1': P[1] += k + elif op in ('ADD', 'SWAP', 'SCALE', 'HOST'): pass + else: raise ValueError('caller op not in the alphabet: %r' % (op,)) + for ty, t in C['types'].items(): + if only and ty != only: continue + for dg in t['diagonal']: + T[int(dg[0]) % 2 + 8 * sum(dg[2:])] += ports[ty] + T[0] = 0; T = +T + return dict(T=T, P=+P, nJ=nJ, ports=ports, lambda_uses=lam, ops=ops) + +def check_slots(C, native=None): + """Per type and stage: mask rank, idle incidences, covered = mask - idles. With a native, compare covered with + the native's mains per port (the ledger law d = m).""" + nst = C['stages']; res = {} + for ty, t in C['types'].items(): + mask = [0] * (nst + 1); idle = [0] * (nst + 1) + for dg in t['diagonal']: + for k, b in enumerate(dg[2:], 1): + if b: mask[k] += 1 + for e in C['events']: + if e[0] == ty and e[1] in ('IDLE_ONE', 'IDLE_ODD'): + for k in e[3]: idle[k] += 1 + cov = [mask[k] - idle[k] for k in range(1, nst + 1)] + res[ty] = dict(mask=mask[1:], idle=idle[1:], covered=cov, header=sum(mask), idle_incidences=sum(idle)) + if native is not None and native.get('mains_per_port'): + mpp = native['mains_per_port'] + for ty, t in C['types'].items(): + ms = {mpp[p] for p in t['ports']} + res[ty]['native_mains'] = sorted(ms) + res[ty]['law_d_eq_m'] = (len(ms) == 1 and all(c == next(iter(ms)) for c in res[ty]['covered'])) + return res + +def caller_from_roots(fields, hosts_per_stage_cap, thin, ports, nst): + """Parametric caller (rule of our #6934 package, check.py outer()). fields = list of (mask stages, host stages); + a field of degree d is a target 8d+1; its unhosted mask stages are idled in maximal consecutive runs (P[8*run]); + a stage hosting more than hosts_per_stage_cap[k] fields pays one rank-8 idle per excess; thin rank-1 queries.""" + T = Counter(); P = Counter(); hc = Counter() + for ms, host in fields: + T[8 * len(ms) + 1] += ports + hs = set(host) + for k in hs: hc[k] += 1 + run = 0; prev = None + for k in sorted(ms): + if k in hs: + if run: P[8 * run] += ports + run = 0; prev = None; continue + if prev is not None and k == prev + 1: run += 1 + else: + if run: P[8 * run] += ports + run = 1 + prev = k + if run: P[8 * run] += ports + for k in range(nst): + if hc[k] > hosts_per_stage_cap[k]: P[8] += ports * (hc[k] - hosts_per_stage_cap[k]) + P[1] += ports * thin + return dict(T=+T, P=+P) + +# ------------------------------------------------------------------------------------------- invoice and price +def invoice(caller, native, calls=None): + """caller: dict from caller_invoice/caller_from_roots; native: native dict. calls defaults to caller nJ.""" + L = caller.get('nJ') if calls is None else calls + if not L: raise ValueError('invoice: number of native calls unknown (no J events and calls=None)') + T = Counter(caller['T']); P = Counter(caller['P']) + _add(T, native['T'], L); _add(P, native['P'], L) + for C_ in (T, P): + for r in list(C_): + if C_[r] == 0: del C_[r] + elif C_[r] < 0: raise ValueError('invoice: negative count at rank %d' % r) + return dict(T=T, P=P, calls=L) + +def S_of(T, P): + return sum(r * (T.get(r, 0) - P.get(r, 0)) for r in set(T) | set(P)) + +def _dec(x): + x = Q(x); return Decimal(x.numerator) / Decimal(x.denominator) + +def _coef(T, P): + return sorted((r, Q(T.get(r, 0)) - Q(P.get(r, 0))) for r in set(T) | set(P) if Q(T.get(r, 0)) != Q(P.get(r, 0))) + +def _F(cs, t, prec): + """F(t) and a rigorous bound on its evaluation error at this precision.""" + with localcontext() as c: + c.prec = prec; t = Decimal(t) + val = Decimal(0); mag = Decimal(0) + for r, k in cs: + p = (Decimal(r).ln() * (1 - t)).exp(); kd = _dec(k) + val += kd * p; mag += abs(kd) * p + err = mag * Decimal(10) ** (-(prec - 6)) + return val, err + +def _sign(cs, t, prec=60): + while prec <= 1200: + v, e = _F(cs, t, prec) + if abs(v) > e: return 1 if v > 0 else -1 + prec *= 2 + raise ArithmeticError('sign of F undetermined at t=%s' % t) + +def _unique(cs, hi, depth=14): + """Certify at most one root of F on [0, hi] by interval bounds (F excluded from 0, or F' < 0 on the piece).""" + with localcontext() as c: + c.prec = 40 + def pw(r, t): return (Decimal(r).ln() * (1 - Decimal(t))).exp() + def piece(a, b): + Fl = sum(_dec(k) * (pw(r, b) if k > 0 else pw(r, a)) for r, k in cs) + Fh = sum(_dec(k) * (pw(r, a) if k > 0 else pw(r, b)) for r, k in cs) + if Fl > 0 or Fh < 0: return 'noroot' + Dl = sum(_dec(k) * Decimal(r).ln() * (pw(r, b) if k > 0 else pw(r, a)) for r, k in cs if r > 1) + return 'mono' if Dl > 0 else 'unknown' + stack = [(Decimal(0), Decimal(hi), 0)]; mono = [] + while stack: + a, b, d = stack.pop(); s = piece(a, b) + if s == 'noroot': continue + if s == 'mono': mono.append((a, b)); continue + if d >= depth: return False + m = (a + b) / 2; stack += [(a, m, d + 1), (m, b, d + 1)] + mono.sort() + return all(mono[i][1] == mono[i + 1][0] for i in range(len(mono) - 1)) # one contiguous decreasing block + +def kappa(T, P, digits=50): + """Certified enclosure of the crossing. Returns dict(lo, hi, S, status). status NO_CROSSING if S <= 0.""" + cs = _coef(T, P); S = S_of(T, P) + if S <= 0: return dict(status='NO_CROSSING', S=S, lo=None, hi=None) + hi = Decimal('0.001') + while _sign(cs, hi) > 0: + hi *= 2 + if hi > 64: return dict(status='NO_SIGN_CHANGE', S=S, lo=None, hi=None) + lo = Decimal(0); uniq = _unique(cs, hi) + width = Decimal(10) ** (-digits - 2) + with localcontext() as c: + c.prec = digits + 30 + while hi - lo > width: + m = (lo + hi) / 2 + if _sign(cs, m, digits + 30) > 0: lo = m + else: hi = m + return dict(status='OK' if uniq else 'OK_UNIQUENESS_UNPROVED', S=S, lo=lo, hi=hi, unique=uniq) + +def Dsum(T, P, t=0, prec=60): + with localcontext() as c: + c.prec = prec; t = Decimal(t) + return sum(_dec(k) * Decimal(r).ln() * (Decimal(r).ln() * (1 - t)).exp() for r, k in _coef(T, P) if r > 1) + +def F_at(T, P, t, prec=60): + return _F(_coef(T, P), t, prec)[0] + +def price(inv, digits=50): + T, P = inv['T'], inv['P']; k = kappa(T, P, digits) + out = dict(S=S_of(T, P), paid_jobs=sum(P.values()), paid_positions=sum(r * v for r, v in P.items()), + targets=sum(T.values()), target_positions=sum(r * v for r, v in T.items()), kappa=k, + D0=Dsum(T, P, 0)) + if k.get('lo') is not None: out['Dk'] = Dsum(T, P, k['lo']) + return out + +def kstr(k, n=30): + if k.get('lo') is None: return k['status'] + return str(k['lo'])[:n + 2] + +def break_even_c(T, P, kappa_ref, ports_calls): + """Linear break-even: extra rank-1 paid per (port x call) at which kappa == kappa_ref. F(kref; c) = F0 - c*ports_calls.""" + return F_at(T, P, kappa_ref) / ports_calls + +# ------------------------------------------------------------------------------------------- ordinary units (#6934 package reading) +def word_ordinary(W): + """Per-call ordinary terms of a native word (allowance, reviewed upper, routes): our #6934 jclib.word_terms.""" + H = W['hosted']; nD = len(H['Dports']); n = len(W['initial']); live = set(W['live']) + main = live | {r + 112 for r in live} | set(H.get('X2', [])) | set(H.get('Y2', [])) + dim = [len(b) for b in W['frames']] + paid = Counter(abs(dim[e[3]] - dim[e[2]]) for e in W['program'] if e[0] == 'MOVE') + hel = Counter(dim[W['final'][r]] - dim[W['initial'][r]] for r in range(n) if r not in main) + life = sum(e[0] == 'INIT' for e in W['program']); allow = upper = 0 + for e in W['program']: + if e[0] in ('INIT', 'CLEAR'): allow += 2 + elif e[0] == 'SCALE': + q = Q(e[2]); a, d = abs(q.numerator), q.denominator + allow += 2 * (abs(a.bit_length() - d.bit_length()) + int(q < 0)); upper += 4 * (a.bit_length() + d.bit_length() + 1) + elif e[0] in ('ADD', 'COPY'): + q = Q(e[3]); a, d = abs(q.numerator), q.denominator + allow += 2 * (3 + a.bit_length() - 1 + bin(a).count('1') + d.bit_length() - 1); upper += 4 * (a.bit_length() + bin(a).count('1') + d.bit_length() + 1) + routes = (sum(paid.values()) + 2 * sum(hel.values()) + 208 + 2 * nD + 4 * 104 + 4 * nD + 4 * life) * 1000 + return dict(allow=allow, upper=upper, routes=routes, n=n, nD=nD) + +def _swaps(sel): + wanted = set(range(len(sel))); return len(list(zip(sorted(set(sel) - wanted), sorted(wanted - set(sel))))) + +def _shim(C, ty): + t = C['types'][ty]; off = t['offset']; ev = [] + for e in C['events']: + if e[0] == 'J': ev.append(['HOST', e[1]]) + elif e[0] == ty: + e = list(e[1:]) + if e[0] in ('ADD', 'SWAP', 'QUERY'): e[1] -= off; e[2] -= off + else: e[1] -= off + ev.append(e) + return dict(events=ev, roles=t['roles'], diagonal=t['diagonal']) + +def _rest(C, ports): + ev = C['events']; roles = C['roles']; mx = mn = mp = 0 + for role, (qp, zero, *sel) in enumerate(C['diagonal']): + axes = ([0] if qp else []) + [a for st, y in enumerate(sel, 1) if y for a in range(9 + 8 * (st - 1), 9 + 8 * st)] + k = _swaps(axes); mx += 2 * k * ports; mn += 2 * bool(qp) * ports; mp += int(qp == -1) * ports + ie = qn = qph = 0 + for e in ev: + if e[0] in ('IDLE_ONE',): + ie += 2 * _swaps([a for st in e[2] for a in range(9 + 8 * (st - 1), 9 + 8 * st)]) * ports + elif e[0] == 'QUERY': + qn += 2 * ports; qph += int(e[3] == -1) * ports + return (32 * roles * ports * len(ev) + sum(e[0] == 'QUERY' for e in ev) * 16 * ports + roles * ports * 26 + + (mx + mn + mp) * 1000 + ie * 1000 + (qn + qph) * 1000) + +def ordinary(C, W, odd_scale=1125): + """Raw ordinary readings of the #6934 package (literal/actual/upper/recharge/per_input). IDLE_ODD is not in that + formula (#7113-era callers): such callers get a flag, not a number for the IDLE_ODD routes.""" + t = word_ordinary(W); ports = caller_ports(C); L = sum(1 for e in C['events'] if e[0] == 'J') + rest = _rest(_shim(C, 'A'), ports['A']) + _rest(_shim(C, 'B'), ports['B']) + lam = sum(ports[e[0]] for e in C['events'] if e[0] != 'J' and e[1] == 'LAMBDA'); debit = 1000 * lam + sc = 4 * (odd_scale.bit_length() + 2) + 4 * (odd_scale.bit_length() + 2) + per_in = L * t['n'] * sc + lit = L * 10 ** 7 + rest; act = L * (t['allow'] + t['routes']) + rest; up = L * (t['upper'] + t['routes']) + rest + return dict(literal=lit, actual=act, upper=up, recharge=up + debit, per_input=up + per_in, per_input_recharge=up + per_in + debit, + lambda_uses=lam, lambda_debit=debit, idle_odd_present=any(e[0] != 'J' and e[1] == 'IDLE_ODD' for e in C['events'])) + +# ------------------------------------------------------------------------------------------- one-call convenience +def run(caller_path, native, lambda_mode='ordinary', digits=50): + C = load(caller_path); ci = caller_invoice(C, lambda_mode) + inv = invoice(ci, native); pr = price(inv, digits) + return dict(caller=ci, invoice=inv, price=pr, slots=check_slots(C, native)) + +def cite(label, pr): + """One-line citation string for RESULT.md files.""" + k = pr['kappa'] + return '[PRICEKIT %s] %s: S=%s kappa=%s D0=%.4f Dk=%s paid=%d' % ( + VERSION, label, pr['S'], kstr(k, 26), float(pr['D0']), ('%.8f' % float(pr['Dk'])) if 'Dk' in pr else '-', pr['paid_jobs']) + +if __name__ == '__main__': + import sys + if len(sys.argv) < 3: + print('usage: pricekit.py CALLER.json WORD.json[.gz] | NATIVE-RESULT.json'); sys.exit(1) + X = load(sys.argv[2]) + nat = native_from_word(X) if 'program' in X else native_from_hist(X['native']['paid'], X['native']['helpers']) + R = run(sys.argv[1], nat) + print(cite(sys.argv[2], R['price'])) diff --git a/research/p104-loan-snapshot-engine-on7621/engine/run7621.py b/research/p104-loan-snapshot-engine-on7621/engine/run7621.py new file mode 100644 index 0000000000..4cd8d10e92 --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/run7621.py @@ -0,0 +1,59 @@ +# Greedy loan+snapshot stacking on #7621 over the shape census, gain-ordered, re-censusing after each pass. Own code. +import sys, time, math, json, gzip, collections, pickle, resource +import os, sys +HERE = os.path.dirname(os.path.abspath(__file__)) +if HERE not in sys.path: sys.path.insert(0, HERE) +import lb, tpl, check +import pricekit as pk +import jl +BASE=pk.load(sys.argv[1]); out=sys.argv[2]; maxpass=int(sys.argv[3]) if len(sys.argv)>3 else 6 +f=lambda x: x*math.log(x) if x>0 else 0.0 +def census(Wd): + W=Wd.W; dim=Wd.dim; P=W['program']; out=[] + for r in range(Wd.n): + S=Wd.seg[r] + for k in range(1,len(S)-1): + X=S[k][0]; info=tpl.r_xinfo(Wd,r,X) + if info is None: continue + A,B=info[0],info[1]; cl=tpl.classify(Wd,r,info[6]) + if cl is None: continue + Wr,Rd=cl; g=f(dim[B]-dim[A])-f(dim[X]-dim[A])-f(dim[B]-dim[X]) + srcs={s for _,s,_ in Wr} + if len(srcs)==1: + p=next(iter(srcs)); js=Wd.segidx(p,X) + if len(js)==1 and js[0]+11e-9: out.append((g-c,r,X,p,'B')) + if len(Rd)==1: + p=Rd[0][1]; js=Wd.segidx(p,X) + if len(js)==1 and js[0]>0: + Z=Wd.seg[p][js[0]-1][0] + if Wd.lt(Z,A): + c=f(dim[X]-dim[Z])-f(dim[A]-dim[Z])-f(dim[X]-dim[A]) + if g-c>1e-9: out.append((g-c,r,X,p,'A')) + return sorted(out,reverse=True) +def kap(W): return jl.kappa(W)['kappa']['lo'] +W=BASE; Wd=lb.Word(W); k0=kap(W); kc=k0; t0=time.time(); log=[]; landed=0 +print('base kappa',k0,flush=True) +for ps in range(maxpass): + cs=census(Wd); print('pass',ps,'census',len(cs),'sum net %.2f'%sum(x[0] for x in cs),flush=True) + acc=0; why=collections.Counter() + for net,r,X,p,side in cs: + if resource.getrusage(resource.RUSAGE_SELF).ru_maxrss>3.8e9: print('MEM BAIL'); break + if tpl.r_xinfo(Wd,r,X) is None: continue # stale after an earlier hit + res,msg=tpl.try_instance(Wd,r,X,p,side,maxc=8) + if res is None: why[str(sorted(msg)) if isinstance(msg,dict) else msg]+=1; continue + W2,cd,sol=res; c=check.full(W,W2) + if not (c['rows_ok'] and c['base_lifetimes_kept'] and c['peak']<=50): why['check']+=1; continue + k2=kap(W2) + if not k2>kc: why['no kappa gain']+=1; continue + W=W2; Wd=lb.Word(W); kc=k2; acc+=1; landed+=1 + rec=dict(r=r,X=X,p=p,side=side,net=net,c=cd.meta['c'],sol={a:str(b) for a,b in sol.items()},kappa=str(k2),new_lifetimes=c['new_lifetimes']) + log.append(rec); print('HIT',landed,rec,'[%.0fs]'%(time.time()-t0),flush=True) + if landed%5==0: json.dump(W,gzip.open(out+'_tmp.json.gz','wt')) + print('pass',ps,'accepted',acc,'kappa',kc,'x%.8f'%(float(kc)/float(k0)),dict(why.most_common(8)),'[%.0fs]'%(time.time()-t0),flush=True) + if not acc: break +json.dump(W,gzip.open(out+'_%d.json.gz'%landed,'wt')); json.dump(log,open(out+'_log.json','w'),indent=0) +print('DONE landed',landed,'kappa',kc,'vs base',k0) diff --git a/research/p104-loan-snapshot-engine-on7621/engine/run_on.py b/research/p104-loan-snapshot-engine-on7621/engine/run_on.py new file mode 100644 index 0000000000..adf0c5881f --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/run_on.py @@ -0,0 +1,33 @@ +#!/usr/bin/env python3 +"""One-command reproduction: census -> greedy loan + snapshot passes -> re-census, on any P104 native word. + + python3 engine/run_on.py WORD.json.gz [OUT_PREFIX] [MAX_PASSES] + +Runs run7621.py (exact two-shape census of the loan + stationary-snapshot move, greedy gain-ordered +try_instance over it, each hit accepted only if check.full passes, owned peak <= 50 and the PRICEKIT kappa rises, +then a re-census) and writes OUT_PREFIX_.json.gz (n = instances landed) and OUT_PREFIX_log.json. +Standard library only; one process; under 1 GB. On #7621's word it lands the 14 instances of hit_loan_14. +""" +import os +import runpy +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) + + +def main(argv): + if len(argv) < 2: + print(__doc__) + return 2 + word = os.path.abspath(argv[1]) + out = os.path.abspath(argv[2]) if len(argv) > 2 else os.path.join(tempfile.mkdtemp(prefix='loan-'), 'out') + maxpass = argv[3] if len(argv) > 3 else '6' + sys.argv = [os.path.join(HERE, 'run7621.py'), word, out, maxpass] + print('output prefix', out, flush=True) + runpy.run_path(sys.argv[0], run_name='__main__') + return 0 + + +if __name__ == '__main__': + sys.exit(main(sys.argv)) diff --git a/research/p104-loan-snapshot-engine-on7621/engine/tpl.py b/research/p104-loan-snapshot-engine-on7621/engine/tpl.py new file mode 100644 index 0000000000..10b5926d9f --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/engine/tpl.py @@ -0,0 +1,240 @@ +# LOANBUILD templates (own code). Side B: partner p (the source of r's X-write) inserts B after X; loan c += beta*r at A, +# c's existing X-transfer into p carries rho to X, readers re-read p (or a stationary snapshot of p), settles at B. +# Side A: partner p (the reader of r at X) inserts A before X; p reads r at A, signed role swap r += b*p, loan r += beta*c +# at A, p picks up r's X-writes from their sources (or stationary snapshots), settle r += eps*c at B. +import collections, math +from fractions import Fraction as Q +import lb + +def fr(x): return Q(x) + +def r_xinfo(Wd, r, X): + """(A,B,kr0,k1,k2,k3, xops) for r's interior stop X, or None""" + S = Wd.seg[r]; ks = Wd.segidx(r, X) + if len(ks) != 1: return None + k = ks[0] + if k == 0 or k + 1 >= len(S): return None + (A, kr0, k1), (_, _, k2), (B, _, k3) = S[k - 1], S[k], S[k + 1] + return A, B, kr0, k1, k2, k3, Wd.ops_at(r, k1, k2) + +def carriers(Wd, A, X, B, excl): + out = [] + for q in range(Wd.n): + if q in excl: continue + S = Wd.seg[q] + for j in range(len(S) - 2): + if S[j][0] == A and S[j + 1][0] == X and S[j + 2][0] == B: + out.append((q, S[j][1], S[j][2], S[j + 1][2], S[j + 2][2])) + return out + +def classify(Wd, r, xops): + P = Wd.W['program']; W = []; R = [] + for i in xops: + e = P[i] + if e[0] != 'ADD': return None + if e[1] == r and e[2] != r and e[2] < Wd.n: W.append((i, e[2], e[3])) + elif e[2] == r and e[1] != r and e[1] < Wd.n: R.append((i, e[1], e[3])) + else: return None + return W, R + +def build_B(Wd, r, X, p, c_info, beta, snap=False): + P = Wd.W['program']; info = r_xinfo(Wd, r, X) + if info is None: return None, 'r path' + A, B, kr0, k1, k2, k3, xops = info + cl = classify(Wd, r, xops) + if cl is None: return None, 'r ops' + Wr, Rd = cl + if not Wr or any(s != p for _, s, _ in Wr): return None, 'writes not from p' + if any(t == p for _, t, _ in Rd): return None, 'p reads r' + js = Wd.segidx(p, X) + if len(js) != 1: return None, 'p path' + j = js[0]; Sp = Wd.seg[p] + if j + 1 >= len(Sp): return None, 'p final at X' + _, pa, pb = Sp[j]; C = Sp[j + 1][0] + if not Wd.lt(B, C) or not Wd.lt(X, B): return None, 'nest' + pXops = Wd.ops_at(p, pa, pb) + q, cA0, ca1, ca2, ca3 = c_info + rA = Wd.ops_at(r, kr0, k1); rLA = rA[-1] if rA else kr0 + rAw = [i for i in rA if P[i][1] == r] + if not kr0 < ca1: return None, 'loan time' + tL = ca1 - 0.5; slide = [i for i in rAw if i > tL]; gen = False + for m, i in enumerate(slide): + s = P[i][2] + if (P[i][0] != 'ADD' or Wd.where(s, ca1) != A or [j for j in Wd.ops_at(s, ca1 - 1, i) if P[j][1] == s] + or [j for j in Wd.ops_at(r, ca1 - 1, i) if P[j][2] == r]): gen = True + if gen: slide = [] + rest = [i for i in rA if i not in slide and not (gen and i > tL)] + tRM = max([tL] + rest) + 0.01 + sig = [i for i, _, _ in Rd] + if any(i < pa for i in sig) and not snap: return None, 'read before p at X' + lpX = max(pXops + sig + [pa]) + tp1 = lpX + 0.3 + if tp1 > pb: return None, 'p leave' + tBc = max(ca2, tRM) + 0.4; tB = max(tp1, tRM) + 0.4 + if not (tB < pb and tBc < ca3): return None, 'settle window' + cd = lb.Cand(Wd) + cd.dele |= set(xops) | {k1, k2, pb} + for m, i in enumerate(slide): + cd.dele.add(i); cd.add(tL - 0.2 + 0.001 * m, list(P[i])) + if gen and not slide_earlier(Wd, cd, r, kr0, k1, tL - 0.1): return None, 'slide r earlier' + cd.add(tL, ['ADD', q, r, str(beta), A]) + cd.add(tRM, ['MOVE', r, A, B, False]) + used = set() + for t_i, (i, t, al) in enumerate(Rd): + if not snap: + cd.add(i + 0.001, ['ADD', t, p, cd.var('$g%d' % t_i), X]) + else: + # stationary snapshot of p taken right after the (removed) write time, read at the read time + tw = max([w for w, _, _ in Wr if w < i] or [None]) if any(w < i for w, _, _ in Wr) else None + if tw is None: cd.add(i + 0.001, ['ADD', t, p, cd.var('$g%d' % t_i), X]); continue + o = lb.free_label(Wd, tw - 1, i + 1, used) + if o is None: return None, 'no label' + used.add(o) + cd.add(tw - 0.003, ['INIT', o, X]); cd.add(tw - 0.002, ['ADD', o, p, '1', X]) + cd.add(i + 0.001, ['ADD', t, o, cd.var('$g%d' % t_i), X]); cd.add(i + 0.002, ['CLEAR', o, X]) + cd.add(tp1, ['MOVE', p, X, B, False]) + cd.add(float(pb), ['MOVE', p, B, C, False]) + cd.add(tB, ['ADD', p, r, cd.var('$z'), B]) + cd.add(tBc + 0.01, ['ADD', q, r, cd.var('$e'), B]) + tW = max(tB, tBc) + 0.1 + for m, (i, s, qq) in enumerate(Wr): + cd.add(tW + 0.001 * m, ['ADD', r, p, qq, B]) + if not slide_later(Wd, cd, r, B, k2, k3, tW + 0.01): return None, 'slide later' + cd.meta = dict(side='B', r=r, X=X, p=p, c=q, beta=str(beta), A=A, B=B, snap=snap) + return cd, 'ok' + +def build_A(Wd, r, X, p, c_info, snap=False, early=False, slide=True): + P = Wd.W['program']; info = r_xinfo(Wd, r, X) + if info is None: return None, 'r path' + A, B, kr0, k1, k2, k3, xops = info + cl = classify(Wd, r, xops) + if cl is None: return None, 'r ops' + Wr, Rd = cl + if len(Rd) != 1 or Rd[0][1] != p: return None, 'reads' + sgm, _, al = Rd[0] + if any(s == p for _, s, _ in Wr): return None, 'p writes r' + if any(w > sgm for w, _, _ in Wr): return None, 'write after read' + js = Wd.segidx(p, X) + if len(js) != 1: return None, 'p path' + j = js[0]; Sp = Wd.seg[p] + if j == 0: return None, 'p initial at X' + Z, pz0, pa = Sp[j - 1]; _, _, pb = Sp[j] + if not Wd.lt(Z, A) or not Wd.lt(A, X): return None, 'nest' + pZ = Wd.ops_at(p, pz0, pa); lpZ = pZ[-1] if pZ else pz0 + pX = Wd.ops_at(p, pa, pb); fpX = pX[0] if pX else pb + q, cA0, ca1, ca2, ca3 = c_info + rA = Wd.ops_at(r, kr0, k1); rLA = rA[-1] if rA else kr0 + hi = min(fpX, ca1); tA = hi - 0.5 + if not (max(kr0, pz0) < tA - 0.5): return None, 'A window' + if rLA > tA - 0.5 or lpZ > tA - 0.5: + if not slide: return None, 'A window' + tL = max(ca1 - 0.5, tA + 0.3); tRM = max(tA + 0.45, tL + 0.01) + tB = max(ca2, tRM) + 0.5 + if not tB < ca3: return None, 'B window' + cd = lb.Cand(Wd) + cd.dele |= set(xops) | {k1, k2, pa} + if not slide_earlier(Wd, cd, r, kr0, k1, tA - 0.45): return None, 'slide r earlier' + if not slide_earlier(Wd, cd, p, pz0, pa, tA - 0.45): return None, 'slide p earlier' + cd.add(tA, ['MOVE', p, Z, A, False]) + cd.add(tA + 0.1, ['ADD', p, r, al, A]) + cd.add(tA + 0.2, ['ADD', r, p, cd.var('$b'), A]) + cd.add(tL, ['ADD', r, q, cd.var('$be'), A]) + cd.add(tA + 0.4, ['MOVE', p, A, X, False]) + cd.add(tRM, ['MOVE', r, A, B, False]) + used = set() + for m, (w, s, qq) in enumerate(Wr): + g = str(Q(al) * Q(qq)) + # direct read if s is at X at the read time and unchanged since the write; else stationary snapshot + tr = (max(w, tA + 0.4) + 0.0005 + 0.0001 * m) if early else (sgm + 0.001 + 0.0001 * m) + sch = Wd.where(s, int(tr) + 1) if s < Wd.n else None + unchanged = not [i for i in Wd.ops_at(s, w, int(tr) + 1) if P[i][1] == s and i != w] + if not snap and sch == X and unchanged: + cd.add(tr, ['ADD', p, s, g, X]) + else: + o = lb.free_label(Wd, w - 1, int(tr) + 1, used) + if o is None: return None, 'no label' + used.add(o) + cd.add(w - 0.003, ['INIT', o, X]); cd.add(w - 0.002, ['ADD', o, s, '1', X]) + cd.add(tr, ['ADD', p, o, g, X]); cd.add(tr + 0.00005, ['CLEAR', o, X]) + cd.add(tB, ['ADD', r, q, cd.var('$ep'), B]) + if not slide_later(Wd, cd, r, B, k2, k3, tB + 0.01): return None, 'slide later' + cd.meta = dict(side='A', r=r, X=X, p=p, c=q, A=A, B=B, snap=snap, early=early) + return cd, 'ok' + + +def slide_earlier(Wd, cd, x, a, b, cut): + """move x's ops in its stay (a,b) that come after cut to just before cut (order kept). Partners must sit at the + chart at cut and be untouched in between (other than by the slid ops).""" + P = Wd.W['program']; ops = [i for i in Wd.ops_at(x, a, b) if i > cut and i not in cd.dele] + if not ops: return True + ch = Wd.where(x, ops[0]) + for m, i in enumerate(ops): + e = P[i]; ys = [y for y in ([e[1], e[2]] if e[0] in ('ADD', 'COPY') else [e[1]]) if y != x] + for y in ys: + if y >= Wd.n or Wd.where(y, int(cut)) != ch: return False + if [j for j in Wd.ops_at(y, int(cut) - 1, i) if j > cut and j != i and j not in ops]: return False + for m, i in enumerate(ops): + cd.dele.add(i); cd.add(cut - 0.4 + 0.001 * m, list(P[i])) + return True + +def slide_later(Wd, cd, r, B, k2, k3, tW): + """move r's ops that precede the cursor (starting at tW, chart B) to just after it, cascading through r's later + stops (r's MOVEs follow). Partner registers must sit at that chart at the new time and be untouched in between.""" + P = Wd.W['program']; S = Wd.seg[r] + k = [j for j, x in enumerate(S) if x[1] == k2][0] + cur = tW; moved = set() + while k < len(S): + Y, a, b = S[k] + ops = [i for i in Wd.ops_at(r, a, b) if i < cur and i not in cd.dele] + t0 = cur + 0.5 + for m, i in enumerate(ops): + e = P[i]; ys = [x for x in ([e[1], e[2]] if e[0] in ('ADD', 'COPY') else [e[1]]) if x != r] + for y in ys: + if y >= Wd.n: return False + if Wd.where(y, int(t0) + 1) != Y: return False + if [j for j in Wd.ops_at(y, i, int(t0) + 1) if j != i and j not in ops and j not in moved]: return False + cd.dele.add(i); cd.add(t0 + 0.001 * m, list(e)); moved.add(i) + last = t0 + 0.001 * len(ops) if ops else cur + if b >= Wd.T or b > last: return True + cd.dele.add(b); cd.add(last + 0.1, list(P[b])); cur = last + 0.1; k += 1 + return True + +def betas(Wd, r, X, p, cinfo): + """loan coefficient candidates from c's transfers into p at X and r's write coefficients""" + P = Wd.W['program']; q, cA0, ca1, ca2, ca3 = cinfo + lam = [Q(P[i][3]) for i in Wd.ops_at(q, ca1, ca2) if P[i][0] == 'ADD' and P[i][1] == p and P[i][2] == q] + info = r_xinfo(Wd, r, X); qs = [Q(P[i][3]) for i in info[6] if P[i][1] == r] + out = [] + for l in lam: + for qq in qs: + out += [1 / (qq * l), -1 / (qq * l), 1 / l, -1 / l] + out += [Q(1), Q(-1), Q(2), Q(-2), Q(1, 2), Q(-1, 2), Q(4), Q(-4), Q(1, 4), Q(-1, 4)] + return list(dict.fromkeys(out)), bool(lam) + +def try_instance(Wd, r, X, p, side, maxc=12): + """returns (W2, cand, sol) or (None, reason)""" + info = r_xinfo(Wd, r, X) + if info is None: return None, 'r path' + A, B = info[0], info[1] + cs = carriers(Wd, A, X, B, {r, p}) + if not cs: return None, 'no carrier' + why = collections.Counter() + for snap in (False, True): + for ci in cs[:maxc]: + if side == 'B': + bl, haslam = betas(Wd, r, X, p, ci) + if not haslam: bl = bl[-10:] + for be in bl: + cd, msg = build_B(Wd, r, X, p, ci, be, snap) + if cd is None: why[msg] += 1; break + sol = cd.solve() + if sol is None: why['nosol'] += 1; continue + return (cd.instantiate(sol), cd, sol), 'ok' + else: + for early in (False, True): + cd, msg = build_A(Wd, r, X, p, ci, snap, early) + if cd is None: why[msg] += 1; continue + sol = cd.solve() + if sol is None: why['nosol'] += 1; continue + return (cd.instantiate(sol), cd, sol), 'ok' + return None, dict(why) diff --git a/research/p104-loan-snapshot-engine-on7621/price.py b/research/p104-loan-snapshot-engine-on7621/price.py new file mode 100644 index 0000000000..483629160c --- /dev/null +++ b/research/p104-loan-snapshot-engine-on7621/price.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Certify the price with the repository's own invoice tools. + +Runs scripts/compare_recursive_invoices.py (#7621 invoice against the new +one, concave order) and scripts/evaluate_recursive_power.py (new invoice +against the empty invoice at r^t, t = 1 - C) and requires: + - the new invoice is uniformly better than #7621's for every increasing + concave tariff, strictly for r^(1-C), 0 < C < 1; + - at C = 0.0036624027915 the new paid cost is strictly below the new + targets (F(C) > 0), so the crossing is above the claim; + - at C = 0.0036624027916 it is strictly above (F(C) < 0); + - at #7621's C = 0.0036614625547 and #7620's C = 0.0036611026181 the new + invoice is strictly favourable. +The invoices themselves are recomputed from the words by check.py. +""" +import json +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[1] +DATA = HERE / 'data' +CLAIM, ABOVE, PR7621, PR7620 = '0.0036624027915', '0.0036624027916', '0.0036614625547', '0.0036611026181' + + +def tool(name, *args): + out = subprocess.run([sys.executable, '-I', '-B', str(ROOT / 'scripts' / name), *map(str, args)], + capture_output=True, text=True, check=True, timeout=120) + return json.loads(out.stdout) + + +def one_minus(c): + from decimal import Decimal + return str(1 - Decimal(c)) + + +def main(): + old, new, zero = DATA / 'invoice-7621.json', DATA / 'invoice-new.json', DATA / 'invoice-zero.json' + rep = tool('compare_recursive_invoices.py', old, new) + assert rep['classification'] == 'uniformly_better', rep['classification'] + assert rep['power_moment']['new_minus_old_sign'] == 'strictly_negative' + assert rep['delta_paid'] == {'1': 17, '2': 221, '3': -476, '4': 221, '5': 17} and rep['delta_targets'] == {} + print('PASS compare_recursive_invoices #7621 -> new: uniformly_better, paid delta', rep['delta_paid']) + for c, want in ((CLAIM, 'new_cost_lower'), (ABOVE, 'new_cost_higher'), (PR7621, 'new_cost_lower'), (PR7620, 'new_cost_lower')): + r = tool('evaluate_recursive_power.py', zero, new, '--exponent', one_minus(c), '--bits', 256) + assert r['classification'] == want, (c, r['classification']) + print('PASS evaluate_recursive_power at C = %s (t = %s): %s' % (c, r['exponent'], want)) + print('PASS_PRICE kappa in (%s, %s)' % (CLAIM, ABOVE)) + + +if __name__ == '__main__': + sys.exit(main())