Native differential + memory-safety harness: run the guest's Core code without a zkVM - #222
Merged
Merged
Conversation
`docs/FUZZING.md` names this "the highest-value next step" and says the project's central claim — that the guest's VerifyScript/sighash/libsecp and the C++ consensus math match real Core — is the one thing fuzzing does NOT cover. It also names the missing prerequisite: standing up $HAZYNC_BASE and compiling the guest's C++ for the host. That prerequisite now exists. `fuzz-native/build.sh` compiles 23 Core TUs plus secp256k1 for x86-64, mirroring `prover/methods/guest/build.rs` exactly minus the rv32im/ilp32 flags, and `verify_input.cpp` — the guest's own shim — links against it. 27 differential checks pass: subsidy against an independent halving schedule and the 21M supply cap, the CVE-2012-2459 merkle mutation, the retarget timespan clamps saturating in both directions, and cumulative work. ⛔ FUZZING.md says "Core consensus code is portable". Core is; THE PROVISIONED TREE IS NOT, and both base patches are why: 0001 adds a Serialize(int) overload — on LP64 `int` IS `int32_t`, so it is a redefinition 0002 routes CSHA256 through RISC0's accelerator — `sys_sha_compress` does not exist off-guest Neither is optional; provision-vps.sh applies both to every tree. The harness compiles pristine copies of exactly those two files and leaves everything else as the guest sees it. Also needs crypto/sha256_sse4.cpp, which the guest omits via its empty bitcoin-config.h. ⚠ Two limits stated rather than buried. ECMULT_WINDOW_SIZE is NOT set here: the guest builds window 21 and regenerates the precomputed table, while the checked-in table hard-errors on a mismatch. The window is a speed/size trade that does not change what verification decides — but that stops being true if this is ever extended to compare timing. And the independent references are written from the protocol rule; where one is derived from the same reasoning as the implementation it is marked, because such a check catches transcription errors and not logic errors. The harness caught its own author first: the initial CVE-2012-2459 vector used [a,b] vs [a,b,b], which is not the mutation shape. The real one duplicates the last hash at an ODD level, so merkle([a,b,c]) collides with merkle([a,b,c,c]). Refs the "What is NOT covered" section of docs/FUZZING.md. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
SECURITY.md calls the wrapper glue a soft spot and records that `MiniReader` was hardened to "trap on
any read past the buffer end (was an unchecked read)". This feeds every buffer-taking export
truncated, empty, bit-flipped and random inputs under ASan+UBSan.
661 cases: 352 trapped (bounds check), 309 threw (uncaught exception), 0 SANITIZER FINDINGS
Both outcomes are FAIL-CLOSED: `__builtin_trap()` aborts the guest, and so does an uncaught
exception, so neither can yield a proof. Nothing read out of bounds.
⛔ Getting the classification right was the whole job, and two versions of this file were wrong:
1. Without a fork, the FIRST truncated input trapped and the run stopped. SIGILL looked like a
crash when it is the hardening working exactly as SECURITY.md describes.
2. With a fork but no stderr capture, every SIGABRT counted as a finding — 156 of them, all
`terminate called ... ReadCompactSize(): size too large`, i.e. uncaught exceptions. A real
AddressSanitizer report would have been one line among 156 false ones and invisible.
So the child's stderr is read and SIGABRT is disambiguated: a sanitizer report is a finding, an
uncaught exception is fail-closed. Return values are deliberately NOT asserted — a malformed tx may
legitimately be rejected or parsed into nonsense; asserting outputs would encode today's behaviour as
specification. The only claim made is that no wild read occurs.
Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
… comma `-fsanitize=address,undefined` unquoted inside an array is read as two elements (SC2054). Quoted, and ASAN=1 re-verified end to end: the instrumented libcoreconsensus.a still builds (44 MB). ⚠ Caught by CI, not by me: I ran `bash -n` on this file and never shellcheck. The first fix then failed differently — a comment whose FIRST WORD is the linter's name is parsed as a DIRECTIVE (SC1072/SC1073), so explaining the warning in prose broke the file worse than the warning did. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
…fyScript
This is the half of docs/FUZZING.md's "highest-value next step" that touches the CENTRAL CLAIM —
that the guest's VerifyScript/sighash/libsecp agree with real Core. The math differential and the
memory-safety pass did not test it at all.
The corpus needs no trusted node to adjudicate: these blocks are IN THE CHAIN, so Bitcoin's own
consensus already ruled every input valid. Disagreement would mean the guest is wrong.
block 130000 (P2SH era) 10 inputs 10 verified
block 140000 (P2SH era) 212 inputs 212 verified
block 741000 (post-segwit) 670 inputs 670 verified P2SH|DERSIG|NULLDUMMY|CLTV|CSV|WITNESS
------------------------------------------------------
892 inputs 892 verified, 0 rejected
⛔ Three mistakes in this harness, each of which made it LOOK like it was working:
1. POLARITY. verify_input returns `ok ? 1 : -(int)err - 1` — ONE is valid, a NEGATIVE is the
ScriptError, and zero is not a success value at all. Reading 0 as valid inverted BOTH halves:
every real input read as rejected AND every mutation read as refused, so the negative control
passed while proving nothing. Both lies pointed the same way.
2. THE NEGATIVE CONTROL IS NOT A PER-TX PROPERTY. A byte-flip is not a reliable invalidator, for two
reasons that are correct Bitcoin behaviour: other inputs' scriptSigs are BLANKED in the sighash,
so corrupting one cannot invalidate input 0 (block 140000 tx 23, P2PKH, refused 0 of 8 flips for
exactly this); and SIGHASH_NONE/SINGLE inputs do not commit to outputs. So the signal is the
corpus-wide REFUSAL RATE (~89%), not any single transaction. Demanding 100% would be demanding
that sighash cover bytes it deliberately does not.
3. NOT FORKING. A flip can produce a transaction MiniReader refuses to parse, and it fails closed
with __builtin_trap() — SIGILL, which killed the whole run. That is the hardening working, so it
counts as REFUSED. memsafety.cpp already learned this; not carrying it here cost a run.
Corpus is generated by mkcorpus.py from the block fixtures already in the repo.
Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL
This was referenced Sep 6, 2026
defenwycke
added a commit
that referenced
this pull request
Sep 7, 2026
The previous commit ignored `fuzz-native/` wholesale. That is wrong in a way that would not have shown up until it bit: hazync#222 tracks the harness SOURCES in that directory — `build.sh`, `differential.cpp`, `memsafety.cpp`, `mkcorpus.py`, `realvector.cpp` — and a blanket ignore hides them silently, which is the same class of failure as the one this rule exists to prevent. Ignore the two output trees instead. #222 already carries `fuzz-native/build/`; it does not cover `fuzz-native/build-asan/`, and 30 of the 63 artefacts that a `git add -A` staged on 2026-09-06 sat in exactly that gap. Both are listed here so the recurrence is blocked on this branch regardless of merge order, with a note in the file that the overlap with #222 resolves by keeping one copy of each line rather than dropping either. Verified with `git check-ignore`: all five harness sources are NOT ignored; `fuzz-native/build/differential` and `fuzz-native/build-asan/memsafety` are. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU
defenwycke
added a commit
that referenced
this pull request
Sep 7, 2026
* Make CORE the canonical channel: wire 0012/0013 into the build, ship its constants hazync#225. The shipped guest becomes Core: stock stays as the digest oracle, Ghost stays experimental. provision-vps.sh applies patches 0012 (field_bigint2 backend) and 0013 (lift_x witness hint) unconditionally in a new phase 5a, and exports the three defines at THREE sites, not one: - phase 6 for HAZYNC_PROVISION=all / deps - the re-derive block for HAZYNC_PROVISION=build - .bashrc for later interactive builds on a provisioned box The middle one is load-bearing and easy to miss. `build` skips phases 1-7, and reproduce/Dockerfile builds the canonical image with exactly that phase, so exporting only in phase 6 would compile 5a-patched source WITHOUT its defines and silently yield a third, unshipped METHOD_ID. There is no "apply now, arm later" state that keeps the id stable: libsecp's VERIFY_CHECK embeds __LINE__, so editing the tree moves the id even where the body is #ifdef'd out. Patch set and defines move together or not at all. Host packing constants become Core's per-curve fit (2026-09-01), which is host-side only and moves no id: COST_PER_EC_OP 141,612 -> 417,798 COST_PER_SCHNORR_OP 1,950,000 -> 462,435 COST_INPUT_BASE 34,000 -> 41,387 COST_PER_INPUT_BYTE 6 -> 2 COST_PER_EC_OP_REPEAT is deliberately LEFT at 104,222 and documented as unrefit. It is probably wrong for Core -- it was derived as 0.736 of a fresh key from a #139 profile, and 0013 removes most of the decompression that discount prices -- but the straggler 1.210 justifying this channel was measured with it at that value, and BUILDS.md's recipe sets only the four above. Changing a fifth constant would invalidate the number being cited. Refit tracked in hazync#226. Verified on a local build: patches apply clean, hazync_fq_{mul,sqr,inv}_limbs and hazync_lift_x_hint all present in the guest ELF, hazync_ecmult_verify and hazync_scalar_inv absent -- i.e. Core, not silently stock and not Ghost. Docs and the canonical METHOD_ID are re-pointed in follow-up commits; the id must come from the reproducible Docker build, not a laptop. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL * Docs: Core ships, stock is the oracle, Ghost is the experiment hazync#225. The README already said "CORE — what ships" and "Core is the project"; v0.20.0 shipped stock, so #224 correctly added a banner saying neither mode ships. v0.21.0 makes the build match the design, so that banner inverts rather than disappears. BUILDS.md: retitled "The two builds" -> "The three channels", because stock is not the absence of a channel, it is the digest ORACLE -- the fidelity floor every acceleration claim is checked against, and the thing that does not get retired when it stops being the default. Core was cleared against it on block 962,000: journal 4fb3e3c5...4656d byte-identical, 4.095x fewer cycles. The Ghost warning is kept and sharpened rather than dropped. A Ghost build still carries a different METHOD_ID and still proves into rejections, and run-workers.sh only checks the id at STARTUP (#99), so the failure is silent and indefinite. Also kept: promoting a channel is never just a flag. New METHOD_ID, full cutover, every existing proof invalidated -- v0.21.0 spent 1,586 blocks of board to do it. docs/RELEASE_NOTES_v0.20.0.md deliberately still says v0.20.0 shipped stock. That was true of v0.20.0 and stays as written. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL * Draft v0.21.0 release notes Covers the three-channel model, why Core is defensible as the shipped guest (both levers have shipped precedent -- 0002 is already a substitution), the digest gate with its real control, and the per-channel packing constants. Answers the question the v0.20.0 notes did not: how to run a non-default channel, and why doing so cannot contribute to the board. The levers are build-time, not runtime -- there is no config switch, because the channel IS the METHOD_ID. Canonical METHOD_ID deliberately left as a pending marker. It comes from the reproducible Docker build, not a laptop, and the release is not cuttable until it lands. Claude-Session: https://claude.ai/code/session_017fQ8BAyHyzxBf3bvdmckVL * Re-pin the canonical guest to 37987b85, and keep the lineage intact v0.21.0 makes Core the guest that ships, which re-baselines the id from `3867611d…` to `37987b85…`. This carries that pin through every place that states it: `reproduce/METHOD_ID` (a full block, with the three-agreeing-builds provenance and the three-export-sites trap), the two embedded literals in `verifier` and `verifier-ffi`, and the docs. The sweep that produced the first draft of this was a blind string substitution, and a blind substitution over a lineage is lossy: `3867611d` was not only a stale reference, it was also a ROW in the chain of guests. Replacing the string deleted it from the lineage and handed the new id its predecessor's date and cause. Repaired here: - `docs/PROVING.md` — the "Releases" paragraph described the new id as the **parallel block validation** re-baseline of 2026-09-04. That was neither its cause nor its date, and the attribution was already wrong before this sweep: parallel block validation is `1d6c3792` (2026-08-23). Rewritten for what 37987b85 actually is, with the 962,000 figures from `METHOD_ID`. The chain table gets its `3867611d` row back. - `docs/PROVING.md` — "only a proof made against `4722cec8` verifies today" named an id two re-baselines stale. It is `37987b85`. - `docs/ROADMAP.md`, `docs/RELEASE_PLAN.md` — same overwrite, same repair. - `docs/RELEASE_PLAN.md` — says plainly that the 37987b85 cutover has NOT happened; the record below it is the PREVIOUS one. - `README.md` — the board-reset paragraph still described v0.20.0. - `prover/testdata/snark/README.md` — the Groth16 fixtures are pinned to `3867611d` and are now STALE. Three CI gates fail until they are re-proved and re-wrapped; that is part of the cutover, not a separate task. `reproduce/METHOD_ID`'s canonical line is re-seated after the newest block, where its predecessor sat, rather than below the "EXPERIMENTAL IDS — NOT LINEAGE" section it had been appended under. Also ignores `fuzz-native/` — a scratch build tree for the native differential/memsafety harnesses. 63 object files, `libcoreconsensus.a` and two ASan binaries were staged by a `git add -A` and were one commit from landing. Nothing in the repo references the directory. Verified: `check-versions`, `check-guest-inputs`, `check-spec`, `check-workflows`, `check-test-surfaces`, `check-rangestate`, `check-utreexo` and `check-workflow-injection` all REAL_EXIT=0; `verifier-ffi` tests 6 passed; the freshly built `hazync-verify` greps 1 for the new id and 0 for the old. ⛔ `scripts/check-dist.sh` FAILS and is expected to: the staged `dist/` artifacts embed `1d6c3792`, two re-baselines stale. They must be rebuilt before the release is publishable. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU * Ignore the fuzz harness OUTPUT trees, not the harness itself The previous commit ignored `fuzz-native/` wholesale. That is wrong in a way that would not have shown up until it bit: hazync#222 tracks the harness SOURCES in that directory — `build.sh`, `differential.cpp`, `memsafety.cpp`, `mkcorpus.py`, `realvector.cpp` — and a blanket ignore hides them silently, which is the same class of failure as the one this rule exists to prevent. Ignore the two output trees instead. #222 already carries `fuzz-native/build/`; it does not cover `fuzz-native/build-asan/`, and 30 of the 63 artefacts that a `git add -A` staged on 2026-09-06 sat in exactly that gap. Both are listed here so the recurrence is blocked on this branch regardless of merge order, with a note in the file that the overlap with #222 resolves by keeping one copy of each line rather than dropping either. Verified with `git check-ignore`: all five harness sources are NOT ignored; `fuzz-native/build/differential` and `fuzz-native/build-asan/memsafety` are. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU * Stop quoting a board size that moves while you read it Four files said the re-baseline costs `1,586` blocks of proven board. Measured against the live coordinator on 2026-09-07 at 09:56Z, `/api/meta` reports frontier **7,852** — and 7,757 nine minutes before that, i.e. it climbs at roughly 10 blocks a minute and has done since the v0.20.0 cutover. `1,586` was true when it was written and is now off by ~5x. The deeper problem is the shape of the claim, not the digits: the board keeps proving right up until the swap, so the cost of the reset is not knowable in advance and any fixed number here is stale the moment it lands — then gets re-quoted, which is how it reached four files. Each site now carries the reading WITH the timestamp it was taken at, says it is still climbing, and points at `/api/meta` as the thing to re-read at cutover rather than trusting the sentence it is written in. Touched: `docs/BUILDS.md`, `docs/RELEASE_NOTES_v0.21.0.md`, `docs/RELEASE_PLAN.md`, `reproduce/METHOD_ID`. `reproduce/METHOD_ID` is not a guest input, so this does not move the id — `check-guest-inputs.sh` REAL_EXIT=0, as do `check-versions`, `check-spec`, `check-workflows` and `check-test-surfaces`. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU * Regenerate the Groth16 fixtures under 37987b85 The re-baseline invalidated `fold_8.snark` and `neg500.snark`, exactly as `prover/testdata/snark/README.md` says it must: they were wrapped under `3867611d…`, and on #228 `accumulator-tests` failed with VERIFICATION FAILED: the proof is not valid for guest 37987b85 verifier REJECTED the genesis-anchored proof (exit 1) That is the guest pin working, not a bug, and the only fix is to re-prove and re-wrap. Done here, on the coordinator rather than a laptop — 16 cores and 62 GB against 8 GB, and a CPU prove holds ~4.7 GB, so this box could not have run it safely. Sequence, every step `REAL_EXIT=0`: the host binary extracted from the `hazync-core-0a7c174` image and `host method-id` asserted equal to the canonical id BEFORE proving anything; 9 blocks proved from bridge bundles (~6-9 min each, strictly serial); the 7-fold aligned tree; 2 Groth16 wraps on the host, where the Docker daemon is reachable. Both gates were checked for the RIGHT outcome, not merely a non-zero exit — which is the distinction this fixture pair exists to enforce: hazync-verify fold_8.snark -> exit 0, genesis-anchored, guest 37987b85 hazync-verify neg500.snark -> exit 2, valid SNARK refused ON THE ANCHOR An exit of 1 there would have meant "invalid proof" and would have passed a weaker test while proving nothing. The wrapped sizes came back 2,353 B and 6,145 B, matching this file's own table, and `host verify-snark fold_8.snark` reproduces the `out_tip_hash` and work totals recorded in `verifier/README.md`. Also corrects a note that was true when written and is not now: bundles from `/api/witness/<h>` still arrive with `txids`, `wtxids` and `new_outputs` as nested lists, but they NO LONGER need flattening — the `packed_bytes` `visit_seq` fix reads them as-is, and all nine proved straight from the endpoint with no preprocessing. Do not write a flattener. Claude-Session: https://claude.ai/code/session_01BGBba1FtGQjp2focJGWtjU
defenwycke
added a commit
that referenced
this pull request
Sep 14, 2026
…uild docs up to v0.21 (#308) * spec: pin in_smt_root in §9 and check the condition list against is_genesis_anchored §9 listed six genesis conditions and omitted the empty coinbase-SMT root that RangeState::is_genesis_anchored, the host's assert_genesis_in_boundary and the verifier all require (audit #3 F-2/F-3). check-spec.sh now takes the field list from the predicate itself and requires §9's condition list, and its count, to match. Also: §10 seam fields, the maximal-Core claim, the full non-Core trust base in §12, rounds 10-11 and the measured wrapped-proof sizes in §14. * docs: state the trust base as maximal-Core in SOUNDNESS, EXPLAINER and EXTERNAL_REVIEW Since v0.21.0 the canonical guest applies patches 0012 (field_bigint2) and 0013 (lift_x hint) beneath libsecp256k1, and the coinbase SMT sits beside the accumulator. None of the three docs said so. Also: H6 in_smt_root, H7 as _frontier_chain implements it, rounds 10-11, #69 closed, the epoch_start_time expectation (it is the out-boundary's period start, not genesis time), and the live board and CONTRIBUTING.md for the public explainer. * docs: bring the fuzzing record up to #222/#223 and mark the accumulator control unverified since #63 fuzz-native (differential, memsafety, realvector), the in-zkVM negative corpus, forest_cache_equivalence and leaf-differential were missing from the passes table, and the native differential was still described as a future step. The reference Stump's recorded tree_of crash predates its hardening in 8e789a9; whether the positive control still fires is unverified and the rerun command is given. * docs: record the field backend and lift_x hint as shipped, with gate 4 open Both have been in the canonical CORE guest since v0.21.0 (c12ad67) and both docs still read as unbuilt or unmeasured. The corrupt-signature negative control (gate 4) has never run on a CORE build and is now recorded as an open soundness item. LIFTX_HINT.md gains the run-time requirement: chunked commands need HAZYNC_LIFTX_HINT=1 or the guest dies with DeserializeUnexpectedEnd. * docs: fold GHOST_NEXT_BUILD into BUILDS and rewrite TOPOLOGY for CORE BUILDS told Ghost to use the default packing constants, which have been Core's refit since c12ad67; Ghost now gets its #227 constants explicitly. Card counts are described as what they are (serial chunk proving turned into cards by formula), with the 8-card fleet result as the check. GHOST_NEXT_BUILD.md is merged as §3.1 with its errors fixed, and deleted. TOPOLOGY priced the stock guest (~29 L40S, 7-9 cards with #139, 9.10x wholesale) and called several measured quantities unmeasured. Rewritten on the BENCH_8xL40S and MILESTONE_966256 measurements; §1 and §4.1 anchors kept. * docs: replace hand-kept id chains with LINEAGE.tsv and restate GOALS and PROVING for v0.21 Both carried guest-id chains with wrong attributions (b62d2a60 is 067062e, not audit #5) and GOALS carried two transcripts whose id lines had been rewritten to the current guest. Both now point to reproduce/LINEAGE.tsv. GOALS' fleet and card-year figures are restated for CORE with inferred figures labelled; PROVING gains the current release line check-versions.sh greps, CUDA 12.8, the real cost constants, the #256 stall retry, the vendored #119 fix and sm_100. * docs: correct the rule-stability and journal-format claims in the durability docs 4722cec8 (audit #5) was soundness hardening, not performance or build work; the 7 ids sharing today's rule set include dfc9eeda itself; 68819a54 changed the witness wire format, not the journal. METHOD_ID_DURABILITY links Discussion #299, counts 16 supersessions, and cuts sections duplicated from PROOF_DURABILITY to pointers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs/FUZZING.mdnames this "the highest-value next step" and says the project's central claim — that the guest'sVerifyScript/sighash/libsecp and the C++ consensus math match real Core — is the one thing fuzzing does not cover. It also names the missing prerequisite: standing up$HAZYNC_BASEand compiling the guest's C++ for the host.That prerequisite now exists.
What it does
fuzz-native/build.shcompiles 23 Core TUs plus secp256k1 for x86-64, mirroringprover/methods/guest/build.rsexactly minus the rv32im/ilp32 flags, and linksverify_input.cpp— the guest's own shim — against it.differential.cppmemsafety.cppNo GPU, no prover, no zkVM.
⛔ A correction to FUZZING.md's premise
It says "Core consensus code is portable". Core is; the provisioned tree is not, and both base patches are why:
0001Serialize(int)overload — on LP64intisint32_t, so it is a redefinition0002CSHA256through RISC0's accelerator —sys_sha_compressdoes not exist off-guestNeither is optional;
provision-vps.shapplies both to every tree. The harness overlays pristine copies of exactly those two files and leaves everything else as the guest sees it.Limits stated, not buried
ECMULT_WINDOW_SIZEis deliberately unset: the guest builds window 21 and regenerates the table, while the checked-in one hard-errors on a mismatch. The window is a speed/size trade that does not change what verification decides — but that stops being true if this is extended to compare timing.It caught its own author three times
The initial CVE-2012-2459 vector was the wrong shape; SIGILL read as a crash when it is
MiniReaderfailing closed as designed; and 156 SIGABRTs counted as findings when every one was an uncaught exception — a real ASan report would have been invisible among them.