Runtime Differential

Runs the same operation on more than one blockchain runtime and reports what each one did.

Scope & stance. This is a measuring instrument. It records no expectations: probes declare what to run, never what should happen, and a reading is whatever the runtime did on the day it was run. When a runtime changes, the reading changes and nothing here needs correcting first, because a harness that encodes what its author believed can only ever confirm or contradict the author.

It is deliberately not:

• a benchmark. Elapsed time is a single wall-clock sample on one machine.
• a claim about any public network. Nothing here touches devnet, testnet, or mainnet.
• a semantic equivalence proof. It observes behaviour on chosen inputs and nothing more.
• a comparison of costs between runtimes. EVM gas and Aptos gas units are denominated in unrelated things, so a ratio between them would be a number with no meaning.

Why

Differential testing sidesteps the oracle problem: with no specification of what a program should do, a second implementation running the same program is the reference. Where two runtimes disagree, one of them is doing something a developer moving between them will not expect, and that is worth knowing whichever one turns out to be "right".

The interesting readings are the quiet ones. A runtime that halts loudly teaches you something the first time you hit it. A runtime that completes and returns a value where another refuses is the case that ships.

What it measures

Outcomes (cmd/differ). Whether an operation completed or halted, what it produced, and the runtime's own account of why, quoted verbatim. Needs no network on either side.

Charges (cmd/meter). What each runtime charged for the same workload across a range of input sizes, alongside how long it actually took, so charge and work can be read against each other within a runtime. The Move side needs a chain, because gas is only reported for a committed transaction.

Both record their readings to readings.json and report anything that has changed since the last entry, which is how a runtime's behaviour changing becomes visible rather than quietly becoming the new normal.

Running it

Requires Go, Foundry, and the Aptos CLI.

go run ./cmd/differ                 # outcomes, seconds
go run ./cmd/meter                  # charges, minutes: starts a local network
go run ./cmd/differ -json           # machine-readable
go run ./cmd/differ -no-record      # compare against the record without writing
go test ./...                       # the parsers

Pass -forge and -aptos if the binaries are not on PATH.

What runs where

Nothing leaves the machine except downloading the toolchains and the Move framework source.

Side	Executes on	Network
EVM	forge, in process	none: no node, no RPC, no fork
MoveVM, outcomes	aptos move test, in process	none
MoveVM, charges	a local network this tool starts on the loopback interface	its own, created and deleted per run

Gas figures therefore come from the local genesis gas schedule and from whichever EVM version Foundry is configured for. Neither is necessarily what a public network is running today, so treat the charges as a property of these toolchains at these versions.

Adding a probe

An outcome probe is an assertion-free test on each runtime plus an entry in probes.json. Keep the tests assertion-free: a test that asserts an outcome turns a reading into a verdict against whoever wrote the assertion.

On the Move side a probe reports its value with report::emit(b"<test name>", value). The label must match the test name: attribution is by label rather than by position, because the test runner interleaves output across tests and a value can appear after the result line of the test that produced it. A label matching no test is a hard error rather than a lost value.

{
  "id": "shift-amount-exceeds-width",
  "operation": "left shift by an amount at least the width of the type",
  "evm":  { "test": "test_shift_amount_exceeds_width" },
  "move": { "test": "shift_amount_exceeds_width" }
}

A metering workload is an entry in meter.json naming the sizes to run at, an EVM test per size, and a Move entry function.

Reading the output

A difference between runtimes exits 0. It is the finding, not a fault. A probe that could not be read exits non-zero, because then the table is incomplete and cannot be trusted.

readings.json records a reading only when it differs from the previous one, so the file is a record of changes rather than of how often the harness ran. Charges are compared; elapsed time is not, since it varies on every run and would bury the changes that mean something.

Limits worth knowing

Charged cost is measured and real work is only sampled, so nothing here establishes that a runtime is mispricing an operation. At the smallest sizes, elapsed time approaches timer resolution and process noise, and those rows carry the least.

Coverage is narrow on purpose: six probe families, two metering workloads, two runtimes. The structure takes more of each without changing.

Probes are cheap. Each is one operation on each runtime and a manifest entry, roughly ten lines, and several of the divergences recorded here were found by writing exactly that much and running it.

Some probes compare things that are alike rather than identical. A mapping entry keyed by address is the closest EVM equivalent of a resource held at an address, and it is not the same thing. Where a probe leans on an analogy, its operation says so, and the reading is worth more than the analogy that framed it.

License

MIT, see LICENSE.