Skip to content

Subgraph mode: compatibility fixes, 30% → 83% of 860 real subgraphs (85% of 2026 repos) - #1759

Merged
DZakh merged 447 commits into
claude/hyperindex-subgraph-mode-bs67pqfrom
moose/subgraph-compat-all
Oct 22, 2026
Merged

DZakh merged 447 commits into
claude/hyperindex-subgraph-mode-bs67pqfrom
moose/subgraph-compat-all

Conversation

@moose-code

@moose-code moose-code commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Every subgraph-mode compatibility fix in one PR against #1626's branch.

  • Round 1: the 13 earlier PRs from the hackathon, listed at the end, plus three branches that never had a PR.
  • Round 2 (20–21 Oct): measured on a corpus of 860 real subgraphs and checked against live graph-node deployments.

447 commits (393 non-merge). Each fix comes with a test that failed first, and cites graph-node or graph-ts behaviour where it matters.

Results: unmodified real subgraphs

Real subgraph repositories (subgraph.yaml + schema + AssemblyScript mappings), cloned at a pinned commit and run unmodified with envio start. ok means at least one batch was committed with no error for 60 s, or the run synced to the end of its block window.

set entries envio@3.14.0-subgraph (#1626 as published) this PR when opened (d2777248b) this PR now
recent: repos created or pushed since March 2026 632 233 (37%) 470 (74%) 538 (85%)
legacy: the October measurement's repos 128 17 (13%) 58 (45%) 99 (77%)
subgraphs deployed on hosted subgraph platforms 117 12 (10%) 66 (56%) 89 (76%)
all 860 257 (30%) 582 (68%) 713 (83%)
  • Size: 708 distinct repositories. The recent set has 537 repos, all pushed between 1 March and 20 Oct 2026; 389 of its 632 entries are repos created in 2026.
  • Deploy options: of the 713 ok, 601 run as-is. 112 need a deploy option a deploying user passes anyway, the same input graph build --network takes: ENVIO_SUBGRAPH_NETWORK for a network pick, or ENVIO_CONFIG for a manifest that isn't subgraph.yaml.
  • Zero-event syncs: 57 of the ok synced a window with 0 events.
    • 40 of those were checked against HyperSync for logs their handlers match, and none had any. The other 17 are on chains HyperSync doesn't serve, so they're unverified.
    • One sync that missed matching logs is counted as not ok.
  • Regressions: 0. Each build re-ran every entry not ok on the previous one, plus a 20% random sample of the ok ones. Any drop was re-run on the previous build with the same entry before being called a regression.

Data parity: 113 unmodified subgraphs were compared against a live graph-node deployment of the same subgraph at a pinned block, using envio subgraph diff.

  • 93 are equal in every entity type. That's 1,194 of 1,280 types, with 41k entities compared field by field. They include:
    • SSV network (53/53 types, 571k events) and BOLT (14/14, 38M events);
    • GMX synthetics (38/38) and Balancer v3 vault and pools;
    • harbor-app (29/29, 3.6M events) and aave-v4-omnigraph (25/25).
  • Every mismatch was triaged. Two were envio bugs and are fixed here: the event.logIndex byte width, and Bytes.empty(). The rest:
    • The deployed build differs from the repo: deploy-time constants or older mappings. Patched copies confirm it.
    • Two deployments are short of events: ours equals the on-chain log count.
    • IPFS files no gateway served.

Remaining failures, by first blocker:

  • 57 are repo setup: mostly ABIs the manifest names that are contract build output not committed to the repo.
  • 13 manifests are written by package scripts envio deliberately won't run (they also deploy).
  • 10 are mapping imports that don't resolve.
  • 23 are external: chains whose only keyless RPC is pruned or too slow.
  • 13 use network names no registry knows and have no RPC.
  • The rest are scattered: perf, one substreams subgraph, and so on.

What changed, by area

  • Mapping runtime (AssemblyScript semantics):
    • === follows AssemblyScript identity semantics.
    • Module-cycle initialisation.
    • Top-level code re-runs per trigger, as graph-node instantiates per trigger.
    • AssemblyScript Map/Set.
    • Arithmetic:
      • u32/i64 bit operations and integer casts (as i64, <T>x);
      • float division and toString;
      • zero-initialised fields and locals.
    • Pointer-level operations:
      • memory builtins and StaticArray;
      • String.UTF8/UTF16 and reinterpret;
      • changetype retagging onto the generated getters.
    • The default sort() comparator.
    • Minimal-byte BigInts from chain data, everywhere graph-node builds them.
  • graph-ts API:
    • Bytes.empty();
    • BigInt.toI32/toU32/toHexString widths;
    • ethereum.encode per token, and decodeParams;
    • JSON number text and key order;
    • dataSource.address();
    • YAML;
    • reserved-word fields;
    • null into a value-typed setter;
    • the missing-non-null-field error on store.set, as graph-node raises it.
  • Generated code:
    • The bundled codegen follows the project's graph-cli version.
    • When a mapping imports a generated file that only existed in the author's tree, it's rebuilt from an unambiguous source (a sibling manifest, an ABI, a fragment), with a warning each time.
    • Workspace packages resolve without an install, and missing libraries are fetched when the repo's install fails.
    • Pre-0.18 graph-ts names are supported.
  • Repo shapes:
    • Manifests rendered with standard tools (mustache, cp, cat, graph).
    • ENVIO_SUBGRAPH_NETWORK picks the network, and --config picks a manifest.
    • Case-only path mismatches.
    • Clear, named errors for everything envio won't do.
  • Newer graph-node features:
    • specVersion 1.4/1.5 and apiVersion 0.0.10.
    • Aggregations and timeseries: graph-node's own aggregation store test is ported here and passes.
    • once/polling handlers on templates.
    • skipDuplicates.
    • IPFS ContentPath, timeouts and hedged gateways.
    • A clear "can't index" refusal, with the reason, for substreams, composition, Amp and non-EVM data sources.
  • Networks:
    • A network name envio doesn't know resolves through the RPC's eth_chainId.
    • New network slugs.
    • Hardened RPC-only sync for chains HyperSync doesn't serve.
  • Contract calls and throughput:
    • JSON-RPC batches;
    • preload lookahead and a per-batch overlay;
    • adaptive concurrency on rate limits;
    • call caches that survive envio start -r;
    • effect cache keys hashed above 2 kB;
    • BigDecimal and hex fast paths, bit-identical against the crate's vectors.
    • Uniswap-v3 forks run about 35% faster, and request counts fall by 80–90% on call-heavy subgraphs.
  • Templates created from block handlers: they now register with graph-node's timing. Before, they were silently missed (yieldyak strategies: 1 → 7,002 events).
  • envio subgraph diff: --wait, --config, aggregation intervals, and pacing for rate-limited references.
  • Tests and CI:

Please review closely

  1. envio now runs a repo's own scripts, narrowly (docs §6a, §6d). It already ran the steps of the codegen script.
    • Codegen steps: the codegen script's steps before graph codegen now also run when the manifest or its ABIs are missing.
    • Manifest writer: when there is no manifest, envio runs the one package.json script that writes it, picked by network, and only up to its first graph-cli step.
    • Guards: each command is logged, ENVIO_* variables are stripped, and stdin is closed. Deploy, publish and docker scripts, and anything running git, curl or ssh, are never run. Tests prove a deploy script and another network's script never run.
  2. Core engine changes, gated to subgraph mode unless noted:
    • Block-handler register pass: FetchState, ChainState and ChainFetching (blockRegisterThrough is None unless an onBlock registration carries a contractRegister).
    • Other gated changes:
      • a once-first block comparator;
      • preload lookahead and overlay in EventProcessing, UserContext and SyncWrites;
      • the register-store frontier.
    • Applies to all effects: the effect cache key becomes sha256: above 2 kB.
    • Coverage: the full envio-tests suite, native included, is green on Postgres and ClickHouse.
  3. Network resolution:
    • Disagreements: when ENVIO_SUBGRAPH_RPC serves a different chain than the registry gives a manifest's only network name, the RPC's chain wins, with a warning, as on graph-node.
    • Translation probe: translation asks each RPC endpoint for eth_chainId once.
  4. Behaviour now matching graph-node. It changes outputs that used to be accepted:
    • a missing non-null field fails at store.set;
    • ipfs.cat returns null after 60 s;
    • BigInt.toI64 overflow aborts;
    • top-level code re-runs per trigger;
    • a non-anonymous topic0 override drops its handler with a warning.
  5. Heuristics, each warning when used: the generated-file fill and the re-export fill for stale generated/ trees.
  6. Test indexer: it reads the undocumented process._getActiveHandles() to avoid probing its own RPC stubs. Without it, it falls back to waiting as before.

Known gaps

  • Refused (§7): a create from a template's unfiltered or polling block handler, and a data-source block handler running over more than 1,000 blocks whose create isn't certain.
  • Accepted but not honoured: nonFatalErrors; Timestamp's sub-millisecond digits.
  • Not done:
    • ABIs that only exist after compiling the repo's contracts are not built. envio names the missing file and the step that writes it.
    • Not emulated: AssemblyScript integer literals above 2^53, i32 arithmetic overflow, f32's shortest toString.
  • For envio-serve: a null list of entities should be served as [], as graph-node does.

Tests and CI

  • CI at c2ea844bb: green.
    • envio-tests: Postgres 3.7 min, ClickHouse 5.4 min.
    • cargo-test, e2e, scenarios and templates all pass.
    • hypersync-health-check fails only because 51.hypersync.xyz reports unhealthy, the same as on main. It's continue-on-error.
  • Locally:
    • cargo test --lib: 1,176 passed.
    • Full envio-tests on Postgres: 430 files passed.
    • test/Subgraph: 227 files.
    • Replay stress: green.

What's in it

Round 1 (merged as one branch, d2777248b):

Round 2, one branch per area, merged in order:

  • runtime, codegen, shapes, hosted, modern;
  • mapping (runtime round 2), recent, calls;
  • hygiene, sweep, shapes2, parity, polish, ci, blockcreate.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: d555d8aa-8b6b-4cd8-9cd3-37f3c6c46dd1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

moose-code and others added 28 commits October 20, 2026 15:35
graph-cli before 0.18 (mapping apiVersion 0.0.3 and older) generates
`super.call("ownerOf", [params])` and `super.tryCall(...)` with no
function signature; graph-node then takes the first function of that
name in the data source's ABI (chain/ethereum/src/runtime/
runtime_adapter.rs: `function_signature: None` → `contract.function(name)`).
The shim only took the three-argument form of later graph-ts and failed
with 'Unreadable contract call signature ""'.

SmartContract.call/tryCall now accept the two-argument form; the
runtime installs a lookup that reads the manifest's ABI of that name
and builds graph codegen's `name(inputs):(outputs)` signature for the
first function so named. A name the ABI lacks is an error, as on
graph-node.

Test: SubgraphLegacyContractCall_test.res

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-node's handlers_for_log returns every handler of a data source whose
event matches a log and runs them in manifest order. The translation gave
each extra handler of an event its own envio event (`Ping_1`), which envio
refuses as two events "the indexer can't tell apart", so the subgraph
didn't start. envio runs every registration on an event since #1469, so a
repeated event now registers its handler on the first one's event; the
event selects receipt fields when any of its handlers asks for them.

Groundwork for swaap-labs/swaap-subgraph-v1 (a subgraph from a hosted
subgraph platform), which hangs a dozen handlers on LOG_CALL.

Test: SubgraphSameEventTwice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sked of RPC concurrently

On a chain HyperSync doesn't serve, a block handler's block.timestamp (and
header) comes from ENVIO_SUBGRAPH_RPC, one eth_getBlockByNumber per block,
and the fallback awaited them one after another: on BlockDAG (evm-1404) a
handler on every block took 277 s to commit its first 500 blocks. graph-node
fetches a range's blocks concurrently; the fallback now issues them at once,
bounded by the RPC client's requests in flight (ENVIO_SUBGRAPH_RPC_CONCURRENCY).

Test: SubgraphRpcBlockTimestamps_test.res (mock RPC records the most
eth_getBlockByNumber requests in flight).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-ts before 0.22 named the transaction's sixth field gasUsed; graph-node
fills that slot with the transaction's gas limit (gas_limit in every
AscEthereumTransaction_0_0_*, chain/ethereum/src/runtime/abi.rs), which
later graph-ts calls gasLimit. Uniswap v3's 2022 mappings
(arbitrum-minimal) store event.transaction.gasUsed on every Transaction,
and it read null, so the batch write failed on the non-null column.

The shim's transaction now carries gasUsed from envio's gas field, and the
field scan selects gas for it.

Test: SubgraphLegacyTransactionGasUsed_test.res, cargo test
selects_the_gas_limit_for_a_pre_0_22_gas_used.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Omni-DeFi's mappings import `../generated/staking/staking`, and graph
codegen writes `generated/Staking/Staking.ts` for the data source
`Staking`. asc resolves that on the macOS or Windows filesystem the
subgraph builds and deploys from; Node on Linux didn't find the module,
and the mapping failed to load.

A relative import from a project file that Node can't find is now
matched against the filesystem one path component at a time: when each
missing component has exactly one entry differing from it only in case,
that file is loaded, with one warning per import. Imports into
node_modules and files outside the project are left alone.

Test: SubgraphImportCase_test.res (failed to load the mapping before).
Omni-DeFi/subgraphs (mode-testnet, hosted): failed to load src/staking.ts;
now indexes (44 events to block 20,032,984 in 150 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the network's

graph-node checks each provider against the chain its network was configured
for (genesis hash and net version) and refuses one that serves another. Here
the endpoint was taken on trust: one of the wrong chain answered a mapping's
contract calls with another chain's state, or on a chain HyperSync doesn't
serve, indexed another chain outright. At startup the endpoint is now asked
its eth_chainId and refused on a mismatch, naming both chains; an endpoint
that doesn't answer is left to the runtime's retries.

Test: subgraph::project_tests::refuses_an_rpc_endpoint_of_another_chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph codegen writes schema.ts, templates.ts, <data source>/<ABI>.ts
and templates/<template>/<ABI>.ts for the manifest it runs on, and no
index.ts. A mapping importing anything else fails `graph build` too
(asc TS6054), and builds only on a machine whose tree is left from an
earlier manifest: revert's uni-v3-staker imports generated/UniV3Staker/
after renaming its data sources UniV3Staker1.0.0/1.0.2; WOOFi's
per-chain manifests import data sources only another chain's manifest
declares; tornado imports a hand-written generated/index.ts. envio
reported only Node's "Cannot find module"/"Directory import"; it now
adds which data source, template or ABI the manifest lacks and what it
has.

Test: SubgraphGeneratedNotWritten_test.res

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A cumulative aggregate carries each group's previous bucket forward
(graph-node rollup.rs, `cumulative: true`), so a rollup needs every
group's carried values. They were read one `getSync` at a time, and each
miss suspends the sync bridge: the handler that crossed a bucket's end ran
once per group, every replay redoing the rollup. graph-node's aggregation
example (graphprotocol/graph-tooling examples/aggregations, a group per
block number modulo 1000) replayed block 930's handler 934 times and
committed nothing in three minutes.

The rollup state rows now carry `aggregation` = `<Aggregation>|<interval>`
(indexed), and a rollup reads its groups' rows with one getWhereSync: at
most one round for all of them.

Test: SubgraphAggregationGroups_test.res, the same 2,000 rows rolled up as
one group and as a group each (4.8 s vs 0.26 s before; now comparable).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…de does

graph-node validates a saved entity, merged onto the stored one with
explicit nulls removed, against the schema (EntityCache::set ->
Entity::validate) and fails the handler with "Entity T[id]: missing value
for non-nullable field `f`". envio accepted the save and the batch write
failed later, on Postgres' NOT NULL or on a null it couldn't serialize,
with nothing pointing at the mapping: Opolis' ensureOpolisPayToken saves
an OpolisPayToken without createdAt: BigInt!.

The translated schema now records which fields are declared non-null, and
store.set checks the merged row before writing it (not in the register
pass). Two tests saved entities that graph-node would refuse: the
receipt-logs mapping now stores a fallback name for an unparsed log, and
the changetype test seeds an entity type without required fields.

Test: SubgraphMissingRequiredField_test.res.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registry added aliases since the snapshot was taken (arctest for
arc-testnet, basesepolia, unisepolia, zksyncera), and the CI drift check
(sync-graph-networks.mjs --check) fails on it. Refreshed with the script.

Test: subgraph::manifest::tests::maps_graph_network_names_to_chain_ids
(arctest, basesepolia).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Preload runs a batch's handlers with their writes dropped, so each one read
the rollup clock as the previous batch left it, found its block past the
end of that clock's bucket, and rolled the bucket up again: a batch of
5,000 block handlers redid one rollup 5,000 times, ten aggregations each in
graph-node's aggregation example, which then committed nothing from its
second batch on. The execute pass, where the clock advances handler by
handler, rolls up as graph-node's Layout::rollup does; preload still reads
the clock, so the execute pass finds it in memory.

graphprotocol/graph-tooling examples/aggregations (mainnet, block handler
on every block, five aggregations, cumulative groups): 0 blocks committed
in 180 s before; 345,000 blocks in 90 s after, with this and the previous
commit.

Test: SubgraphAggregationPreload_test.res, a batch after a split run vs
the same rows in one run (2.5 s vs 0.35 s before; now comparable).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ore translating

cope-market's erc4626-vault commits subgraph.template.yaml and writes
subgraph.yaml in the first step of its codegen script: `"codegen": "node
scripts/build-manifest.mjs ${NETWORK:-arc-testnet} && graph codegen"`. envio
runs a codegen script's other steps anyway (§6a), but the missing manifest
stopped it at translation, before it got there.

When the manifest is missing and no mustache script renders it, the codegen
script's steps before `graph codegen` now run first, in the package
directory with its bins on PATH. A variable a step reads is set to
ENVIO_SUBGRAPH_NETWORK when that is set; unset, the script's default
applies. The manifest is recorded as envio's, so a later start writes it
again, and the runtime's codegen skips the steps already run, which would
otherwise write the script's default network over the one picked. A failing
step stops envio with the script's own error. The "config.yaml not found"
hint no longer offers a `*.template.yaml` as a manifest to run.

Tests: project_tests::runs_the_codegen_steps_that_write_the_manifest,
SubgraphCodegenPrelude_test.res (graph codegen got the default network's
manifest without the skip). cope-market/subgraphs erc4626-vault
(arc-testnet), unmodified: "Found subgraph.template.yaml but no
subgraph.yaml" before; now starts and indexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
balancer/gauges-subgraph builds one tree from two manifests:
`graph codegen subgraph.yaml --output-dir src/types/ && graph codegen
subgraph.arbitrum.yaml --output-dir src/types/`, and its mappings
import data sources only the arbitrum manifest declares. envio ran
every `graph codegen` step of the codegen script on its own manifest,
dropping the one the step names, so that half was never written; the
bundled codegen wrote only envio's manifest.

A step naming another manifest of the project now generates that one,
into the directory the step names; envio's own manifest is generated
where a step names it or no manifest at all, and last otherwise, so its
files win. The bundled codegen follows the same plan.

Test: SubgraphCodegenScriptManifests_test.res (graph-cli),
SubgraphBundledCodegenManifests_test.res (bundled)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s topics

graph-node keys an event handler on `topic0` when the manifest gives one
(MappingEventHandler::topic0) and parses an anonymous event's log with every
topic an indexed param. Balancer v1's pools emit an anonymous
`LOG_CALL(bytes4 indexed sig, address indexed caller, bytes data)` and their
subgraphs hang one handler per called selector on it; translation refused
those as "topic0 is not the event signature's hash".

- manifest: a topic0 other than the signature's hash is kept on the handler
  (a value that isn't a 32-byte word is an unknown-error).
- translation: on an event the ABI declares anonymous, each (event, topic0)
  is its own envio event carrying that topic0; elsewhere, or with topic1-3
  filters, it is still refused.
- config: an EVM event config may carry a `topic0` (not in config.yaml's
  schema): on an anonymous ABI event it becomes the sighash and the event is
  marked anonymous in the internal config.
- runtime: an anonymous event's topic count is its indexed params alone, and
  the decoder reads a registration whose topic count equals its indexed
  params as anonymous (no signature topic).

swaap-labs/swaap-subgraph-v1, a subgraph from a hosted subgraph platform:
indexes on polygon (setSwapFee/finalize/bind decode to the values on chain)
until its next blocker, a Transaction saved without its non-null gasUsed.

Tests: SubgraphAnonymousTopic0,
evm_hypersync_source::decode::tests::decodes_an_anonymous_event_from_all_of_its_topics,
subgraph::manifest keeps_a_topic0_other_than_the_signature_hash and
refuses_a_topic0_that_is_not_a_word, and the translation's topic0 test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s created

Preload runs a batch's handlers against the store as it was before the batch,
so a handler reading an entity an earlier event of the same batch creates (a
v3 pool's Mint after its PoolCreated) finds nothing, throws or returns early,
and makes none of its contract calls. The execute pass then asks them one
event at a time: a round trip per call, in series.

Preload reads that find an entity absent are now remembered per item. When
execute creates that entity, those items are preloaded again against the
current store, as long as they are within ENVIO_SUBGRAPH_PRELOAD_LOOKAHEAD
(default 256, 0 = off) items of the one executing; an item whose preloads
already ran for 20 ms of CPU is not preloaded again. Subgraph mode only.
Preload never writes, and every value it fetches is keyed by its full input,
so what execute computes is unchanged.

Test: SubgraphPreloadLookahead_test (16 handlers reading a pool created
earlier in the batch: calls in flight at once 1 -> 16, results equal).
sushiswap v3 (mainnet, from its start block), 180 s cap, 200 ms per RPC
request: 44 -> 1,044 events committed; execute rounds waiting on eth_call
489 (100 s) -> 91 (33 s). Row-for-row equal to a lookahead-off run over the
same 544 events (23 tables, 0 differing rows).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…othing else runs

The codegen script's steps that run before translation (ca39bb2) now
print a line naming each step, and the variables set from
ENVIO_SUBGRAPH_NETWORK, before running it. A test pins the scope: with
preinstall, install, postinstall, prepare, build and manifest scripts that
would write the manifest, and a codegen script that is only `graph codegen`,
none runs, and the error names the one that looks like a writer. §6d says so.

Test: project_tests::runs_no_script_but_the_codegen_scripts_steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-ts' ByteArray.toI32()/toI64(), which BigInt's defer to
(common/collections.ts), assert only that the bytes past the width equal
the sign padding of the last byte, then read the low bytes as two's
complement. A uint32 of 2^32 - 1 is `ff ff ff ff 00` and reads as -1, and
any value from -2^32 to 2^32 - 1 (2^64 for i64) wraps into the signed
range. The shim refused everything outside i32 (and BigInt.toI64 returned
values past i64 unchecked).

poa-box/subgraph-pop calls `event.params.maxMembers.toI32()` on
`type(uint32).max` in handleSubjectCreated: the run stopped there; now
synced (1,307 events in the window).

Test: SubgraphBigIntSignedWrap_test.res.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…repo's build passes

iExec's PoCo subgraph names no network in subgraph.yaml and keeps addresses
in networks.json; its build runs `graph build --network
${NETWORK_NAME:-arbitrum-sepolia}`. A bare `envio start` stopped at "declares
no indexable data source", asking for ENVIO_SUBGRAPH_NETWORK.

When ENVIO_SUBGRAPH_NETWORK is unset, a data source names no network and
networks.json exists, the network is now the one the package.json scripts
pass `--network` — a `${VAR:-default}` default or a literal — when they all
agree on it and networks.json has it. Startup logs the network built. Scripts
that pass several still leave the choice to ENVIO_SUBGRAPH_NETWORK.

Test: project_tests::builds_the_manifest_for_the_network_the_repo_build_passes
(also: two literal networks stay an error). iexec-poco, unmodified: now
starts on arbitrum-sepolia; its next blocker is an RPC without archive
state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-node instantiates the mapping's WASM module for every trigger
(runtime/wasm/src/mapping.rs: instantiate_module before each
handle_trigger): the start function runs again in that trigger's context
and no global keeps what an earlier handler left in it. envio evaluated
each file once, so module state leaked across events, replays and the
preload pass: SporkDAO numbers TransferData from a module counter,
morpho-blue caches its protocol entity in a module variable, fabcrowd
keeps a per-block cache; and a top-level dataSource.context() read the
data source the file was loaded for, not the current instance's.

The post-transform pass now splits a module into stages. Classes whose
definition runs no code, constant globals and globals built only from
graph-ts values (BigInt.fromI32(0), Address.fromString(...)) are set once
in __envio_static, as before. Every other top-level statement moves, in
order, into __envio_start, a declaration becoming an assignment to its
var; it runs as the module loads (a failure there is left to the first
trigger, which runs it in a real context) and again through
__envio_init(generation), which first calls the same function of each
relative module imported or re-exported. The runtime calls the mapping's
__envio_init as every handler, register or file data source round starts.

Test: SubgraphModuleState_test.res, cargo test
splits_a_module_into_static_and_per_trigger_stages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph codegen never cleans its output directory, so the tree a project
with one manifest per network was built with holds the data sources of
every manifest it generated, and the mappings import them: WOOFi's bsc
mappings import generated/WooRouterV1_3/ (only avax.yaml declares it),
DODO's bsc ones src/types/dodoex/DVMFactory/ (mainnet), flaunch's
generated/BuyBackAndBurnFlay/ (base-sepolia). `graph build` of that one
manifest fails on a clean checkout (asc TS6054), but the deployed
subgraph was compiled against the developer's tree.

After codegen (graph-cli or bundled), the generated files the mappings
still miss are written from the first other manifest of the project
that declares that data source or template with that ABI
(codegen::fill_from_manifests, exported as subgraphFillGenerated), with
a warning naming the files, the manifest and why. Only missing files
are written; schema.ts and envio's own manifest's files are never
touched. ENVIO_SUBGRAPH_CODEGEN=graph-cli keeps graph-cli's behaviour.

WOOFi bsc (with its documented constants copy) and DODO bsc now index.

Test: SubgraphGeneratedFromSiblingManifest_test.res

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…receipt logs from RPC alone

A block handler's timestamp and header, and event.receipt.logs, asked
<chain>.hypersync.xyz first and fell back to ENVIO_SUBGRAPH_RPC on failure,
even for a chain HyperSync doesn't serve. A refused connection costs little,
but where a firewall drops it instead every batch waited out the request
deadline (30 s) first. The translation knows which chains HyperSync serves;
it now hands the runtime the others (rpcOnlyChains), whose headers and
receipts go to the RPC directly.

Test: SubgraphRpcBlockTimestamps_test.res (a HyperSync host that never
answers is not asked).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-node parses the link handed to `ipfs.cat`, `ipfs.map` and a file
data source with `ContentPath::new` (graph/src/ipfs/content_path.rs):
`<CID>[/<path>]`, `/ipfs/…`, `ipfs://…`, or an http(s) URL with
`/ipfs/<CID>…` in its path or the CID in `?arg=`; an empty string, a
non-CID or a path with a query fail to parse, and `ipfs.cat` returns null
for them (runtime/wasm/src/module/context.rs) while `ipfs.map` fails.

The shim sent anything it couldn't normalize to the gateway as is:
SoulSystemProtocol/SoulSystem-Subgraph's `ipfs.cat("")` (the last segment
of an empty URI) fetched the gateway's root, which answered 429, and the
run stalled asking again; a gateway URL was appended to the gateway.

SubgraphIpfs_test.res used `QmMissing`/`QmFlaky`, which graph-node
refuses as CIDs; they are CIDv0-shaped now.

Test: SubgraphIpfsContentPath_test.res.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, with its variables

Aave's protocol-subgraphs writes subgraph.yaml from one script per
deployment: `"deploy:hosted:mainnet-v3": "VERSION=v3 BLOCKCHAIN=v3
NETWORK=mainnet env-cmd npm run prepare:subgraph && …"`, where
`prepare:subgraph` is `mustache ./config/${NETWORK}-${VERSION}.json
./templates/${BLOCKCHAIN}.subgraph.template.yaml > subgraph.yaml && … npm
run generate:schema && npm run subgraph:codegen`, and `generate:schema`
copies `schemas/${VERSION}.schema.graphql` to schema.graphql. envio saw three
unbound variables and named the script instead of rendering.

A step that is `npm run <script>` (`yarn <script>`, `pnpm <script>`) is now
read as that script's steps, with the variables the call sets bound in them,
as a shell hands them on; `cp <from> <to>` is repeated, and `env-cmd` is a
wrapper. A script that only calls the one that renders (zora's `codegen` runs
`yarn gen-yaml-config`) is dropped for it, and the runtime's codegen skips
every script the rendering one inlined. Also fixed: a mustache view holding a
variable (`mustache config/$NETWORK.json …`) was taken for an option.

Test: project_tests::renders_through_the_scripts_a_deployment_script_runs
(names the network per deployment: mainnet-v3, arbitrum-v3). Aave v3
mainnet, unmodified, ENVIO_SUBGRAPH_NETWORK=mainnet-v3: "no subgraph.yaml"
before; now renders, copies its schema and indexes (57,751 events, blocks
16.29M -> 16.80M in 180 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every contract call, balance and block lookup was its own HTTP request, so a
batch's thousands of calls cost thousands of round trips through 48
connections. Requests that queue while those are busy now leave together as
one JSON-RPC batch of up to ENVIO_SUBGRAPH_RPC_BATCH (default 32; 1 = off).

Each request keeps the answer it would get alone: answers are matched by id
(in any order), a JSON-RPC error stays that item's error, so a revert reaches
only the try_ call that reverted, and an HTTP failure fails the items it
carried for untilAnswered to ask again. An endpoint that refuses a batch
(HTTP 4xx, an error object in place of the array, or a per-item "batch of
more than N requests" as dRPC's free plan answers) gets smaller batches from
then on, down to plain single requests. viem's own batching matches answers
by position and can't shrink, so the transport is built on its HTTP client
and error types instead.

Test: SubgraphRpcBatch_test (40 calls, one reverting: one HTTP request per
>= 4 calls, results equal; all answered by an endpoint allowing 3 per batch
and by one that takes no batches). sushiswap v3 at 200 ms per HTTP request,
180 s: HTTP requests 1,899 -> 335 for ~1,900 calls, 544 -> 1,044 events;
row-for-row equal to a run without (23 tables).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
graph-ts' `ByteArray.empty()` returns `ByteArray.fromI32(0)` and
`Bytes.empty()` casts it (common/collections.ts, unchanged from 0.27 to
0.38): four zero bytes. The shim returned an empty array, so every field a
mapping defaults with `Bytes.empty()` stored `0x` where graph-node stores
`0x00000000`, and `Bytes.empty().length` read 0 instead of 4. 148 mapping
files in modern's 2025-26 survey call it, among them KlimaDAO's
polygon-digital-carbon, valory-xyz/autonolas-subgraph and
Proof-Of-Humanity v2.

Test: SubgraphBytesEmpty_test.res.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A node's own failure arrives as JSON-RPC -32603 "Internal error" over a 200,
and the same request answers on the next try. The host-op retry only knew
rate-limit codes and stopped the run on it: Miljan9602/dragonswap-subgraph-sei's
register pass failed on one symbol() call ('An internal error was received')
that answered when asked again. graph-node retries every eth_call error that
isn't a revert (chain/ethereum/src/call_helper.rs interpret_eth_call_error).

Test: SubgraphRpcResilience_test.res (an internal error, then the value).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AssemblyScript's std Map and Set aren't JavaScript's (asc 0.19.10
std/assembly/map.ts, set.ts): keys() and values() return arrays, map[key]
is the @operator("[]") get and map[key] = v its set, set[key] is has, and
get of a missing key throws "Key does not exist". On JavaScript's Map,
Intent-X's analytics loop over dictionary.keys().length ran zero times
and Reserve's RSR_CHAINLINK_CONTRACT_ADDRESS[network] read undefined.

The transform turns new Map / new Set (when the module doesn't bind its
own) into __envio_Map / __envio_Set, which the runtime installs: Map and
Set subclasses with those methods, behind a proxy that serves a property
access that isn't a member as the [] operator (a numeric spelling as a
number key when the collection holds numbers).

Test: SubgraphAsMap_test.res, cargo test builds_assemblyscript_maps_and_sets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moose-code and others added 3 commits October 21, 2026 06:14
Clean: the diff tool's rate-limit spacing, --wait, --config and
timeseries/aggregation comparison, and event.logIndex in graph-node's
minimal bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n up as missing

A file data source whose gateways all answered 429 was skipped after
ENVIO_SUBGRAPH_FILE_ATTEMPTS rounds, as if the file weren't there
(HandProtocol lost three RoundMetadata files while public gateways
rate-limited the run). graph-node's polling monitor backs off and asks
again after an error as after a miss (core/src/polling_monitor/mod.rs).

Only rounds in which every gateway answered 404 now count toward
ENVIO_SUBGRAPH_FILE_ATTEMPTS. A failure asking again can mend (429, 5xx,
timeout) is asked again with backoff for ENVIO_SUBGRAPH_IPFS_TIMEOUT_MS,
the ipfs.cat budget; a file still out of reach then skips its handler
uncached, with a warning that says so.

Test: SubgraphFileRateLimit_test (stub gateway on an OS-assigned port).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t to

InternalTestIndexer.fromSubgraph wrote each fixture's project to
os.tmpdir()/envio-subgraph-* and never removed it (25.7k directories,
478 MB on one machine). A parse-only fixture's workspace is now removed
once the parse is done or has failed; one that runs tests is removed by
an afterAll on the suite that declared it, after its tests.

Test: SubgraphFixtureWorkspace_test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moose-code and others added 6 commits October 21, 2026 06:38
docs/subgraph-compat.md: the register pass moved ahead of §4a on integration; the block handlers' paragraph and the §7 row are re-applied there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ENVIO_SUBGRAPH_RPC, ENVIO_SUBGRAPH_NETWORK and ENVIO_API_TOKEN come from
the process environment over the project's .env (ProjectEnv), as every
envio setting does; the batch size and the block handler header lag were
read with std::env::var alone, so a .env line setting them did nothing,
and a fixture's own environment couldn't set them either
(SubgraphEventConstructor set process.env instead).

translate() now takes the environment map parse_subgraph has, and the
project-files path fills it from ProjectEnv with TRANSLATION_ENV_VARS.

Test: SubgraphEnvSettings_test (.env and the fixture env both set them).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s minimal bytes

graph-node builds every BigInt it hands a mapping from chain data with
BigInt::from(u64) or BigInt::from_unsigned_u256 and writes it as
to_signed_bytes_le (chain/ethereum/src/runtime/abi.rs: Log's
transaction_index/log_index via asc_new_or_null_u64, EthereumBlockData's
number), so 189 is bd00 and 258 is 0201. Following 79a64fd
(event.logIndex), two sites still used fromI32's four bytes:
receipt.logs entries' transactionIndex, logIndex and transactionLogIndex,
and a block handler's block.number. A mapping reading them as bytes
(Bytes.fromBigInt, byteLength, changetype<Bytes>) built other ids.

Test: SubgraphChainBigIntWidth reads all 60 chain BigInts an event, its
receipt and receipt logs, a block handler's block and a call hand over
(an RPC-only chain, stub RPC on an OS-assigned port) and checks each
one's bytes against num_bigint's to_signed_bytes_le; only these 8 differed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CLI (check_generated_code) and the runtime (a mapping importing the
tree it couldn't build) each wrote the message, and they had come apart:
"npx graph codegen" in Rust, "pnpm exec graph codegen" in TypeScript.
Neither works when the message shows, with graph-cli not installed: npx
reads "graph" as a package of that name. The runtime now asks the CLI's
code for it (napi subgraph_missing_graph_cli_message), and the manual
command names the package: npx @graphprotocol/graph-cli codegen.

Test: SubgraphMissingGraphCliMessage.test.ts pins both entries to one text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chain

createTestIndexer() reads the project synchronously, and reading a
subgraph asks ENVIO_SUBGRAPH_RPC its eth_chainId. A test's own RPC stub
answers on the thread that read blocks, so the ask waited out its 15 s
deadline (scenarios/subgraph_test, and any user test with a stub beside
createTestIndexer()).

getConfigJson now passes the ports servers in the calling process listen
on (process._getActiveHandles; nothing when Node lacks it), and an
endpoint on a local address and one of those ports isn't asked: it counts
as an endpoint that doesn't answer, so a known network name keeps its
table chain, and an unknown one's error says why it wasn't asked. An
endpoint in another process is asked as before, so its chain still wins
over an unknown or mismatched name.

Tests: cargo an_endpoint_this_process_serves_is_not_asked (a silent
listener, 30 s -> 0 s); subgraph_test's createTestIndexer() 15 s -> <1 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moose-code and others added 8 commits October 21, 2026 06:53
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r a sure create

d8d46d8 gave a data source's block handler a register pass at each of
its blocks whenever the create analysis said its code can reach a create,
and that analysis counts anything it can't follow as one. kiln's vsuite
handleBlock (unfiltered) verifies BLS12-381 signatures in a module the
analysis can't follow and creates nothing; its register pass, reading a
store without the entities its handler pass reads, threw before advancing
its tracker and redid the pairings at every block: the indexer stopped at
101 events where integration syncs the window in 22 s.

A data source's unfiltered or polling handler that runs on more than 1,000
blocks now gets a pass only when its code surely reaches a template's
generated class (creates::sure_creates: tinlake's handleBlock reaches all
six). A once handler, or one on a data source of at most 1,000 blocks
(beefy-balances' one-block binder), keeps its pass on suspicion. A create
from a block handler left without a pass is refused where it runs, as a
template's unfiltered handler's is, rather than lost.

Tests: SubgraphBlockHandlerUnsureCreate_test (an unfiltered handler's raw
DataSourceTemplate.create is refused at the create; it got a pass before);
SubgraphPollingBlockHandlerCreate_test now reaches the template through
its generated-style class. kiln-vsuite: ok 101 events -> synced 5,465 in
27 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6548d15 gave a data source's unfiltered or polling handler on more than 1,000 blocks a pass only for a sure create; §4 and the §7 row say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test parsed its project with the addon at ENVIO_DEV_ADDON, the debug
build globalSetup compiles. Where envio's platform package is installed
(CI installs the release build of the commit under test), every other
test loads that one, and without the debug build this test failed with
"Cannot find module target/debug/envio.node". The child process now
imports envio's Core and parses through getAddon(), which picks the
platform package first and the dev build otherwise, as the CLI does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
globalSetup ran `cargo build --lib` before every vitest run. envio loads
its platform package before any dev build (Core.loadAddon), and CI
installs that package from the release build of the commit under test,
so on CI the build produced a debug addon no test loads. On a fresh
runner with no cargo cache it was a cold build of the whole crate: 3m14s
of the envio-tests job at 76a793f and 5m00s at d3e211a, which put
the job past its 9-minute limit.

globalSetup now resolves envio's platform package the way Core.loadAddon
does (envio-<platform>-<arch>, then -musl) and builds only when neither
is installed, which is the case where workers fall back to the dev build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
InternalTestIndexer imports TypeChecker.ts, so every test file that uses
it (294 of 440) loaded the compiler while vitest collected it: about
250 ms per forked process. Only the fromUserApi fixtures with handlers or
a test, and completionsAt, type-check; the subgraph fixtures and
parse-only fixtures never do. The compiler, its host and the language
service are now built on the first check, and the TypeScript version pin
is checked there.

Full `pnpm test` on Postgres, 4 CPUs (taskset -c 0-3): vitest's summed
import time 221 s -> 145 s, wall 153 s -> 117 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
solana.hypersync.xyz resolves to five addresses and one of them,
193.32.220.135, accepts no TCP connection (checked with curl --resolve
against each). A test whose first connection lands there waits about
30 s before the source retries on another address. SvmWhereLive and
SvmBytesLive allow 300 s for that; SvmAccountKeysLive ran under vitest's
30 s default and failed with "Test timed out in 30000ms" in 2 of 6
local runs of the three live SVM files. CI hit the same stall on
SvmWhereLive (31.4 s) in the cancelled run at d3e211a.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ainStuckAtHead (#1690)

Cherry-pick of main's 04798e0, which this branch's base (merge-base
04e33b9, 5 Oct) predates:

- globalSetup skips the dev addon cargo build when envio's platform
  package is installed, as CI installs it.
- SvmAccountKeysLive gets a 300 s timeout.
- MultichainStuckAtHead runs the scenario with a 2 s realtime stall
  timeout (newBlockStallTimeoutRealtime threaded through IndexerState and
  ChainState to SourceManager, as reducedPollingInterval is) instead of
  waiting out the real 10..20 s twice: 34 s -> 3.9 s locally.

globalSetup.ts and SvmAccountKeysLive_test.res conflicted with this
branch's own equivalents (face3e6, d587bda) and are resolved to
main's text, so a later merge of main finds the same lines on both sides.

(cherry picked from commit 04798e0)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moose-code and others added 2 commits October 21, 2026 08:51
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…new fork's address is stored

Array.includes compared the rows by reference, so its loop always ran to the 20 s deadline (the assertion after it compares deeply and passed). 20 s -> 80 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DZakh and others added 2 commits October 21, 2026 09:07
* Don't start a write once the indexer has stopped

Chain metadata writes are throttled: staging arms a timer that later
schedules the write fiber. A restage after stop()'s flush - finalizing
the backfill commits ready_at, and the loop restages the chain's
metadata - left that timer armed, so the write started after stop() had
returned. In envio-tests the schema is dropped by then, the write fails,
and the harness's default onError calls process.exit, which kills the
vitest worker with no test named (seen as "Worker exited unexpectedly"
on ResponseBlocksMetric_test).

Writing.schedule now starts nothing once the indexer is stopped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ETueLEEQ8tbFKY69q1ys9k

* Fail the test on an indexer error instead of exiting the worker

IndexerRunner's default onError logged the error and called
process.exit, and test logs are silenced, so any indexer failure in a
test without its own onError killed the vitest fork with only "Worker
exited unexpectedly" - no test name, no error.

The default now records the first failure and fails the run with it.
The body is raced against the failure so a test waiting on progress the
failed indexer will never make fails right away instead of timing out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ETueLEEQ8tbFKY69q1ys9k

* Add Somnia Testnet (50312) as a HyperSync chain

The HyperSync API now lists it, which failed the hypersync health
check's drift check against the Network enum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ETueLEEQ8tbFKY69q1ys9k

* Refuse a restart once the test run is tearing down

The run stops waiting on the body once the indexer fails, but the body
keeps running. A restart() it made after that started an indexer and a
Postgres client that teardown had already passed, so nothing stopped or
closed them. make() now refuses to start, or to start the loop after its
awaits, once teardown has begun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ETueLEEQ8tbFKY69q1ys9k

---------

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit 773752a)
FinalizeIndexLogging_test and WorkerResumeLogging_test each read a pino
log file while its writer thread was still appending. A read landing
mid-line handed the partial line to JSON.parseOrThrow, which threw
("Unexpected end of JSON input") instead of letting the poll read
again - seen failing envio-tests (clickhouse) on #1693.

The two copies of the reader are now one helper, LogFile.messages, which
parses only lines that end in a newline.

Claude-Session: https://claude.ai/code/session_01ETueLEEQ8tbFKY69q1ys9k

Co-authored-by: Claude <noreply@anthropic.com>
(cherry picked from commit b0bd566)
@moose-code moose-code changed the title Subgraph mode: every compatibility fix in one PR (20 → 89 of 130 real subgraphs) Subgraph mode: compatibility fixes, 30% → 83% of 860 real subgraphs (85% of 2026 repos) Oct 21, 2026
@moose-code
moose-code requested a review from DZakh October 21, 2026 09:47
@DZakh
DZakh merged commit fdca61e into claude/hyperindex-subgraph-mode-bs67pq Oct 22, 2026
9 of 10 checks passed
@DZakh
DZakh deleted the moose/subgraph-compat-all branch October 22, 2026 11:33

This branch was successfully deployed

1 active deployment
internal — c2ea844b Deployed Oct 21, 2026 by moose-code via authorize #3804
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants