Skip to content

feat(snaps): optional public-endpoint fallback for bitcoin and stellar reads - #528

Closed
rvelaz wants to merge 12 commits into
mainfrom
feat/stale-balances-public-endpoint-fallback
Closed

rvelaz wants to merge 12 commits into
mainfrom
feat/stale-balances-public-endpoint-fallback

Conversation

@rvelaz

@rvelaz rvelaz commented Oct 26, 2026 •

Copy link
Copy Markdown

Draft. Manifest shasum not yet regenerated, so snap checks may fail.
Updated on top of #331 by a merge from main (via the updated 07 branch, no rebase) and re-tested: snap-networks-utils 44 suites / 571 tests, bitcoin 28 / 746, stellar 94 / 1533 pass; tsc --noEmit clean.
Optional / exploratory. Depends on PR 05 and PR 07 and includes their commits (it was built on a local merge of both, which GitHub cannot stack on two branches). Review those first; the diff here will shrink once they merge.

  • Packages: @metamask/snap-networks-utils, @metamask/bitcoin-wallet-snap, @metamask/stellar-wallet-snap
  • Status: optional / for discussion. It is off unless configured, and it has a privacy trade-off that needs a product and security decision before it could ship.

Explanation

Bug

While the primary Infura endpoint fails at the transport level (the coalesced empty 404 without CORS), the bitcoin and stellar snaps have no other way to read the chain. With PRs 01–07 the balance is correctly "unavailable", but it stays that way until the connection heals.

How to reproduce

  1. Build this branch's bitcoin and stellar snaps with the fallback variables set (ESPLORA_BITCOIN_FALLBACK, STELLAR_HORIZON_FALLBACK_URL_MAINNET, STELLAR_RPC_FALLBACK_URL_MAINNET; see .env.example). Wire them into an extension release/13.54.0 (acf000d) yarn build:dev, with core PR 10 and extension PR 12.
  2. Load it unpacked in Chrome or Chromium with default settings (HTTP/2 on). This is profile A.
  3. Import a test wallet that holds BTC, TRX, XLM and SOL (we used a team test wallet). Onboarding creates the Bitcoin, Tron, Stellar and Solana accounts.
  4. Let the snaps' requests coalesce. In Chrome, a request to bitcoin-mainnet / stellar-mainnet.infura.io can reuse the HTTP/2 connection already open to another *.infura.io host (in our runs the Solana snap usually opened it first). The reply is an empty 404 without CORS headers, so the snap sees Failed to fetch. We reproduced this on a test machine where every *.infura.io name resolved to the same IP. It depends on the network path and may not reproduce on every network. In our runs we moved the shared connection between hosts on purpose: we dropped the test browser's sockets while briefly blocking the other Infura hosts, which "heals" one chain and breaks the others.
  5. Keep the bitcoin and stellar connections broken (coalesced) for several minutes.
  6. Control (profile B): the same steps with Chrome started with --disable-http2 give no coalescing, and all four balances are correct.

Expected: after the primary's own retries fail, the reads go to the fallback, and BTC/XLM show correct balances even though Infura keeps failing.
Actual (before this change): BTC/XLM stay "Balance unavailable" until the Infura connection heals.

Without a browser: EsploraClientAdapter.test.ts › fallback endpoint › scans against the fallback endpoint when the primary keeps failing with a network error, and NetworkService.test.ts › read retries and fallback endpoints.

Summary

This adds an opt-in fallback endpoint for read calls: Esplora for bitcoin mainnet, Horizon and Soroban RPC for stellar. The fallback is used only after the primary Infura endpoint has failed at the transport level, after its own fetch-policy retries. Nothing changes unless the build sets the new environment variables.

Why

In Chrome, HTTP/2 connection coalescing across *.infura.io can send a snap's request over a connection opened for another Infura host. The provider router answers with an empty 404 and no CORS headers, which shows up as Failed to fetch. PRs 01–07 make the snaps honest about it: unknown or stale balances are reported as unavailable instead of a fake 0, and requests are retried with backoff. They cannot make the request succeed while the coalesced connection lasts. A non-Infura endpoint is on a different connection, so it is not affected.

What changed

  • snap-networks-utils/src/utils/fetchPolicy/withEndpointFallback.ts (new, exported):
    • shouldUseFallbackEndpoint(error): true for a bare Failed to fetch (which covers the CORS-blocked coalesced 404), a timeout, an open circuit, a 5xx or a 429. A readable 4xx is an answer about the request and never goes elsewhere.
    • withEndpointFallback(primary, fallback, { onFallback }): runs the fallback only for those errors. If both fail, it rethrows the primary error with fallbackError attached. With no fallback configured, it rethrows unchanged.
  • bitcoin:
    • ESPLORA_BITCOIN_FALLBACK: config.ts chain.fallbackUrl.bitcoin (an empty value disables it), entities/config.ts, snap.config.ts, .env.example;
    • EsploraClientAdapter creates a second endpoint with its own esplora_fallback_* fetch policies, and #read wraps primary/fallback. Broadcasts never fall back and are never retried.
  • stellar:
    • STELLAR_HORIZON_FALLBACK_URL_{MAINNET,TESTNET} and STELLAR_RPC_FALLBACK_URL_{MAINNET,TESTNET} in config.ts, snap.config.ts and .env.example;
    • NetworkService builds the fallback Horizon/RPC clients and #read wraps them. Submissions are not affected.
  • Tests: withEndpointFallback.test.ts; Esplora fallback tests (scan falls back after 1+1 primary attempts; no fallback configured; broadcast never falls back); 4 stellar NetworkService fallback tests.

Validation

Checked locally: this branch's changed test files were run with the source of its base (cd091d7, the merge of PRs 05 and 07), then on this branch.

Unit tests that fail before and pass after

  • EsploraClientAdapter › fallback endpoint › scans against the fallback endpoint when the primary keeps failing with a network error
  • NetworkService › read retries and fallback endpoints › uses the Horizon fallback endpoint when the primary keeps failing, uses the RPC fallback endpoint for getAccount, reports the primary error when the fallback also fails
  • withEndpointFallback.test.ts (snap-networks-utils) cannot run on the base because the module is new.

Full run on this branch

  • snap-networks-utils: 44 suites, 571 tests pass.
  • bitcoin: 28 suites, 746 tests pass (--testPathIgnorePatterns integration-test).
  • stellar: 93 suites, 1503 tests pass.
  • tsc --noEmit is clean for all three packages.

Manual A/B

Not exercised. The fallback variables were left unset in every local build (v1–v7), so no A/B number exists for this branch, and every number in PRs 01–07 comes from the default configuration. To evaluate it, run A (default HTTP/2) with a fallback set (for example https://blockstream.info/api, https://horizon.stellar.org). Measure the time to a correct BTC/XLM balance while Infura stays coalesced, and the share of requests that go to the fallback.

Risks

  • Privacy: a fallback outside Infura sees the wallet's addresses and/or script hashes and the user's IP. That is why it is unset by default and documented as such in .env.example.
  • Consistency: a public endpoint can lag or differ from Infura. Reads only; sends are never routed to it.
  • Rate limits on public endpoints. The fallback has its own fetch policy and circuit breaker.

Changelog (if adopted)

  • packages/snap-networks-utils/CHANGELOG.md, Unreleased → Added: Add withEndpointFallback and shouldUseFallbackEndpoint to retry a read against a fallback endpoint after a transport-level failure (#528)
  • packages/bitcoin-wallet-snap/CHANGELOG.md, Unreleased → Added: Add an optional ESPLORA_BITCOIN_FALLBACK read endpoint, unset by default (#528)
  • packages/stellar-wallet-snap/CHANGELOG.md, Unreleased → Added: Add optional Horizon / Soroban RPC fallback read endpoints, unset by default (#528)

References

Dependencies

  • Built on PRs 01–05 (bitcoin) and 06–07 (stellar). It reuses their fetch-policy #read paths.
  • A decision on whether MetaMask wants a non-Infura read path at all, and which provider.

Stale non-EVM balances series (Chrome HTTP/2 coalescing across *.infura.io):

Checklist

  • I've updated the test suite for new or updated code as appropriate
  • I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate
  • I've communicated my changes to consumers by updating changelogs for packages I've changed
  • I've introduced breaking changes in this PR and have prepared draft pull requests for clients and consumer packages to resolve them

local added 9 commits October 26, 2026 10:11
…tead of zero

- getAccountBalances throws AccountBalanceUnavailableException (an expected state, not sent to snap_trackError) when the account has no snapshot and is not confirmed unfunded, and schedules a deduplicated SynchronizeAccounts (at most once per 30 s per account), so Retry triggers a real sync.
- Sync persists a confirmed-unfunded marker (AccountNotActivatedException) and emits a native 0 once.
- An empty-body Horizon 404 is no longer read as 'account not found': it is a transport failure (e.g. a provider router rejecting a mis-routed, HTTP/2-coalesced request), not a confirmed unfunded account.
…le balances as unavailable

- loadOnChainAccount (Horizon) and getAccount (Soroban RPC) run under createFetchPolicy from #462: retries with backoff on transient failures and a circuit breaker. A confirmed missing account is a non-retryable 404; an empty-body 404 or a bare 'Failed to fetch' is a network failure. The original SDK error is rethrown so callers keep classifying it. The snap bundle provides the performance shim the policy needs.
- A balance is stale when the last successful sync is older than 90 s and the syncs since then have failed. getAccountBalances then throws AccountBalanceStaleException (expected state, untracked) and requests an on-demand sync through the existing 30 s dedupe. The failure-streak start is persisted per account only on transitions; the error clears on the next successful sync.
getAccountBalances throws the expected-state BalanceNotSynchronizedError
while the wallet still sits on the genesis checkpoint, instead of answering
a confident zero, and the cron holds back balance events for such accounts.
The error is matched by its stable message marker (the keyring SDK
re-creates it as a plain Error) and is never sent to Sentry.
…meouts

Scans, syncs, REST lookups and fee estimates run under createFetchPolicy
from @metamask/snap-networks-utils (bounded retries with backoff on
transient failures, circuit breaker); a broadcast is never retried. A whole
sync is capped at 60 s and a full scan at 120 s, so a request stalled by a
broken connection can no longer keep the coalesced sync stuck. performance
is provided to the bundle through a ProvidePlugin shim.
… scan

- FullScanQueue: persistent per-account full-scan queue (reuses the
  rescanV1 state) with backoff 30 s..5 min; a failing account rotates to
  the tail, so neither new accounts nor the repair migration are blocked by
  a stuck one; never-synced accounts are re-queued
- the balance stays unknown until the first full scan succeeds; the cron
  holds back partial balances
- one scan per account: single-flight full scans, discovery scans reused
  for account creation (10 min TTL), at most 2 immediate scans on create
- a balance request on an unknown account (the wallet's Retry) requests an
  on-demand full scan (30 s..2 min backoff)
- the first successful sync after a failed scan requests a recovery scan
  within 5 s
The first full scan of a new account uses the discovery stop gap (5)
instead of the standard gap (20). When that scan finds activity,
extendDiscoveryScan checks only the addresses the standard gap adds (one
Esplora history request each, 10 s timeout) and runs the gap-20 full scan
only if one of them has history or the check fails, so coverage is
unchanged. In the A/B runs this cut the first-2-minute bitcoin requests
from 257 to 137.
…ailing

SyncHealthStore tracks per account whether its syncs are failing and
since when its stored balance was last confirmed; the streak is persisted
on transitions so a snap restart does not reset it. After 90 s of failed
syncs since the last success (three missed 30 s syncs, matching the
client's 90 s re-check), getAccountBalances throws the expected-state
BalanceStaleError (not sent to Sentry) and schedules a sync, rate limited
to one per 10 s; the next successful sync clears the state and emits the
balances-updated event.
…r reads (draft)

- snap-networks-utils: withEndpointFallback / shouldUseFallbackEndpoint
  repeat a read against a fallback endpoint only after the primary failed at
  the transport level (bare Failed to fetch, timeout, open circuit, 5xx or
  429); a readable 4xx is never sent elsewhere; if both fail the primary
  error is rethrown
- bitcoin: ESPLORA_BITCOIN_FALLBACK (mainnet reads only; broadcasts never
  fall back)
- stellar: STELLAR_HORIZON_FALLBACK_URL_* / STELLAR_RPC_FALLBACK_URL_*
- unset by default: sending wallet addresses to a third-party endpoint has
  privacy implications
local added 3 commits October 26, 2026 11:13
…ke-zero

# Conflicts:
#	packages/stellar-wallet-snap/src/handlers/keyring/keyring.test.ts
#	packages/stellar-wallet-snap/src/handlers/keyring/keyring.ts
#	packages/stellar-wallet-snap/src/services/on-chain-account/OnChainAccountService.ts
…ests-staleness

# Conflicts:
#	packages/stellar-wallet-snap/src/handlers/keyring/keyring.test.ts
#	packages/stellar-wallet-snap/src/handlers/keyring/keyring.ts
#	packages/stellar-wallet-snap/src/services/on-chain-account/OnChainAccountService.ts
@rvelaz rvelaz closed this Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant