Cache live chain reads in Redis with ledger-aware invalidation and single-flight #98

Overview

Three public endpoints read contract state live on every request by simulating a Soroban call through Stellar RPC: GET /markets/on-chain/:onChainMarketId/state (market_core.get_market), GET /oracle/markets/:marketId/resolution (resolver.state and market_core.get_market), and GET /portfolio (treasury.available_capital). Each simulation builds a fresh binding client, round-trips to RPC and costs tens to hundreds of milliseconds. The frontend polls these, and a market page shared on social media can produce bursts of identical requests.

Today traffic translates one-to-one into RPC load. Public RPC providers rate-limit aggressively, so a traffic spike degrades every chain-dependent feature at once, including the indexer and agent writes that share the same rpcServer and the same provider quota. Identical concurrent requests each perform their own simulation even though they would return the same value for the same ledger.

This issue adds a read-through cache for chain views in Redis, keyed by contract, method and arguments, storing each value together with the ledger it was read at. Entries are invalidated by indexer events that affect them (trades, resolutions, resolver events) and bounded by a short TTL, with in-process and cross-instance single-flight so concurrent misses trigger one RPC call. Responses expose the ledger they reflect, so clients can reason about freshness. The contracts stay the source of truth: the cache only ever serves values that were read from the chain, never values derived in the backend.

Background & current behavior

• src/services/chain.service.ts:

  async function read<T>(contract: ContractKey, method: string, call: () => Promise<AssembledTransaction<unknown>>): Promise<T>

  builds the transaction via call() (which constructs a new binding Client from readClients.* in src/services/stellar/clients.ts and simulates), and unwraps with simulatedValue(), throwing ChainError (502 CHAIN_ERROR with contractError). The simulation response's latestLedger (available on tx.simulation) is discarded. Exported reads: marketCount, getOnChainMarket, getPrice, quoteBuy, getPosition, getOnChainTrace, resolverState, treasuryAvailableCapital.
• src/controllers/market.controller.ts getOnChainMarketState: parseU64, getOnChainMarket(id), returns printable({...market}) plus status: marketStatusView(...), yesBps from priceYesBps(reserve_yes, reserve_no), source: 'chain'.
• src/services/oracle.service.ts getResolutionStatus(marketId): DB lookup, then Promise.all([resolverState(id), getOnChainMarket(id)]).
• src/controllers/portfolio.controller.ts getPortfolio: treasuryAvailableCapital() with a catch that returns null.
• Values contain bigint (reserves, close_time, fees_accrued, sets_minted, available_capital), Buffer (question_hash, resolution_hash) and tagged unions (status: { tag, values }, category). printable() converts bigint → string and Buffer → hex for output but is lossy for round-tripping.
• src/services/indexer.service.ts applySideEffects() already reacts to market-core Trade and MarketResolved and reasoning-registry TracePublished; treasury and resolver events are stored in chain_events without side effects.
• Backend writes (agentCreateMarket, agentBuy, agentSell, publishTrace) change the same state the cache would hold.
• src/lib/redis.ts provides getRedis(); Redis is already required (SEP-10 challenges).

Goals

• src/lib/chain-cache.ts with a typed, lossless codec for binding values (bigint, Buffer, tagged enums, nested structs).
• cachedRead(key, ttl, loader) that stores { value, ledger, readAt }, with in-process and Redis single-flight.
• Event-driven invalidation per market and per contract, plus write-through invalidation after the backend's own writes.
• Cache applied to getOnChainMarket, resolverState and treasuryAvailableCapital (and available to other reads).
• Responses include ledger and cached: boolean.
• Never return a value older than an invalidation the process has seen.
• Tests with a counting fake RPC.

Non-goals

• Batching reads via getLedgerEntries (separate issue); this caches whatever read() does.
• RPC failover pools (separate issue).
• Caching user-specific reads (getPosition) in a shared cache unless keyed by holder; out of scope for the first iteration.
• HTTP-level caching headers (separate ETag issue).

Proposed implementation

1. Capture the ledger: change read() to return { value: T; ledger: number } internally (latestLedger from tx.simulation when it is a success response; 0 if missing) and add readWithLedger() exports alongside the existing value-only exports so current callers keep compiling.
2. Codec in new src/lib/chain-cache.ts:

   export function encode(value: unknown): string; // JSON with tagged wrappers: {"$bigint":"123"}, {"$buf":"hex"}
   export function decode<T>(text: string): T;

   Round-trip tests over real binding shapes (Market from src/generated/market-core.ts, ResolutionState, i128). Do not reuse printable().
3. Cache API:

   export interface CacheEntry<T> { value: T; ledger: number; readAt: number; gen: number }
   export interface ChainCache {
     get<T>(key: string, opts: { ttlMs: number; tags: string[] }, load: () => Promise<{ value: T; ledger: number }>): Promise<CacheEntry<T> & { cached: boolean }>;
     invalidateTag(tag: string, atLedger?: number): Promise<void>;
   }
   export function redisChainCache(redis: Redis): ChainCache;
   export function memoryChainCache(): ChainCache;

   Keys: chain:v1:<network-hash>:<contractId>:<method>:<sha256(encode(args))>; include a hash of STELLAR_NETWORK_PASSPHRASE and the contract id (not the logical name) so overrides or redeployments never serve stale entries.
4. Invalidation via tag generations (avoids scanning keys): each tag (e.g. market:12, resolver:12, treasury:capital) has a generation counter chain:gen:<tag> in Redis. An entry stores the generations of its tags at load time; on read, MGET the current generations and treat the entry as a miss if any differs. invalidateTag does INCR. Also record chain:minLedger:<tag> = atLedger so a load that started before the invalidation but returns a value read at an older ledger is not written back (compare ledger with minLedger).
5. Single-flight: in-process, a Map<string, Promise<...>> coalescing identical concurrent misses. Cross-instance, a Redis lock SET chain:lock:<key> <token> NX PX 5000; non-holders poll the cache key for up to CHAIN_CACHE_WAIT_MS (default 1500) and then load themselves (never block indefinitely on a dead holder). Errors are not cached; ChainError with a contract error (e.g. MarketNotFound) may be negatively cached for a short TTL (CHAIN_CACHE_NEG_TTL_MS, default 2000) to blunt enumeration of unknown ids.
6. Wire the reads in src/services/chain.service.ts:

   export const getOnChainMarketCached = (id: bigint) => cache.get(key('marketCore','get_market',{id}), { ttlMs: CHAIN_CACHE_TTL_MS, tags: [`market:${id}`] }, () => readWithLedger(...));

   Similarly resolverStateCached (tags resolver:<id>, market:<id>) and treasuryAvailableCapitalCached (tag treasury:capital). Default TTL CHAIN_CACHE_TTL_MS = 5000 (about one ledger) in src/config/index.ts and .env.example; CHAIN_CACHE_ENABLED defaults to true and to false under NODE_ENV=test unless the test opts in.
7. Indexer invalidation in applySideEffects() (before emitting, so clients that refetch on the socket event see fresh data): market-core Trade, MarketResolved, Redeemed, FeesSwept, MarketCreated → market:<id>; resolver ResolutionProposed, ResolutionFinalized, PriceResolution → resolver:<id> and market:<id>; treasury TradeAuthorized, PositionClosed, FeesCollected and market-core Trade with the treasury as trader → treasury:capital. Pass event.ledger as atLedger. Note: invalidation for events that currently have no side-effect branch must be added as a generic pre-step, not inside the existing branches.
8. Write-through invalidation: in write() after a successful live submission, invalidate the tags the method affects (agent_create_market → treasury:capital; agent_buy/agent_sell → market:<id>, treasury:capital), so the backend never serves its own pre-write state right after writing.
9. Controllers: getOnChainMarketState returns ledger and cached; getResolutionStatus returns ledger (the minimum of the two reads) and cached; getPortfolio adds availableCapitalLedger. Update docs/api.md for each response.
10. Observability: count hits, misses, coalesced waits and stale-rejects in a small exported chainCacheStats() for the metrics issue.

Edge cases & pitfalls

• RPC nodes behind a load balancer can return different latestLedgers; never write back an entry whose ledger is lower than an already cached entry for the same key.
• Indexer lag means invalidation arrives seconds after the chain changed; the short TTL bounds staleness. Document the worst case (TTL + indexer poll interval).
• Buffer values in bindings are sometimes Uint8Array; the codec must normalise both.
• Redis down: bypass the cache entirely (direct reads) rather than failing requests; log once per minute.
• Key explosion: only cache whitelisted methods; quoteBuy with arbitrary amounts must not be cached.
• Tests set STELLAR_RPC_URL to http://127.0.0.1:1; inject the loader so no real RPC is touched.

Testing plan

New tests/chain-cache.test.ts using memoryChainCache() and, when REDIS_URL is reachable, redisChainCache(), with a counting fake loader:

• codec round-trips Market, ResolutionState and i128 values losslessly
• second read within TTL is served from cache with cached true and the original ledger
• 100 concurrent misses trigger exactly one loader call
• invalidateTag forces a reload on the next read
• a load that started before invalidation and returns an older ledger is not cached
• contract errors are negatively cached only for the negative TTL
• Redis unavailable falls back to direct reads
• two cache instances sharing Redis coalesce a miss across instances (Redis-gated)
• Indexer: Trade event invalidates market tag before TRADE_EXECUTED is emitted (in tests/indexer.test.ts)
• HTTP: GET /markets/on-chain/:id/state includes ledger and cached

Commands: npm test, npm run build.

Acceptance criteria

• read() captures the simulation's latestLedger; existing read exports keep their signatures.
• src/lib/chain-cache.ts provides a lossless codec, tag-generation invalidation, and in-process plus Redis single-flight.
• getOnChainMarket, resolverState and treasuryAvailableCapital are served through the cache in the three endpoints.
• The indexer invalidates affected tags for market-core, resolver and treasury events before emitting socket events.
• Live backend writes invalidate the tags they affect.
• Older-ledger values never overwrite newer entries or survive a newer invalidation (tested).
• Responses expose ledger and cached; docs/api.md is updated.
• CHAIN_CACHE_* env vars are validated and documented in .env.example.
• npm run build, npm test and CI pass.

Resources

• src/services/chain.service.ts, src/services/stellar/clients.ts, src/services/indexer.service.ts, src/services/oracle.service.ts
• src/controllers/market.controller.ts, src/controllers/portfolio.controller.ts, src/lib/redis.ts, src/generated/market-core.ts, docs/api.md
• Stellar RPC simulateTransaction (response includes latestLedger): https://developers.stellar.org/docs/data/apis/rpc/api-reference/methods/simulateTransaction
• Redis SET with NX/PX for locks: https://redis.io/docs/latest/commands/set/
• Stellar JS SDK contract.AssembledTransaction: https://stellar.github.io/js-stellar-sdk/

Generated by Claude Code