feat: signed outbound webhooks for market, trade, resolution and trace events (#128)

[alansamdev]

Summary

Adds signed outbound webhooks for market.created, trade.executed, market.resolved and trace.published:

• Storage: a durable Postgres outbox and delivery queue.
• Signing: HMAC-SHA256 with a timestamp, plus secret rotation.
• Delivery: exponential backoff with jitter (honouring Retry-After), one delivery in flight per subscription in event order, dead-lettering, and auto-disable of failing endpoints.
• Security: a strict SSRF guard with DNS pinning; secrets encrypted at rest.
• API: an owner-scoped API with a delivery log and redelivery.

Changes

Schema (prisma/migrations/20261018000000_webhooks)

• webhook_subscriptions: URL, eventTypes[], marketIds[], categories[], encrypted secret / previousSecret / secretRotatedAt, status (+ disabledReason), consecutiveFailures, failingSince.
• webhook_events: the outbox. id = "<type>:<chain_events.id>" is the dedup key.
• webhook_deliveries: unique per (event, subscription), indexed for the due-delivery claim and per-subscription ordering.
• The migration SQL was generated with prisma migrate diff from the schema change, so CI's schema/migration check stays green. The schema diff is purely additive.

SSRF guard (src/lib/ssrf-guard.ts)

• https: only. http: is allowed only outside production with WEBHOOKS_ALLOW_HTTP=true. No credentials in the URL; ports 443/8443 only in production.
• Resolves the host and rejects if any address is private, loopback, link-local, metadata (169.254.169.254), CGNAT, multicast or reserved, including IPv4-mapped IPv6 (::ffff:127.0.0.1, ::ffff:7f00:1) and numeric aliases (2130706433, 0x7f.1, normalised by WHATWG URL parsing first). Checks run on resolved IPs, never on strings.
• safeFetch pins the socket to the validated IP with a custom lookup (SNI/Host keep the hostname), so DNS rebinding between check and connect has no effect. It re-validates before every delivery, never follows redirects (3xx counts as a failure), caps the response read at 64 KB, and has a 10 s timeout.

Service (src/services/webhooks.service.ts)

• Outbox: enqueueEvent does INSERT … ON CONFLICT DO NOTHING for the event, then INSERT … SELECT deliveries for ACTIVE subscriptions matching type, market id and category. Replayed chain events insert nothing.
• Worker (started from server.ts next to the indexer, stopped on shutdown, off when WEBHOOKS_ENABLED=false):
  • Every 2 s it leases up to 20 due deliveries with UPDATE … WHERE id IN (SELECT … FOR UPDATE SKIP LOCKED).
  • Only the oldest pending delivery per subscription (by creation order) is eligible, so one receiver that's down never blocks the others, and multiple instances can't double-send.
• Envelope: { id, type, createdAt, data } as canonical bytes (canonicalBytes). The exact bytes sent are the bytes signed.
• Headers: OracleDesk-Signature: t=<unix>,v1=<hex> over t + "." + rawBody, plus OracleDesk-Event-Id, OracleDesk-Event-Type and User-Agent: OracleDesk-Webhooks/1.
• Rotation: for 24 h after a rotation, the signature header carries both v1=<new>,v1=<old>.
• Retries: only 2xx counts as success. Backoff starts at 1 min and doubles up to 24 h with ±20% jitter, and a longer Retry-After wins. After 15 attempts → DEAD.
• Auto-disable: after 50 consecutive failures or 5 days of failing → DISABLED with a reason, plus a warning in the logs.
• Encryption: signing secrets are stored with AES-256-GCM, keyed by WEBHOOK_SECRET_KEY.
• Verifier: verifyWebhookSignature(rawBody, header, secrets, tolerance=300) is the reference verifier, used in tests.
• Retention: delivered and dead deliveries are pruned daily after 30 days.

Indexer (src/services/indexer.service.ts)

• Trade → trade.executed, with the payload identical to TRADE_EXECUTED; the payload is now built once and used for both the socket and the webhook.
• TracePublished → trace.published, identical to REASONING_PUBLISHED.
• MarketResolved → market.resolved { onChainMarketId, marketId, outcome, voided, resolvedAt, txHash, ledger }. It's emitted even when the market is unknown to the backend.
• MarketCreated → market.created { onChainMarketId, marketId, creator, category, closeTime, seedAmountRaw, initialYesBps, txHash, ledger }. Field names were taken from the market-core contract spec.
• Enqueue failures are logged and never break indexing.

API (/api/v1/webhooks, requireAuth, owner-scoped)

• POST /webhooks → 201. Runs assertSafeUrl and returns the plaintext secret once. Max 10 per user.
• GET /webhooks, GET/PATCH/DELETE /webhooks/:id. PATCH status=ACTIVE re-enables and resets failure counters.
• POST /webhooks/:id/rotate-secret → { secret }.
• GET /webhooks/:id/deliveries?status=&page=&limit= and POST /webhooks/:id/deliveries/:deliveryId/redeliver.

Config

• WEBHOOKS_ENABLED (default true).
• WEBHOOK_SECRET_KEY: 32 bytes, base64. Required in production when webhooks are enabled; ephemeral otherwise, like JWT_SECRET.
• WEBHOOKS_ALLOW_HTTP: rejected in production.
• All three are validated in src/config/index.ts, documented in .env.example, and the key is added to the logger redact list.

Docs

• New Webhooks section in docs/api.md: event types and payloads, endpoints, the request format, signature verification (Node and Python), retries/ordering/disable policy, URL rules, and config.

Testing

New tests/webhooks.test.ts: 15/15 pass.

• Signing: the reference verifier accepts valid signatures and rejects a tampered body, wrong secret, stale timestamp or missing timestamp; dual signatures during rotation, with the old secret dropped after 24 h; AES-GCM is randomised and tamper-evident.
• SSRF: isBlockedAddress across private, loopback, link-local, metadata, CGNAT, multicast, IPv6 ULA and link-local, and mapped forms. assertSafeUrl rejects http in production, credentials, bad ports, the metadata IP, decimal and hex aliases, [::ffff:127.0.0.1], [::1], hostnames that partly resolve to private IPs, and unresolvable hosts.
• Rebinding: a resolver that returns a public IP first and 127.0.0.1 second. The pinned lookup still returns the validated IP, and DNS is consulted only once.
• Redirects: a local 302 to the metadata IP is returned, not followed.
• Retry policy: the backoff schedule and jitter bounds; Retry-After in seconds and as an HTTP date; DEAD at 15 attempts; auto-disable at 50 failures and at 5 days.
• Outbox: enqueueEvent dedups by chain event id (a replay inserts nothing); the canonical envelope bytes are stable.
• End to end: deliverOne sends to a local http.createServer receiver. The signature verifies, the headers are right, data matches the TRADE_EXECUTED shape, the delivery is SUCCEEDED and the failure streak resets. A 503 with Retry-After schedules a retry and counts a failure.
• Integration: publishWebhookEvent does nothing when disabled and never throws.

tests/config.test.ts gains a webhook config validation test, and the existing ephemeral-secrets assertion now includes WEBHOOK_SECRET_KEY.

Test-suite note (pre-existing on develop)

develop currently has 14 TypeScript errors in files this PR doesn't touch: monitor.cron.ts, copy-trade.service.ts, resilience.ts, stellar/clients.ts, subscription.service.ts and trace-audit.service.ts. Because of them, npm run build and npm test (which uses noEmitOnError) already fail on develop. To test this PR I compiled with --noEmitOnError false and ran the full suite:

	tests	pass	fail
develop	39	29	10
this PR	55	45	10

The 10 failures are the identical set on both (audit-constraints, constraints, copy-trade-verify, resilience, trace-audit, and 5 HTTP route tests), so this PR introduces no regressions. All 16 new tests pass, and the existing indexer and config tests still pass. tsc --noEmit reports the same 14 errors before and after, with no new type errors.

Not in this PR (follow-ups)

• Same-transaction enqueue: the event is enqueued right after chain_events is stored and its side effect runs, not in the same DB transaction. Dedup by chain event id makes the enqueue idempotent, but a crash between storing and enqueueing could drop that one webhook. Folding store and the side effects into one interactive transaction touches the whole indexer and its mocks, so it's best done separately.
• Pending cap: the 10,000-per-subscription limit on pending deliveries during catch-up, and batched inserts.
• Test-ping endpoint (POST /webhooks/:id/ping).
• Email notice on auto-disable: for now it's a log warning plus disabledReason.

---

Closes #128
Closes #124
Closes #125
Closes #127

[drips-wave]

@alansamdev Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits