feat(api): Idempotency-Key support on state-changing endpoints (+ recordPayment P2002 race fix)

[chonilius]

Summary

Implements Idempotency-Key support (IETF draft-ietf-httpapi-idempotency-key-header, the Stripe model) on every state-changing endpoint, backed by Postgres. It also fixes the recordPayment concurrent-duplicate race, which returned a 500, whether or not a key is sent.

Base branch: develop, as the issue requires.

Why

Retries are routine (flaky mobile connections, React Query mutation retries, restored tabs), and today a retry repeats the side effect:

• POST /trade/copy → another PENDING CopyTrade per retry.
• POST /markets/generate → another runMarketMakerCycle(): more paid LLM calls and, in live mode, another treasury-funded market.
• POST /traces/:id/unlock: two concurrent requests with the same txHash both pass the pre-check, and the loser hits the PaymentEvent.txHash unique index (P2002) → 500 INTERNAL_ERROR.

What changed

1. feat(db): idempotency_keys table

• IdempotencyKey model exactly as specified in the issue: unique (scope, method, route, key), index on expiresAt, @@map("idempotency_keys").
• Forward-only migration 20261017000008_idempotency_keys, generated with prisma migrate diff from the previous schema, so it matches the datamodel byte-for-byte for CI's migrate diff --exit-code drift check.

2. feat(api): idempotent() middleware (src/middlewares/idempotency.middleware.ts)

Situation	Result
No header, required	400 IDEMPOTENCY_KEY_REQUIRED
No header, optional	passes through unchanged
Malformed key (not 1–255 printable ASCII)	400 IDEMPOTENCY_KEY_INVALID
First request	claims the key with INSERT … ON CONFLICT DO NOTHING RETURNING id, runs the handler, stores the final response
Exact repeat after completion	replays stored status + body + allowlisted headers with Idempotent-Replayed: true; handler not run
Same key, different request	422 IDEMPOTENCY_KEY_MISMATCH
Repeat while the first is in flight	409 IDEMPOTENCY_IN_PROGRESS, Retry-After: 1
Original crashed (lock expired)	conditional UPDATE … WHERE lockedUntil < now() takes over; only one retry can win
Stored body was > 64 KB	409 IDEMPOTENCY_RESPONSE_UNAVAILABLE

• Scope = (req.user.userId ?? 'anon', method, route template, key). The route template is req.baseUrl + req.route.path (e.g. /api/v1/trade/copy/:id/confirm), so the same key on another route or from another user never collides.
• Fingerprint = sha256Hex(canonicalJson({ method, route, params, query, body })), reusing src/utils/hash.util.ts. The parsed req.query is used and canonicalJson sorts keys, so JSON key order and query-parameter order don't change it.
• Response capture wraps res.json (every handler and errorMiddleware end in sendSuccess/sendError → res.status().json()). The record is written on res.on('finish').
• What's stored: 2xx and 4xx only. 5xx deletes the row so a retry re-executes (the draft's guidance for transient errors).
• Safety: replayed headers are allowlisted (Content-Type, Location, API-Version); Set-Cookie/Authorization are never stored. Body values under keys matching token|secret|authorization|password|private key are redacted before storage, as defence in depth like the pino redact list. Bodies are capped at 64 KB.
• TTL 24 h; lock 60 s by default, above POST /traces/verify's 20 s IPFS timeout.
• Concurrency: the ON CONFLICT insert is the only synchronization point (no in-memory map), so this is safe across multiple instances. All SQL is raw ($queryRaw/$executeRaw), behind a small IdempotencyStore interface with a setIdempotencyStore() test seam (same pattern as setTransactionReader).

Routes

Route	Header
POST /markets/generate	required. The stored 202 contains the jobId, so a retry returns the same job and starts no second cycle.
POST /trade/copy	required
PATCH /trade/copy/:id/confirm, PUT /traces/access/allowance, POST /traces/verify, POST /traces/:id/unlock	optional
POST /auth/challenge, POST /auth/verify	excluded, with a comment explaining why: single-use handshake, and /auth/verify returns a JWT that must never be stored

The middleware runs after requireAuth/requireAdmin (so it can scope by user) and after express.json().

CORS (src/app.ts): allowedHeaders: ['Content-Type', 'Authorization', 'Idempotency-Key'] and exposedHeaders: ['Idempotent-Replayed', 'Retry-After']. Note this replaces the cors default (reflecting whatever the browser requests) with an explicit list. I grepped src/: the only custom request header read is Authorization, so nothing the app uses is dropped.

Cleanup (src/cron/monitor.cron.ts): 10 * * * * runs purgeExpiredIdempotencyKeys(), deleting expired rows in batches of 5 000 (capped at 100 batches per run) and logging the count.

3. fix(payments): recordPayment race (independent of the header)

In src/services/subscription.service.ts, P2002 from the $transaction is caught, the winning paymentEvent is re-read by txHash, and the call either returns the existing subscription (same user) or throws 409 PAYMENT_ALREADY_USED (another user). A concurrent duplicate without a key no longer 500s. Behaviour is otherwise unchanged.

4. test,docs

• docs/api.md: new Idempotency section covering the route table, key format, scope, TTL, fingerprint, the full response/error table, 5xx and 4xx-replay semantics ("use a new key after changing state"), the redaction/header policy, and a React Query snippet (crypto.randomUUID() per action, reused across retries).
• tests/idempotency.test.ts: 14 tests with an in-memory store, covering every case in the issue's testing plan:
  replay + Idempotent-Replayed (including JSON key order not mattering) · mismatch 422 · in-flight 409 + Retry-After · crashed-lock takeover · 5xx not stored / retry re-executes · 4xx stored and replayed · per-user and per-route scope (and a different :id → 422) · no auth/cookie headers stored, secrets redacted, > 64 KB not replayed · optional/invalid keys · purge deletes only expired rows · POST /trade/copy without a key → 400 (real app) · POST /markets/generate replay returns the same jobId and runMarketMakerCycle runs once (real app, cycle monkeypatched) · CORS preflight and exposed headers · recordPayment simulated P2002 → existing subscription / 409.

Testing

On Node 22 (the .nvmrc / CI version):

node --test dist-tests/tests/idempotency.test.js
# tests 14   # pass 14   # fail 0

Full suite on Node 22, before vs after:

	tests	pass	fail
develop (baseline)	39	34	5
this branch	53	48	5

The same 5 files fail in both (audit-constraints, constraints, copy-trade-verify, resilience, trace-audit). They're pre-existing and unrelated: the constraint suites need a live Postgres, and the others match the type errors below.

⚠️ Pre-existing issues on develop (not introduced here)

• npm run build / npm test already fail on develop: tsc reports 16 type errors (in copy-trade.service.ts, monitor.cron.ts's existing sweepCopyTrades, stellar/clients.ts, subscription.service.ts:265 SettlementCurrency, trace-audit.service.ts, tests/copy-trade-verify.test.ts, tests/resilience.test.ts). The set is identical with and without this PR (diffed). Because tsconfig.test.json sets noEmitOnError: true, npm test can't compile on develop at all. I ran the suites by compiling with --noEmitOnError false.
• On Node ≥ 24/26, src/middlewares/abort.middleware.ts (req.signal = controller.signal) throws Cannot set property signal of #<IncomingMessage> which has only a getter, so every request 500s (the existing http.test fails 5/7 there too). CI is on Node 22, so it's unaffected today, but it'll break on a Node upgrade. Worth a separate issue.

Not in scope (per the issue's non-goals)

Making the background market-maker cycle itself resumable, and idempotency for agent chain writes. This PR only guarantees that one key starts at most one generation job.

Closes #123
Closes #120
Closes #121
Closes #122

[drips-wave]

@chonilius Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits