Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,11 @@ TRACE_AUDIT_MAX_BYTES=2097152
COPY_TRADE_TTL_MINUTES=30
COPY_TRADE_MAX_PENDING=5

# ── Outbound webhooks ──────────────────────────────────────────────────────
# Signed webhook deliveries for market/trade/resolution/trace events.
WEBHOOKS_ENABLED=true
# 32 random bytes, base64. Encrypts webhook signing secrets at rest.
# Required in production. Generate with: openssl rand -base64 32
WEBHOOK_SECRET_KEY=
# Allow http:// webhook URLs (local receivers). Never in production.
WEBHOOKS_ALLOW_HTTP=false
107 changes: 107 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -620,6 +620,113 @@ Emitted when the indexer stores a reasoning-registry publish event.
}
```

## Webhooks

Signed, server-to-server push for integrators who can't hold a socket open.
Delivery is **at-least-once**: deduplicate on `OracleDesk-Event-Id`.

### Event types

| Type | When | `data` |
|---|---|---|
| `market.created` | market-core `MarketCreated` indexed | `{ onChainMarketId, marketId, creator, category, closeTime, seedAmountRaw, initialYesBps, txHash, ledger }` (`category` is the contract tag: `Macro`, `Geopolitics`, `Crypto`, `Sports`, `Culture`) |
| `trade.executed` | market-core `Trade` indexed | identical to the `TRADE_EXECUTED` socket payload |
| `market.resolved` | market-core `MarketResolved` indexed | `{ onChainMarketId, marketId, outcome: "Yes"\|"No"\|null, voided, resolvedAt, txHash, ledger }` |
| `trace.published` | reasoning-registry `TracePublished` indexed | identical to the `REASONING_PUBLISHED` socket payload |

Each event is produced once per chain event (id `"<type>:<chain event id>"`),
so re-indexing the same ledgers never sends duplicates.

### Endpoints

All require `Authorization: Bearer <jwt>` and only see the caller's own subscriptions.

| Method & path | Body / query | Response |
|---|---|---|
| `POST /webhooks` | `{ url, eventTypes[], onChainMarketIds?: string[], categories?: MarketCategory[] }` | `201` subscription **plus `secret` (shown once)**. Max 10 per user (`409 WEBHOOK_LIMIT`); unsafe URLs → `400 INVALID_WEBHOOK_URL`. |
| `GET /webhooks` | | subscriptions (never the secret) |
| `GET /webhooks/:id` | | subscription |
| `PATCH /webhooks/:id` | any of `url`, `eventTypes`, `onChainMarketIds`, `categories`, `status` | `status: "ACTIVE"` re-enables a disabled subscription and resets its failure counters |
| `DELETE /webhooks/:id` | | `{ deleted: true }` |
| `POST /webhooks/:id/rotate-secret` | | `{ id, secret }` (new secret, shown once) |
| `GET /webhooks/:id/deliveries` | `?status=PENDING\|SUCCEEDED\|DEAD&page=&limit=` | delivery log: `attempts`, `lastStatusCode`, `lastError`, `nextAttemptAt`, `eventType` |
| `POST /webhooks/:id/deliveries/:deliveryId/redeliver` | | requeues now with a fresh retry budget |

Empty `onChainMarketIds` / `categories` match everything. A category filter
only matches events whose market is known to the backend.

### Request

```http
POST <your url>
Content-Type: application/json
User-Agent: OracleDesk-Webhooks/1
OracleDesk-Event-Id: trade.executed:0004951600-0000000001
OracleDesk-Event-Type: trade.executed
OracleDesk-Signature: t=1760000000,v1=5257a869e7ec…

{"createdAt":"2026-10-17T12:00:00.000Z","data":{…},"id":"trade.executed:0004951600-0000000001","type":"trade.executed"}
```

Any `2xx` is success. Everything else (including `3xx`; redirects are never
followed) is a failure.

### Verifying signatures

`v1 = hex(HMAC_SHA256(secret, t + "." + rawBody))`. Verify against the **raw
request bytes** (don't re-serialise the JSON) and reject timestamps older
than 5 minutes. During the 24 h after a secret rotation the header carries two
`v1=` values (new and old secret); accept if any matches.

```js
// Node
const crypto = require('crypto');
function verify(rawBody, header, secret, toleranceSec = 300) {
const parts = header.split(',');
const t = Number(parts.find((p) => p.startsWith('t='))?.slice(2));
if (!t || Math.abs(Date.now() / 1000 - t) > toleranceSec) return false;
const expected = crypto.createHmac('sha256', secret).update(`${t}.`).update(rawBody).digest();
return parts.filter((p) => p.startsWith('v1=')).some((p) => {
const sig = Buffer.from(p.slice(3), 'hex');
return sig.length === expected.length && crypto.timingSafeEqual(sig, expected);
});
}
```

```python
# Python
import hashlib, hmac, time
def verify(raw_body: bytes, header: str, secret: str, tolerance=300) -> bool:
parts = header.split(",")
t = next((p[2:] for p in parts if p.startswith("t=")), None)
if not t or abs(time.time() - int(t)) > tolerance:
return False
expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest()
return any(hmac.compare_digest(p[3:], expected) for p in parts if p.startswith("v1="))
```

### Retries, ordering and disabling

- **Backoff:** 1 min, doubling per attempt, capped at 24 h, ±20% jitter. A `Retry-After` header (seconds or HTTP date) is honoured when it is longer. After **15 attempts** a delivery is dead-lettered (`DEAD`) and the next one proceeds.
- **Ordering:** per subscription, deliveries are sent one at a time in event order, so a down receiver only delays its own queue.
- **Timeouts:** 10 s per attempt; at most 64 KB of the response is read.
- **Auto-disable:** after 50 consecutive failed deliveries, or 5 days of continuous failure, the subscription is `DISABLED` with a `disabledReason`. Fix the endpoint, then `PATCH { "status": "ACTIVE" }`.
- **Retention:** delivered and dead deliveries are pruned after 30 days.

### URL rules (SSRF protection)

- `https://` only (`http://` only outside production with `WEBHOOKS_ALLOW_HTTP=true`); no credentials in the URL; ports 443/8443 only in production.
- The host is resolved and rejected if **any** address is private, loopback, link-local, cloud-metadata, CGNAT, multicast or reserved (IPv4, IPv6 and IPv4-mapped IPv6; numeric aliases like `2130706433` or `0x7f.1` are normalised first).
- Checked at create/update **and before every delivery**. The connection is pinned to the validated IP, so DNS rebinding can't redirect it.

### Configuration

| Variable | Default | Notes |
|---|---|---|
| `WEBHOOKS_ENABLED` | `true` | `false` stops enqueueing and the delivery worker |
| `WEBHOOK_SECRET_KEY` | ephemeral | 32 bytes, base64 (`openssl rand -base64 32`). Encrypts signing secrets at rest (AES-256-GCM). **Required in production** when webhooks are enabled. |
| `WEBHOOKS_ALLOW_HTTP` | `false` | Allow `http://` URLs for local receivers. Rejected in production. |

## Frontend call map

Every call in the frontend's `lib/api/*.ts` and where it lands:
Expand Down
81 changes: 81 additions & 0 deletions prisma/migrations/20261018000000_webhooks/migration.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
-- Migration: 20261018000000_webhooks (#128)
-- Outbound webhooks: subscriptions, outbox events and per-subscription deliveries.

-- CreateEnum
CREATE TYPE "WebhookSubscriptionStatus" AS ENUM ('ACTIVE', 'DISABLED');

-- CreateEnum
CREATE TYPE "WebhookDeliveryStatus" AS ENUM ('PENDING', 'SUCCEEDED', 'DEAD');

-- CreateTable
CREATE TABLE "webhook_subscriptions" (
"id" TEXT NOT NULL,
"userId" TEXT NOT NULL,
"url" TEXT NOT NULL,
"eventTypes" TEXT[],
"marketIds" BIGINT[],
"categories" "MarketCategory"[],
"secret" TEXT NOT NULL,
"previousSecret" TEXT,
"secretRotatedAt" TIMESTAMP(3),
"status" "WebhookSubscriptionStatus" NOT NULL DEFAULT 'ACTIVE',
"disabledReason" TEXT,
"consecutiveFailures" INTEGER NOT NULL DEFAULT 0,
"failingSince" TIMESTAMP(3),
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMP(3) NOT NULL,

CONSTRAINT "webhook_subscriptions_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "webhook_events" (
"id" TEXT NOT NULL,
"type" TEXT NOT NULL,
"payload" JSONB NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,

CONSTRAINT "webhook_events_pkey" PRIMARY KEY ("id")
);

-- CreateTable
CREATE TABLE "webhook_deliveries" (
"id" TEXT NOT NULL,
"eventId" TEXT NOT NULL,
"subscriptionId" TEXT NOT NULL,
"status" "WebhookDeliveryStatus" NOT NULL DEFAULT 'PENDING',
"attempts" INTEGER NOT NULL DEFAULT 0,
"nextAttemptAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"lastStatusCode" INTEGER,
"lastError" TEXT,
"deliveredAt" TIMESTAMP(3),
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMP(3) NOT NULL,

CONSTRAINT "webhook_deliveries_pkey" PRIMARY KEY ("id")
);

-- CreateIndex
CREATE INDEX "webhook_subscriptions_status_idx" ON "webhook_subscriptions"("status");

-- CreateIndex
CREATE INDEX "webhook_subscriptions_userId_idx" ON "webhook_subscriptions"("userId");

-- CreateIndex
CREATE INDEX "webhook_deliveries_status_nextAttemptAt_idx" ON "webhook_deliveries"("status", "nextAttemptAt");

-- CreateIndex
CREATE INDEX "webhook_deliveries_subscriptionId_status_idx" ON "webhook_deliveries"("subscriptionId", "status");

-- CreateIndex
CREATE UNIQUE INDEX "webhook_deliveries_eventId_subscriptionId_key" ON "webhook_deliveries"("eventId", "subscriptionId");

-- AddForeignKey
ALTER TABLE "webhook_subscriptions" ADD CONSTRAINT "webhook_subscriptions_userId_fkey" FOREIGN KEY ("userId") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "webhook_deliveries" ADD CONSTRAINT "webhook_deliveries_eventId_fkey" FOREIGN KEY ("eventId") REFERENCES "webhook_events"("id") ON DELETE CASCADE ON UPDATE CASCADE;

-- AddForeignKey
ALTER TABLE "webhook_deliveries" ADD CONSTRAINT "webhook_deliveries_subscriptionId_fkey" FOREIGN KEY ("subscriptionId") REFERENCES "webhook_subscriptions"("id") ON DELETE CASCADE ON UPDATE CASCADE;

76 changes: 76 additions & 0 deletions prisma/schema.prisma
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@ datasource db {

// ─────────────────────────── ENUMS ───────────────────────────

enum WebhookSubscriptionStatus {
ACTIVE
DISABLED
}

enum WebhookDeliveryStatus {
PENDING
SUCCEEDED
DEAD
}

enum MarketStatus {
PENDING
ACTIVE
Expand Down Expand Up @@ -140,6 +151,7 @@ model User {
copyTrades CopyTrade[]
paymentEvents PaymentEvent[]
spendingAllowance SpendingAllowance?
webhookSubscriptions WebhookSubscription[]

@@index([walletAddress])
@@map("users")
Expand Down Expand Up @@ -549,6 +561,70 @@ model ConstraintQuarantine {
@@map("constraint_quarantine")
}

// ─────────────────────────── WEBHOOKS (#128) ───────────────────────────

/// A user-registered outbound webhook endpoint. Empty filter arrays match all.
model WebhookSubscription {
id String @id @default(uuid())
userId String
url String
eventTypes String[]
marketIds BigInt[]
categories MarketCategory[]
/// AES-256-GCM ciphertext (iv.tag.data, base64), keyed by WEBHOOK_SECRET_KEY.
secret String
/// Encrypted previous secret; deliveries carry both signatures for 24 h after rotation.
previousSecret String?
secretRotatedAt DateTime?
status WebhookSubscriptionStatus @default(ACTIVE)
disabledReason String?
consecutiveFailures Int @default(0)
/// Set on the first failure of a failing streak; cleared on success.
failingSince DateTime?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt

user User @relation(fields: [userId], references: [id], onDelete: Cascade)
deliveries WebhookDelivery[]

@@index([status])
@@index([userId])
@@map("webhook_subscriptions")
}

/// One row per domain event. id = "<type>:<chain_events.id>" (dedup key).
model WebhookEvent {
id String @id
type String
payload Json
createdAt DateTime @default(now())

deliveries WebhookDelivery[]

@@map("webhook_events")
}

/// One row per (event, subscription). Delivered at-least-once, in order per subscription.
model WebhookDelivery {
id String @id @default(uuid())
eventId String
subscriptionId String
status WebhookDeliveryStatus @default(PENDING)
attempts Int @default(0)
nextAttemptAt DateTime @default(now())
lastStatusCode Int?
lastError String?
deliveredAt DateTime?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt

event WebhookEvent @relation(fields: [eventId], references: [id], onDelete: Cascade)
subscription WebhookSubscription @relation(fields: [subscriptionId], references: [id], onDelete: Cascade)

@@unique([eventId, subscriptionId])
@@index([status, nextAttemptAt])
@@index([subscriptionId, status])
@@map("webhook_deliveries")
/// Idempotency-Key records for state-changing endpoints (#123). One row per
/// (scope, method, route, key); see src/middlewares/idempotency.middleware.ts.
model IdempotencyKey {
Expand Down
23 changes: 23 additions & 0 deletions src/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,16 @@ const envSchema = z
// ── Copy Trading (Issue #151) ─────────────────────────────────────────
COPY_TRADE_TTL_MINUTES: z.string().default('30').transform(Number),
COPY_TRADE_MAX_PENDING: z.string().default('5').transform(Number),

// ── Outbound webhooks (Issue #128) ────────────────────────────────────
WEBHOOKS_ENABLED: z.enum(['true', 'false']).default('true').transform((v) => v === 'true'),
// 32-byte key (base64) encrypting webhook signing secrets at rest
// (AES-256-GCM). Required in production when webhooks are enabled; an
// ephemeral one is generated otherwise (stored secrets then stop
// decrypting after a restart).
WEBHOOK_SECRET_KEY: optionalString,
// Allow http:// webhook URLs. Development/test only; rejected in production.
WEBHOOKS_ALLOW_HTTP: z.enum(['true', 'false']).default('false').transform((v) => v === 'true'),
})
.superRefine((env, ctx) => {
if (env.JWT_SECRET && env.JWT_SECRET.length < 32) {
Expand All @@ -144,6 +154,15 @@ const envSchema = z
ctx.addIssue({ code: 'custom', path: [key], message: 'must be a Stellar secret seed (S…)' });
}
}
if (env.WEBHOOK_SECRET_KEY && Buffer.from(env.WEBHOOK_SECRET_KEY, 'base64').length !== 32) {
ctx.addIssue({ code: 'custom', path: ['WEBHOOK_SECRET_KEY'], message: 'must be 32 bytes, base64-encoded' });
}
if (env.NODE_ENV === 'production' && env.WEBHOOKS_ENABLED && !env.WEBHOOK_SECRET_KEY) {
ctx.addIssue({ code: 'custom', path: ['WEBHOOK_SECRET_KEY'], message: 'is required in production when WEBHOOKS_ENABLED=true' });
}
if (env.NODE_ENV === 'production' && env.WEBHOOKS_ALLOW_HTTP) {
ctx.addIssue({ code: 'custom', path: ['WEBHOOKS_ALLOW_HTTP'], message: 'must not be true in production' });
}
if (env.CHAIN_EXECUTION_MODE === 'live') {
if (!env.AGENT_SECRET_KEY) {
ctx.addIssue({ code: 'custom', path: ['AGENT_SECRET_KEY'], message: 'is required when CHAIN_EXECUTION_MODE=live' });
Expand Down Expand Up @@ -182,6 +201,10 @@ export function loadConfig(env: NodeJS.ProcessEnv): Config {
data.AUTH_SIGNING_SECRET = Keypair.random().secret();
ephemeralSecrets.push('AUTH_SIGNING_SECRET');
}
if (!data.WEBHOOK_SECRET_KEY) {
data.WEBHOOK_SECRET_KEY = crypto.randomBytes(32).toString('base64');
if (data.WEBHOOKS_ENABLED) ephemeralSecrets.push('WEBHOOK_SECRET_KEY');
}
if (!data.AUTH_WEB_AUTH_DOMAIN) data.AUTH_WEB_AUTH_DOMAIN = data.AUTH_HOME_DOMAIN;
return { ...data, ephemeralSecrets };
}
Expand Down
Loading