Skip to content

feat: x402 e2e test, structured LLM outputs, prompt-injection guard and shared LLM budgets (#64 #65 #70 #71) - #175

Merged
Levi-Ojukwu merged 3 commits into
OracleDesk:developfrom
TheCodingChef-eth:feat/x402-e2e-structured-llm-injection-budgets
Oct 18, 2026
Merged

Levi-Ojukwu merged 3 commits into
OracleDesk:developfrom
TheCodingChef-eth:feat/x402-e2e-structured-llm-injection-budgets

Conversation

@TheCodingChef-eth

Copy link
Copy Markdown
Contributor

Closes #64, closes #65, closes #70, closes #71

Base repair (first commit)

develop does not compile: 3ce175e (PR #172) was committed with a bad conflict
resolution (schema syntax error, missing Prisma models and config keys,
completeLLM imported but not exported, syntax errors in the cron files). The
first commit redoes the merge of 7a8d096 and ced038e keeping both sides, plus
the few type fixes needed to compile. Please review it on its own.

  • closes Run the contracts x402 trace API in an end-to-end test against backend-published traces #64 x402 end-to-end

  • tests/x402-e2e.test.ts runs the submodule's real createTraceApi against
    traces from generateReasoningTrace + pinAndPublishDocument: 402 challenge,
    paid 200 (content equals the canonical payload), tampered 502, unknown 404,
    and no content when settlement fails.

  • In-memory IPFS, in-process stub facilitator, and a fetch guard that fails the
    test on any non-loopback request.

  • tests/canonical-parity.test.ts compares canonicalJson with the contracts
    canonicalize over an edge-case corpus; pinJson now rejects non-plain
    objects (IPFS_NON_PLAIN_DOCUMENT) because Date diverges (ISO vs {}).

  • npm run test:x402:prepare and npm run test:e2e:x402; CI installs and
    caches the x402 deps. Skips locally with guidance, fails when CI=true.

  • closes Use provider-native structured outputs generated from the zod proposal schemas #65 Structured outputs

  • src/lib/llm-schema.ts is the single schema registry (zod, Anthropic tool
    schema, Gemini responseSchema, postValidate).

  • callLLMStructured(): Anthropic forced tool use, Gemini responseSchema, one
    repair round per provider, LLM_STRUCTURED_OUTPUT_FAILED. Fence stripping is
    gone from llm.ts, gemini.ts and market.service.ts.

  • generateMarketQuestion / generateHedgeConditions migrated; seams
    setAnthropicClient / setGeminiModelFactory; fixtures in tests/fixtures/llm/.

  • closes Defend market generation against prompt injection in ingested signals #70 Prompt injection

  • llm-guard.service.ts: sanitiser (NFKC, control/zero-width/bidi, caps, URLs
    moved to url), quarantine, nonce-delimited untrusted blocks, checkProposal
    (grounding, probability, expiry, oracle allowlist, content policy).

  • All fetchers sanitise before caching; GDELT URL is stored in url.

  • Rejections/quarantines go to agent_logs with reason codes and hashes only.

  • Corpus: 28 attack + 18 benign cases in tests/fixtures/injection/.

  • closes Enforce LLM token budgets and a shared circuit breaker across instances #71 Budgets and shared breaker

  • llm-budget.ts + atomic Lua: per-provider token bucket, circuit breaker with
    half-open probe, daily token/cost budgets with an admin reserve, reservations
    released on error/missing usage, crashed leases reclaimed.

  • Usage read from Anthropic usage and Gemini usageMetadata; cron stops at
    budget minus reserve, admin runs to the full budget; LLM_BUDGET_EXCEEDED
    logged once per day. LLM_REDIS_FAIL_MODE=closed|open.

  • Same scenario table runs against the JS reference store and real Redis.

Notes

  • New env vars are in .env.example and docs/architecture.md.
  • npm test has 15 failures that also fail on the repaired base (request-id
    tests, copy-trade/reconcile, some resilience tests, DB-dependent tests).
  • LLM fixtures are hand-built in the real API shapes, not live captures.

TheCodingChef-eth and others added 3 commits October 18, 2026 06:07
The conflict resolution in 3ce175e dropped one side of several files: the
LLM router wiring (completeLLM was imported but not exported), Prisma models
for IPFS content / transparency reports / treasury ops, config keys, cron
handlers (syntax errors) and trace-publication/audit schema. Re-do the merge
of 7a8d096 and ced038e, keep both sides, and fix the few type errors that
remained so the tree compiles again.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ned traces (OracleDesk#64)

- tests/x402-e2e.test.ts: real createTraceApi from the submodule, traces from
  generateReasoningTrace + pinAndPublishDocument, in-memory IPFS, stub
  facilitator, hermetic fetch guard (402, paid 200, tampered 502, unknown 404,
  settle failure).
- tests/canonical-parity.test.ts: canonicalJson vs contracts canonicalize.
- pinJson rejects non-plain objects (IPFS_NON_PLAIN_DOCUMENT).
- npm scripts test:x402:prepare / test:e2e:x402, CI install + cache + step.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…n budgets (OracleDesk#65, OracleDesk#70, OracleDesk#71)

OracleDesk#65 Structured outputs
- src/lib/llm-schema.ts: zod schemas as the single source; derived Anthropic
  tool input_schema and Gemini responseSchema (toGeminiSchema), postValidate.
- callLLMStructured(): forced tool use / responseSchema, one repair round per
  provider, LLM_STRUCTURED_OUTPUT_FAILED; no fence stripping anywhere.
- setAnthropicClient / setGeminiModelFactory seams, recorded response fixtures.

OracleDesk#70 Prompt injection
- llm-guard.service.ts: sanitiser, quarantine, nonce-delimited untrusted
  blocks, deterministic checkProposal (PROPOSAL_REJECTED_BY_GUARD).
- Ingestion sanitises before caching; GDELT URLs move to url.
- 28 attack + 18 benign corpus cases replayed through generateMarketQuestion.

OracleDesk#71 Budgets and shared guards
- llm-budget.ts + atomic Lua (token bucket, circuit breaker with half-open
  probe, daily token/cost budgets, admin reserve) in Redis; fail-open/closed.
- Providers take a lease per attempt; usage read from Anthropic usage and
  Gemini usageMetadata; budget exhaustion logged once per day.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@drips-wave

drips-wave Bot commented Oct 18, 2026

Copy link
Copy Markdown

@TheCodingChef-eth Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Levi-Ojukwu
Levi-Ojukwu merged commit 7013adb into OracleDesk:develop Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants