diff --git a/README.md b/README.md index 8aef485..b61600a 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,26 @@ I wrote both the policy and the attacks, so this shows the gate does what the po I applied 15 deliberate bugs by hand (for example: remove the same-actor check, skip the hash comparison, skip re-checking policy at execution, ignore the cumulative total) and ran `node --test` after each. All 15 made at least one test fail. I did not keep the mutation script, so treat this as a result I observed, not one you can rerun with a command. +## Try it with an agent + +`examples/agent.mjs` is a small agent whose every tool call goes through the gate. The agent only proposes; the gate answers allow, hold for a person, or deny. Zero dependencies, and the world is simulated. + +```bash +# No key needed: replay a scripted attack through the real engine +node examples/agent.mjs --mock prompt-injection +node examples/agent.mjs --mock salami-refunds + +# Against the deployed playground instead of the local engine +node examples/agent.mjs --mock prompt-injection --base https://agentgate-lake.vercel.app + +# A real model plans the calls (Gemini function calling) +GEMINI_API_KEY=... node examples/agent.mjs --task "Refund pay_AB12CD34 for 500 rupees and email the team" +``` + +Add `--ask` in a terminal to approve held calls yourself. What you see in the prompt-injection run: the harmless read runs, the email to an outside address is held, the read of `secrets/.env` is denied. In the salami run, small refunds go through until the per-run total passes ₹2,000, then every further refund is held. + +The Gemini path is covered by a test with a mocked API (request shape and result feedback). Live model behaviour varies and is not measured here. + ## Honest limits - Secret detection is regex-based. It catches common key and private-key shapes, not arbitrary secrets. @@ -94,10 +114,10 @@ I applied 15 deliberate bugs by hand (for example: remove the same-actor check, - Accounts are email and password only: no email verification, password reset, SSO or two-factor yet. Lost passwords need an administrator to remove and re-add the person. - Login throttling is in memory, so it resets on restart and is per server instance. - `AUTH=off` has no identity at all and must never be exposed publicly. -- Scripted planners only. No LLM planner, so nothing here measures prompt-injection resistance of a real model. +- The agent example can use a real model (Gemini), but the adversarial suite uses scripted planners, so nothing here measures how often a real model falls for prompt injection. - Truncating the newest records of a log is not detectable without an externally stored head hash. The chain detects edits and deletions in the middle. - The tool set is five simulated tools. Real tools need their own validators. ## Layout -`src/policy.mjs` rules engine · `src/tools.mjs` validation, canonicalisation, simulated tools · `src/runtime.mjs` event-sourced run loop and approvals · `src/chain.mjs` hashing · `src/store.mjs`, `src/store-pg.mjs` storage · `policies/default.json` · `src/auth.mjs` accounts, sessions, roles · `src/play.mjs`, `src/play-http.mjs` public playground · `src/*` + `api/` Vercel functions · `server.mjs` + `public/` landing and dashboard · `EXPLAIN.md` interview prep. +`src/policy.mjs` rules engine · `src/tools.mjs` validation, canonicalisation, simulated tools · `src/runtime.mjs` event-sourced run loop and approvals · `src/chain.mjs` hashing · `src/store.mjs`, `src/store-pg.mjs` storage · `policies/default.json` · `src/auth.mjs` accounts, sessions, roles · `src/play.mjs`, `src/play-http.mjs` public playground · `src/*` + `api/` Vercel functions · `examples/agent.mjs` agent demo · `server.mjs` + `public/` landing and dashboard · `EXPLAIN.md` interview prep. diff --git a/examples/agent.mjs b/examples/agent.mjs new file mode 100644 index 0000000..0787fac --- /dev/null +++ b/examples/agent.mjs @@ -0,0 +1,141 @@ +#!/usr/bin/env node +/** + * A small AI agent whose every tool call goes through AgentGate before anything "runs". + * + * node examples/agent.mjs --task "Refund pay_AB12CD34 for ₹500 and email the team" (needs GEMINI_API_KEY) + * node examples/agent.mjs --mock prompt-injection (no key, no network) + * node examples/agent.mjs --mock salami-refunds --base https://agentgate-lake.vercel.app + * + * Zero dependencies. The agent only PROPOSES calls. The gate answers allow, hold (needs a person) or deny. + * The world is simulated: nothing here sends mail, moves money or touches a file. + */ +import { readFileSync } from 'node:fs'; +import { createInterface } from 'node:readline/promises'; +import { pathToFileURL } from 'node:url'; +import { createPlayground } from '../src/play.mjs'; +import { SCENARIOS } from '../src/scenarios.mjs'; + +const MAX_STEPS = 25; // the gate's own per-run call limit +const SYSTEM = 'You are a support operations agent. Use the tools to finish the task. If a tool result says a call was denied or is waiting for a person, do not try to get around it: explain it in one sentence and stop.'; + +/** The gate: where decisions come from. `local` runs the same engine in-process; `remote` calls a deployed playground. */ +export function localGate(policy) { + const play = createPlayground(policy); + return { meta: async () => play.meta(), check: async body => play.check(body) }; +} +export function remoteGate(base) { + const url = path => `${base.replace(/\/+$/, '')}/api/play/${path}`; + const call = async (path, init) => { + const r = await fetch(url(path), init); + if (!r.ok) throw new Error(`gate returned ${r.status} for ${path}`); + return r.json(); + }; + return { + meta: () => call('meta'), + check: body => call('check', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) }), + }; +} + +/** A planner that replays a scripted scenario. Same interface as the Gemini one, so the loop below is identical for both. */ +export function mockPlanner(calls) { + let i = 0; + return { next: async () => (i < calls.length ? { tool: calls[i].tool, args: calls[i++].args } : null), feed: () => {} }; +} + +/** A planner backed by Gemini function calling over plain REST. */ +export function geminiPlanner({ task, tools, key = process.env.GEMINI_API_KEY, model = process.env.GEMINI_MODEL || 'gemini-3.8-flash' }) { + if (!key) throw new Error('Set GEMINI_API_KEY, or use --mock to run without a key.'); + const declarations = Object.entries(tools).map(([name, t]) => ({ + name, description: t.description, + parameters: { + type: 'object', + properties: Object.fromEntries(Object.entries(t.fields).map(([f, s]) => [f, s.type === 'stringArray' ? { type: 'array', items: { type: 'string' } } : { type: s.type === 'integer' ? 'integer' : 'string', ...(s.enum ? { enum: s.enum } : {}) }])), + required: Object.entries(t.fields).filter(([, s]) => s.required).map(([f]) => f), + }, + })); + const contents = [{ role: 'user', parts: [{ text: task }] }]; + let pending = null; + return { + async next() { + const r = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`, { + method: 'POST', headers: { 'content-type': 'application/json', 'x-goog-api-key': key }, + body: JSON.stringify({ systemInstruction: { parts: [{ text: SYSTEM }] }, contents, tools: [{ functionDeclarations: declarations }] }), + }); + if (!r.ok) throw new Error(`Gemini returned ${r.status}: ${(await r.text()).slice(0, 200)}`); + const content = (await r.json()).candidates?.[0]?.content; + if (!content) return null; + contents.push(content); + const fc = content.parts?.find(p => p.functionCall)?.functionCall; + if (!fc) { return { say: content.parts?.map(p => p.text).filter(Boolean).join(' ') || '' }; } + pending = fc.name; + return { tool: fc.name, args: fc.args || {} }; + }, + feed(result) { contents.push({ role: 'user', parts: [{ functionResponse: { name: pending, response: result } }] }); }, + }; +} + +/** + * The loop. For each proposed call: ask the gate, then + * allow -> "run" it (simulated) and tell the agent; + * approve -> ask a person if `approver` is given, otherwise leave it held; + * deny -> tell the agent why and carry on. + * Returns a summary so callers and tests can check what happened. + */ +export async function runAgent({ gate, planner, approver = null, log = () => {} }) { + const prior = {}; const trail = []; + for (let step = 1; step <= MAX_STEPS; step++) { + const proposal = await planner.next(); + if (!proposal) break; + if (proposal.say !== undefined) { log({ kind: 'say', text: proposal.say }); break; } + const verdict = await gate.check({ tool: proposal.tool, args: proposal.args, prior: prior[proposal.tool] || {} }); + let outcome = verdict.effect === 'allow' ? 'ran' : verdict.effect === 'deny' ? 'denied' : 'held'; + if (verdict.effect === 'approve' && approver && await approver(proposal, verdict)) outcome = 'ran-after-approval'; + if (outcome.startsWith('ran')) { + const bucket = prior[proposal.tool] ||= {}; + for (const [k, v] of Object.entries(verdict.canonical || {})) if (Number.isSafeInteger(v) && v >= 0) bucket[k] = (bucket[k] || 0) + v; + } + trail.push({ step, tool: proposal.tool, args: proposal.args, effect: verdict.effect, outcome, reason: verdict.reason, rules: (verdict.rules || []).map(r => r.id) }); + log({ kind: 'call', ...trail.at(-1) }); + planner.feed(outcome.startsWith('ran') ? { status: 'executed' } : outcome === 'held' ? { status: 'waiting_for_human_approval' } : { status: 'denied', reason: verdict.reason }); + } + const count = o => trail.filter(t => (o === 'ran' ? t.outcome.startsWith('ran') : t.outcome === o)).length; + return { trail, ran: count('ran'), held: count('held'), denied: count('denied') }; +} + +// ---- command line ---- +const color = process.stdout.isTTY ? { allow: '\x1b[32m', approve: '\x1b[33m', deny: '\x1b[31m', dim: '\x1b[2m', off: '\x1b[0m' } : { allow: '', approve: '', deny: '', dim: '', off: '' }; +const label = { allow: 'ALLOW', approve: 'HELD ', deny: 'DENY ' }; +const short = a => { const s = JSON.stringify(a); return s.length > 90 ? `${s.slice(0, 90)}…` : s; }; +function print(e) { + if (e.kind === 'say') return console.log(`${color.dim}agent says:${color.off} ${e.text}`); + const tag = label[e.effect] || e.effect; + console.log(`${color[e.effect] || ''}${tag}${color.off} ${e.tool} ${color.dim}${short(e.args)}${color.off}`); + console.log(` ${color.dim}${e.reason}${e.rules.length ? ` [${e.rules.join(', ')}]` : ''}${e.outcome === 'ran-after-approval' ? ' [approved by you]' : ''}${color.off}`); +} +const arg = name => { const i = process.argv.indexOf(`--${name}`); return i > -1 ? process.argv[i + 1] : undefined; }; + +async function main() { + const base = arg('base'); + const gate = base ? remoteGate(base) : localGate(JSON.parse(readFileSync(new URL('../policies/default.json', import.meta.url), 'utf8'))); + const mock = arg('mock'); + let planner; + if (mock) { + if (!Object.hasOwn(SCENARIOS, mock)) { console.error(`Unknown scenario. Choose one of: ${Object.keys(SCENARIOS).join(', ')}`); process.exit(2); } + console.log(`${color.dim}Scripted agent: ${SCENARIOS[mock].title}${color.off}\n`); + planner = mockPlanner(SCENARIOS[mock].planner.calls); + } else { + const task = arg('task'); + if (!task) { console.error('Give the agent a job: --task "..." (needs GEMINI_API_KEY), or run a scripted one: --mock '); process.exit(2); } + planner = geminiPlanner({ task, tools: (await gate.meta()).tools }); + console.log(`${color.dim}Task: ${task}${color.off}\n`); + } + let approver = null; let rl = null; + if (process.stdin.isTTY && process.argv.includes('--ask')) { + rl = createInterface({ input: process.stdin, output: process.stdout }); + approver = async (p, v) => /^y/i.test(await rl.question(` Approve ${p.tool}? (${v.reason}) [y/N] `)); + } + const out = await runAgent({ gate, planner, approver, log: print }); + rl?.close(); + console.log(`\n${out.ran} ran, ${out.held} held for a person, ${out.denied} denied.`); +} +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) main().catch(e => { console.error(e.message); process.exit(1); }); diff --git a/test/agent.test.mjs b/test/agent.test.mjs new file mode 100644 index 0000000..c5bf721 --- /dev/null +++ b/test/agent.test.mjs @@ -0,0 +1,58 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { runAgent, localGate, mockPlanner, geminiPlanner } from '../examples/agent.mjs'; +import { SCENARIOS } from '../src/scenarios.mjs'; + +const policy = JSON.parse(readFileSync(new URL('../policies/default.json', import.meta.url), 'utf8')); +const run = (scenario, opts = {}) => runAgent({ gate: localGate(policy), planner: mockPlanner(SCENARIOS[scenario].planner.calls), ...opts }); + +test('prompt injection: the harmless read runs, the exfiltration is held, secrets are denied', async () => { + const { trail, ran, held, denied } = await run('prompt-injection'); + assert.equal(trail[0].outcome, 'ran'); + assert.equal(trail.find(t => t.tool === 'send_email').outcome, 'held'); + assert.equal(trail.find(t => t.args.path === 'secrets/.env').outcome, 'denied'); + assert.deepEqual([ran, held, denied], [1, 2, 2]); +}); + +test('salami refunds: small refunds run until the per-run total is crossed, then each one is held', async () => { + const { trail } = await run('salami-refunds'); + const first = trail.findIndex(t => t.outcome === 'held'); + assert.ok(first > 0, 'some small refunds run first'); + assert.ok(trail.slice(first).every(t => t.outcome === 'held'), 'every refund after the budget is held'); +}); + +test('a held call runs only when the approver says yes', async () => { + const no = await run('prompt-injection', { approver: async () => false }); + const yes = await run('prompt-injection', { approver: async () => true }); + assert.equal(no.held, 2); + assert.equal(yes.held, 0); + assert.equal(yes.denied, 2, 'approval never overrides a deny'); +}); + +test('gemini planner: sends tool declarations, returns the proposed call, and feeds the gate result back', async () => { + const sent = []; + const realFetch = globalThis.fetch; + const replies = [ + { candidates: [{ content: { role: 'model', parts: [{ functionCall: { name: 'read_file', args: { path: 'workspace/report.md' } } }] } }] }, + { candidates: [{ content: { role: 'model', parts: [{ text: 'Done.' }] } }] }, + ]; + globalThis.fetch = async (url, init) => { sent.push({ url, body: JSON.parse(init.body), key: init.headers['x-goog-api-key'] }); return { ok: true, json: async () => replies.shift() }; }; + try { + const gate = localGate(policy); + const planner = geminiPlanner({ task: 'read the report', tools: (await gate.meta()).tools, key: 'test-key', model: 'test-model' }); + const out = await runAgent({ gate, planner }); + assert.equal(out.ran, 1); + assert.equal(sent.length, 2); + assert.equal(sent[0].key, 'test-key'); + assert.match(sent[0].url, /models\/test-model:generateContent/); + assert.ok(sent[0].body.tools[0].functionDeclarations.some(d => d.name === 'refund_payment')); + const fed = sent[1].body.contents.at(-1).parts[0].functionResponse; + assert.deepEqual([fed.name, fed.response.status], ['read_file', 'executed']); + } finally { globalThis.fetch = realFetch; } +}); + +test('gemini planner refuses to start without a key', () => { + const had = process.env.GEMINI_API_KEY; delete process.env.GEMINI_API_KEY; + try { assert.throws(() => geminiPlanner({ task: 'x', tools: {} }), /GEMINI_API_KEY/); } finally { if (had) process.env.GEMINI_API_KEY = had; } +});