diff --git a/.agents/skills/dsh-dev/SKILL.md b/.agents/skills/dsh-dev/SKILL.md index af1845829..d846d6a8e 100644 --- a/.agents/skills/dsh-dev/SKILL.md +++ b/.agents/skills/dsh-dev/SKILL.md @@ -236,6 +236,8 @@ A saved AX model id may disappear from the pinned native catalog. Preserve the l ## Pitfall log +DSH 0.2.2-rc.1 can report the gateway wording `Input too long: model limit N tokens, requested M tokens` as `INVALID_REQUEST`, bypassing native overflow recovery. Tobs handles that precise missed case through its owned `agent/request-error` listener and the public compaction Service. In a real Preset, resolve the isolated Provider with `agentPresets.serviceFor(agent, 'compaction')`; a flat Context test with `agent.ctx.get('compaction')` does not prove this lookup. Verify the before/after Host path with `scripts/e2e-context-overflow.mjs`, await the native Session Store flush checkpoint and verify the exact replacement through a read-only Session persistence handle before retrying, retain the replaced request context and committed reply evidence, and keep DSH dependency bytes unchanged (#4253). + ### Windows background command consoles (#3565) On a detached Windows DSH Host, a console executable launched by `execFile` without diff --git a/.agents/skills/verify-deepseekbot/features/README.md b/.agents/skills/verify-deepseekbot/features/README.md index ce5225110..9c9993497 100644 --- a/.agents/skills/verify-deepseekbot/features/README.md +++ b/.agents/skills/verify-deepseekbot/features/README.md @@ -44,7 +44,7 @@ One file per user-facing feature. Each file has four sections: `Sub-features`, ` | Bot-to-Bot Group mentions and wake | yes | `scripts/e2e-llm-scenarios.mjs` (nightly) | [bot-group-mention.md](bot-group-mention.md) | | Group send review | yes for A/B; no for direct tools | `scripts/e2e-group-send-review.mjs`, `scripts/e2e-group-peer-review.mjs`, `scripts/e2e-group-busy-review.mjs`, `scripts/e2e-group-responsibility.mjs` | [group-send-review.md](group-send-review.md) | | Group invitation candidates | no | `verify-app` Group Members path | [group-invitations.md](group-invitations.md) | -| Conversation timeline: static ordinary avatars, memory display, new-messages divider, return to latest, bidirectional pages, kept place, folding, image preview, reactions, search, member mention, iOS/Android raw-source selection, GenUI documents and reports | for replies; GenUI model optional | `scripts/e2e-timeline-navigation.mjs`, `scripts/e2e-genui-reports.mjs`, `scripts/e2e-genui-documents.mjs` | [conversation-timeline.md](conversation-timeline.md) | +| Conversation timeline: static ordinary avatars, memory display, new-messages divider, return to latest, bidirectional pages, kept place, folding, image preview, reactions, search, member mention, iOS/Android raw-source selection, GenUI documents and reports | for replies; GenUI model optional | `scripts/e2e-context-overflow.mjs`, `scripts/e2e-timeline-navigation.mjs`, `scripts/e2e-genui-reports.mjs`, `scripts/e2e-genui-documents.mjs` | [conversation-timeline.md](conversation-timeline.md) | | PersonaBot creation and restart persistence | for Bot-created Bots | `scripts/e2e-personabot-create.mjs` (CI) | [personabot-create.md](personabot-create.md) | | Stop a working Bot (Web Client) | yes | `scripts/e2e-web-stop.mjs` | [bot-stop.md](bot-stop.md) | | Phone pairing, QR images, app variants and LAN/Tailscale guidance | no | `scripts/e2e-mobile-pairing-web.mjs`, `scripts/e2e-device-pairing.mjs`, `scripts/e2e-mobile-sim.mjs`, `scripts/e2e-mobile-pair-android.mjs`, `scripts/e2e-phone-recovery.mjs` | [phone-pairing.md](phone-pairing.md) | diff --git a/.agents/skills/verify-deepseekbot/features/conversation-timeline.md b/.agents/skills/verify-deepseekbot/features/conversation-timeline.md index b6ec4835a..abbbfafe3 100644 --- a/.agents/skills/verify-deepseekbot/features/conversation-timeline.md +++ b/.agents/skills/verify-deepseekbot/features/conversation-timeline.md @@ -31,6 +31,8 @@ For the Web document's large-text journey, use Harness Settings → General → ## How to get to it (user POV) +- **Automatic gateway overflow recovery (#4253):** launch an isolated Host with `verify-app launch --no-ax-model` and `DSH_OVERFLOW_FIXTURE_KEY` set to a dummy local value; run `scripts/e2e-context-overflow.mjs` with `BH_E2E_INSTANCE` pointing at that instance. The local HTTP fixture seeds a Bot DM, returns the incident's token-limit rejection, then serves the native summary and reply. Verify `OVERFLOW_RECOVERED` commits, one summary request occurs, and the committed reply stays in the original Session. At the first retry, the fixture reads the persisted Session summary and replacement, records their sequence numbers, and verifies the request contains the summary without the old seed. Set `BH_E2E_EXPECT=blocked` against the base build for the before control. No real provider key or IM destination is used; stop this task's Host afterward. This is a CLI/Host journey, not visual or live-provider qualification. + - Open Bot settings → Messaging to enable Auto-collapse long messages. Long messages start expanded; when folding is enabled, Show all reveals the full body. - In a Bot DM with Memory updates from another conversation, open its Channel menu → Show what this Bot did elsewhere → Memory display: Compact / Detailed. The same choice is under Channel settings → Show what this Bot did elsewhere → Memory display. Hide Memory updates to disable the choice; show them again to restore it. diff --git a/apps/docs/test/docs-reference.test.ts b/apps/docs/test/docs-reference.test.ts index 15765f460..00d421a42 100644 --- a/apps/docs/test/docs-reference.test.ts +++ b/apps/docs/test/docs-reference.test.ts @@ -104,6 +104,7 @@ describe("generated developer reference", () => { { name: "agent/inbox/discarded", direction: "consumes", operation: "on" }, { name: "agent/pre-step", direction: "consumes", operation: "on" }, { name: "agent/request-error", direction: "consumes", operation: "on" }, + { name: "agent/request-error", direction: "consumes", operation: "on" }, { name: "agent/status", direction: "consumes", operation: "on" }, { name: "agent/status", direction: "consumes", operation: "on" }, { name: "approval/request", direction: "consumes", operation: "on" }, diff --git a/changes/4253-context-overflow-recovery.md b/changes/4253-context-overflow-recovery.md new file mode 100644 index 000000000..c2361be50 --- /dev/null +++ b/changes/4253-context-overflow-recovery.md @@ -0,0 +1,11 @@ +--- +section: Fixed +--- + +## en + +- Bots now recover from gateway token-limit errors that DSH reports as ordinary invalid requests: Tobs requests native context compaction and retries once after committed progress, instead of repeatedly failing until paused ([#4253](https://github.com/BotHarness/DeepSeekBot/issues/4253)). + +## zh + +- 修复网关上下文超限错误被 DSH 归类为普通无效请求后 Bot 连续失败直至暂停的问题:Tobs 调用原生上下文压缩,在确认变更已提交后重试一次([#4253](https://github.com/BotHarness/DeepSeekBot/issues/4253))。 diff --git a/docs/architecture/botharness-architecture.md b/docs/architecture/botharness-architecture.md index 8551265cc..f68522607 100644 --- a/docs/architecture/botharness-architecture.md +++ b/docs/architecture/botharness-architecture.md @@ -328,6 +328,8 @@ application-defined Memory Service 使用 `Consumer → Service Definition → P Idle Compaction(#2310):当前 Orchestrator Session 没有进行中的轮次、也没有待处理的 Bot Inbox Admission 时,Host 按最后一次模型回答的时间,在该线路提示缓存寿命的一半(最少 15 分钟)经 DSH 命令注册表为该 Agent 执行 `/compact`(`compactNow`;compaction 服务隔离在每个 Agent 的 preset 组内,Host 无法直接取得),只压缩超过该线路 DSH 默认自动压缩阈值一半的会话;摘要格式与 `compaction/end` 刷新 Soul/Core Memory 的行为不变,期间到达的 Human 消息由 DSH 排队到压缩结束。缓存寿命依次取 Host 设置中的每线路值、模型目录、观测(请求前空闲间隔与该请求缓存命中率)与 DeepSeek 的 2 小时默认值,不足 15 分钟或未知则跳过。Assignment 与审批复核会话不参与,聊天中不显示;每次执行写入 `usage_idle_compactions` 并在 Bot 活动中返回。开关与每线路寿命是 Profile 级 Host 设置,Web 设置页与 `deepseekbot idle-compaction*` 共用同一 bridge 方法。 +Tobs also handles the explicit gateway wording `Input too long: model limit N tokens, requested M tokens` when DSH reports it as `INVALID_REQUEST` (#4253). Its application-defined `agent/request-error` listener applies only to Tobs-owned Sessions, delegates existing recovery first, and resolves the native compaction Provider through `agentPresets.serviceFor(agent, 'compaction')` on isolated Presets. It calls `compactIfNeeded(..., 'context-overflow', signal)` without modifying the Provider or DSH failure. A retry requires Session surface replacement and a successful native Session Store flush checkpoint plus an exact readback of that Session replacement through a read-only persistence handle and is limited to one recovery attempt per Agent turn/step. Cancellation, unavailable services and failures without replacement retain the original request failure; bounded developer records contain identifiers, phase, duration and outcome, never the provider error text. + Message action presentation is owned by the Web `ds/action-button` and native `primitives/button` adapters; standalone failed reads use their platform ReadFailure module. The shared application-defined `session-maintenance` controller scopes in-flight state and outcomes to the Host client and Bot DM, so error cards and usage panels cannot submit competing maintenance operations. These modules do not own approvals, read retries or durable Session state. Orchestrator context recovery reuses the application-defined Bot Runtime queue and canonical Channel failure reference. Web and mobile failure cards maintain the current Bot DM Orchestrator Session by compacting or replacing it, without retrying any message. The CLI recovery-and-retry path can still re-admit only the original safely retryable Human DM. The owner refuses busy Bots, stale Session references and requests that already began external effects; arrivals during maintenance queue normally. Explicit replacement archives the old Session and preserves Channel history, Memory and independent Assignments (ADR-0024). The Web `/new` command performs replacement without retry. Model-declared context capacity comes from DSH Model Info; native automatic compaction remains enabled by default. Unknown capacity uses an explicitly estimated 200,000-token fallback with output reservation and safety headroom, compacting balanced native Session regions before the next model step. The read-only `botContext` query and `bot-context` CLI expose current Session token estimates and capacity; the existing Web composer chip displays them while retaining cumulative usage in the details popover. That popover exposes explicit compact/new maintenance only in the Bot Human DM. Failed-turn feedback freezes a redacted, bounded projection of the canonical failed turn input and tool call/result records in the existing feedback preview token, then passes it to the existing uploader only on explicit submit. Missing turn correlation refuses; full diagnostic ZIPs remain download-only. diff --git a/packages/core/package.json b/packages/core/package.json index d7cb0db5a..30a1597ee 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -40,6 +40,7 @@ "devDependencies": { "@deepseek-ai/cordis": "4.1.0", "@deepseek-ai/dsh-agent": "0.2.2-rc.1", + "@deepseek-ai/dsh-agent-preset-registry": "0.2.2-rc.1", "@deepseek-ai/dsh-api-gateway": "0.2.2-rc.1", "@deepseek-ai/dsh-api-remote-access": "0.2.2-rc.1", "@deepseek-ai/dsh-attachment": "0.2.2-rc.1", @@ -56,6 +57,7 @@ "@deepseek-ai/dsh-schedule": "0.2.2-rc.1", "@deepseek-ai/dsh-scope": "0.2.2-rc.1", "@deepseek-ai/dsh-session": "0.2.2-rc.1", + "@deepseek-ai/dsh-session-persistence": "0.2.2-rc.1", "@deepseek-ai/dsh-session-projection": "0.2.2-rc.1", "@deepseek-ai/dsh-settings": "0.2.2-rc.1", "@deepseek-ai/dsh-shell": "0.2.2-rc.1", diff --git a/packages/core/src/plugin.ts b/packages/core/src/plugin.ts index 8aec274fa..474ced63a 100644 --- a/packages/core/src/plugin.ts +++ b/packages/core/src/plugin.ts @@ -22,6 +22,7 @@ import { type PricedSurface, } from './runtime/context-window.js'; import { createBotRequestRetry } from './runtime/bot-request-retry.js'; +import { createContextOverflowRecovery } from './runtime/context-overflow-recovery.js'; import { messageText, systemPromptFromLog } from './runtime/prompt-inspection.js'; import { createWindowZoomSetup, WINDOW_ZOOM_SETUP_PATH } from './window-zoom/setup.js'; @@ -2451,6 +2452,14 @@ export function apply(ctx: Context, config: BotHarnessConfig): void { }), { global: true }, ); + const contextOverflowRecovery = createContextOverflowRecovery({ + isBotSession: (sessionId) => core.ownership.resolve(sessionId) !== undefined, + log: (event) => + ctx.logger[event['outcome'] === 'declined' ? 'warn' : 'info']( + `context-overflow-recovery ${JSON.stringify(event)}`, + ), + }); + ctx.on('agent/request-error', contextOverflowRecovery, { global: true, prepend: true }); const approvalTemplates = createApprovalTemplateLibrary( join(dshHome, 'botharness', 'approval-templates'), ); diff --git a/packages/core/src/runtime/context-overflow-recovery.ts b/packages/core/src/runtime/context-overflow-recovery.ts new file mode 100644 index 000000000..a784d1fd2 --- /dev/null +++ b/packages/core/src/runtime/context-overflow-recovery.ts @@ -0,0 +1,120 @@ +import type { Agent, RequestErrorAction } from '@deepseek-ai/dsh-agent'; +import type {} from '@deepseek-ai/dsh-agent-preset-registry'; +import type { LlmFailure } from '@deepseek-ai/dsh-llm'; +import type {} from '@deepseek-ai/dsh-session-persistence'; +import { isDeepStrictEqual } from 'node:util'; + +export interface ContextOverflowRequest { + agent: Pick; + turn: number; + step: number; + provider: string; + failure: LlmFailure; + signal: AbortSignal; +} + +export interface ContextOverflowRecoveryOptions { + isBotSession(sessionId: string): boolean; + log(event: Record): void; +} + +const TOKEN_LIMIT = + /\b(?:input|prompt|request)\s+(?:is\s+)?too\s+(?:long|large):\s*model\s+limit\s+\d+\s+tokens?,\s*requested\s+\d+\s+tokens?\b/i; + +export function createContextOverflowRecovery(options: ContextOverflowRecoveryOptions) { + const attempted = new WeakMap(); + return async ( + request: ContextOverflowRequest, + next: () => Promise, + ): Promise => { + const { agent, turn, step, failure, signal } = request; + if ( + failure.code !== 'INVALID_REQUEST' || + !TOKEN_LIMIT.test(failure.message) || + signal.aborted || + !options.isBotSession(agent.session.id) + ) + return next(); + const delegated = await next(); + if (delegated !== undefined || signal.aborted) return delegated; + const previous = attempted.get(agent); + if (previous?.turn === turn && previous.step === step) return undefined; + const presets = agent.ctx.get('agentPresets'); + const compaction = + typeof presets?.serviceFor === 'function' + ? presets.serviceFor(agent, 'compaction') + : agent.ctx.get('compaction'); + if (compaction === undefined) { + options.log({ + event: 'skipped', + outcome: 'declined', + reason: 'compaction-unavailable', + sessionId: agent.session.id, + }); + return undefined; + } + attempted.set(agent, { turn, step }); + const generation = agent.session.surface.replaceGeneration; + const startedAt = Date.now(); + const event = { sessionId: agent.session.id, turn, step, provider: request.provider }; + options.log({ event: 'started', ...event }); + let failed = false; + try { + await compaction.compactIfNeeded(agent, 'context-overflow', signal); + } catch { + failed = true; + } + const progress = agent.session.surface.replaceGeneration > generation; + let persisted = false; + let flushFailed = false; + if (progress && !signal.aborted) { + try { + const sessions = agent.ctx.get('sessions'); + const storage = agent.ctx.get('sessionPersistence'); + const replacement = agent.session + .snapshotEvents() + .findLast( + (entry) => typeof entry.surfaceOp === 'object' && entry.surfaceOp.op === 'replace', + ); + if ( + storage && + replacement && + sessions && + 'flush' in sessions && + typeof sessions.flush === 'function' && + (await sessions.flush(agent.session)) === true + ) { + const reader = await storage.open(agent.session.id, 'read', { signal }); + try { + const stored = await reader.read(replacement.seq, 1, { signal }); + persisted = isDeepStrictEqual(stored.events[0], replacement); + } finally { + await reader.close(); + } + } + } catch { + persisted = false; + flushFailed = true; + } + } + const retry = progress && persisted && !signal.aborted; + options.log({ + event: 'finished', + ...event, + durationMs: Date.now() - startedAt, + outcome: retry ? 'retry' : 'declined', + reason: signal.aborted + ? 'cancelled' + : progress + ? flushFailed + ? 'persistence-failed' + : persisted + ? 'context-replaced' + : 'persistence-unavailable' + : failed + ? 'compaction-failed' + : 'no-progress', + }); + return retry ? { kind: 'retry' } : undefined; + }; +} diff --git a/packages/core/test/context-overflow-recovery.test.ts b/packages/core/test/context-overflow-recovery.test.ts new file mode 100644 index 000000000..24e55b6cb --- /dev/null +++ b/packages/core/test/context-overflow-recovery.test.ts @@ -0,0 +1,278 @@ +import { Context, Service } from '@deepseek-ai/cordis'; +import CompactionEngine from '@deepseek-ai/dsh-compaction'; +import { createUserMessage } from '@deepseek-ai/dsh-llm'; +import { SessionId, SessionStore, type SessionEvent } from '@deepseek-ai/dsh-session'; +import { expect, it, onTestFinished } from 'vitest'; +import { + createContextOverflowRecovery, + type ContextOverflowRequest, +} from '../src/runtime/context-overflow-recovery.js'; + +const OVERFLOW = + '400: {"type":"invalid_request_error","message":"Input too long: model limit 1048576 tokens, requested 1123530 tokens"}'; + +class TestPersistence extends Service { + events: readonly SessionEvent[] = []; + readFailure = false; + closed = 0; + constructor(ctx: Context) { + super(ctx, 'sessionPersistence'); + } + async open() { + return { + read: async (offset: number, length: number) => { + if (this.readFailure) throw new Error('private read failure'); + return { events: this.events.slice(offset, offset + length) }; + }, + close: async () => { + this.closed++; + }, + }; + } +} + +class TestCompaction extends CompactionEngine { + calls = 0; + action = async () => {}; + async compactIfNeeded() { + this.calls++; + await this.action(); + return null; + } + async compactRegion(): Promise { + throw new Error('not used'); + } + async compactNow(): Promise { + throw new Error('not used'); + } +} + +function fixture(owned = true, available = true) { + const ctx = new Context(); + onTestFinished(() => ctx.fiber.dispose()); + const engine = available ? new TestCompaction(ctx) : undefined; + const sessions = new SessionStore(ctx); + const session = sessions.create(SessionId('overflow-test')); + const persistence = new TestPersistence(ctx); + let writer = true; + let flush = async () => {}; + const disposeFlush = ctx.on('session/flush', async () => { + await flush(); + if (writer) persistence.events = session.snapshotEvents(); + }); + const old = session.append( + 'user/message', + createUserMessage({ + content: [{ type: 'text', text: 'Earlier context' }], + source: { kind: 'user' }, + }), + { surfaceOp: 'append' }, + ); + session.append( + 'user/message', + createUserMessage({ + content: [{ type: 'text', text: 'Current request' }], + source: { kind: 'user' }, + }), + { surfaceOp: 'append' }, + ); + const progress = () => + session.append( + 'user/message', + createUserMessage({ + content: [{ type: 'text', text: 'Summary' }], + source: { kind: 'user' }, + }), + { + surfaceOp: { op: 'replace', startSeq: old.seq, endSeq: old.seq }, + sourceEventSeqs: [old.seq], + }, + ); + if (engine) + engine.action = async () => { + progress(); + }; + const logs: Array> = []; + const handle = createContextOverflowRecovery({ + isBotSession: () => owned, + log: (event) => logs.push(event), + }); + const request: ContextOverflowRequest = { + agent: { ctx, session, options: {} }, + turn: 1, + step: 1, + provider: 'opencode-go', + failure: { code: 'INVALID_REQUEST', message: OVERFLOW }, + signal: new AbortController().signal, + }; + let delegated = 0; + const next = async () => { + delegated++; + return undefined; + }; + return { + handle, + request, + next, + engine, + logs, + progress, + delegated: () => delegated, + disposeFlush, + persistence, + removeWriter: () => { + writer = false; + }, + setFlush: (action: () => Promise) => { + flush = action; + }, + }; +} + +it('refuses a participating flush listener without a writer for this Session', async () => { + const f = fixture(); + f.removeWriter(); + expect(await f.handle(f.request, f.next)).toBeUndefined(); + expect(f.request.agent.session.surface.replaceGeneration).toBe(1); +}); + +it('refuses a failed stored replacement read and closes the read handle', async () => { + const f = fixture(); + f.persistence.readFailure = true; + expect(await f.handle(f.request, f.next)).toBeUndefined(); + expect(f.persistence.closed).toBe(1); + expect(JSON.stringify(f.logs)).not.toContain('private read failure'); +}); + +it('waits for the persistence checkpoint before returning a retry', async () => { + const f = fixture(); + let release = () => {}; + const checkpoint = new Promise((done) => { + release = done; + }); + let reached = () => {}; + const entered = new Promise((done) => { + reached = done; + }); + f.setFlush(async () => { + reached(); + await checkpoint; + }); + let settled = false; + const recovery = f.handle(f.request, f.next).finally(() => { + settled = true; + }); + await Promise.race([entered, recovery]); + expect(settled).toBe(false); + release(); + expect(await recovery).toEqual({ kind: 'retry' }); +}); + +it('refuses retry after replacement when persistence fails or has no owner', async () => { + for (const failed of [true, false]) { + const f = fixture(); + if (failed) + f.setFlush(async () => { + throw new Error('private disk failure'); + }); + else f.disposeFlush(); + expect(await f.handle(f.request, f.next)).toBeUndefined(); + expect(f.request.agent.session.surface.replaceGeneration).toBe(1); + expect(JSON.stringify(f.logs)).not.toContain('private disk failure'); + } +}); + +it('does not retry when cancellation arrives during the persistence checkpoint', async () => { + const f = fixture(); + const controller = new AbortController(); + f.setFlush(async () => { + controller.abort(); + }); + expect(await f.handle({ ...f.request, signal: controller.signal }, f.next)).toBeUndefined(); +}); + +it('recovers a missed token-limit error only after native context progress', async () => { + const f = fixture(); + const original = { ...f.request.failure }; + expect(await f.handle(f.request, f.next)).toEqual({ kind: 'retry' }); + expect(f.engine?.calls).toBe(1); + expect(f.request.failure).toEqual(original); + expect(f.request.agent.session.surface.replaceGeneration).toBe(1); +}); + +it('bounds recovery to one attempt in a step, with a fresh budget for a later turn', async () => { + const f = fixture(); + expect(await f.handle(f.request, f.next)).toEqual({ kind: 'retry' }); + expect(await f.handle(f.request, f.next)).toBeUndefined(); + expect(f.engine?.calls).toBe(1); + if (f.engine) + f.engine.action = async () => { + throw new Error('no progress'); + }; + expect(await f.handle({ ...f.request, turn: 2 }, f.next)).toBeUndefined(); + expect(f.engine?.calls).toBe(2); +}); + +it.each([ + ['INVALID_REQUEST', 'Input too long'], + ['INVALID_REQUEST', 'Input too long: model limit 100 characters, requested 120 characters'], + ['AUTH', OVERFLOW], + ['CONTEXT_WINDOW_EXCEEDED', OVERFLOW], +])('leaves %s / %s to the existing handlers', async (code, message) => { + const f = fixture(); + expect(await f.handle({ ...f.request, failure: { code, message } }, f.next)).toBeUndefined(); + expect(f.engine?.calls).toBe(0); + expect(f.delegated()).toBe(1); +}); + +it('does not recover unowned sessions, unavailable services or cancelled turns', async () => { + for (const { f, cancel } of [ + { f: fixture(false), cancel: false }, + { f: fixture(true, false), cancel: false }, + { f: fixture(), cancel: true }, + ]) { + const controller = new AbortController(); + if (cancel) controller.abort(); + expect(await f.handle({ ...f.request, signal: controller.signal }, f.next)).toBeUndefined(); + expect(f.engine?.calls ?? 0).toBe(0); + } +}); + +it('preserves failures and refuses a retry when compaction makes no durable progress', async () => { + const f = fixture(); + if (f.engine) f.engine.action = async () => {}; + expect(await f.handle(f.request, f.next)).toBeUndefined(); + expect(f.engine?.calls).toBe(1); + if (f.engine) + f.engine.action = async () => { + throw new Error('private provider detail'); + }; + expect(await f.handle({ ...f.request, turn: 2 }, f.next)).toBeUndefined(); + expect(JSON.stringify(f.logs)).not.toContain('private provider detail'); +}); + +it('preserves an existing recovery decision without compacting twice', async () => { + const f = fixture(); + expect(await f.handle(f.request, async () => ({ kind: 'retry' }))).toEqual({ kind: 'retry' }); + expect(f.engine?.calls).toBe(0); +}); + +it('accepts committed progress after a summary error, but never retries a cancelled turn', async () => { + const f = fixture(); + if (f.engine) + f.engine.action = async () => { + f.progress(); + throw new Error('private detail'); + }; + expect(await f.handle(f.request, f.next)).toEqual({ kind: 'retry' }); + const cancelled = fixture(); + const controller = new AbortController(); + if (cancelled.engine) + cancelled.engine.action = async () => { + cancelled.progress(); + controller.abort(); + }; + expect( + await cancelled.handle({ ...cancelled.request, signal: controller.signal }, cancelled.next), + ).toBeUndefined(); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 47cb139cc..e33f848b3 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -931,36 +931,39 @@ importers: '@deepseek-ai/cordis': specifier: 4.1.0 version: 4.1.0(@deepseek-ai/cordis-plugin-include@1.1.0)(@deepseek-ai/cordis-plugin-loader@1.1.0) + '@deepseek-ai/dsh-agent': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(d9740b81bf2b48fd86b0c38a41e906a0) + '@deepseek-ai/dsh-agent-preset-registry': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(7137409e0258bf069afa9a47c7e154d1) '@deepseek-ai/dsh-api-gateway': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) '@deepseek-ai/dsh-api-remote-access': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) + '@deepseek-ai/dsh-attachment': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-brand@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) '@deepseek-ai/dsh-client-connection': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-scope@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) '@deepseek-ai/dsh-client-modules': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) - '@deepseek-ai/dsh-experimental-liveshare-protocol': - specifier: 0.2.2-rc.1 - version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) - '@deepseek-ai/dsh-host-webserver': - specifier: 0.2.2-rc.1 - version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(supports-color@10.2.2) - '@deepseek-ai/dsh-agent': - specifier: 0.2.2-rc.1 - version: 0.2.2-rc.1(d9740b81bf2b48fd86b0c38a41e906a0) - '@deepseek-ai/dsh-attachment': - specifier: 0.2.2-rc.1 - version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-brand@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) '@deepseek-ai/dsh-credentials': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) '@deepseek-ai/dsh-credentials-local': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-atomic-write@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0))(@deepseek-ai/dsh-credentials@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0))(@deepseek-ai/dsh-home-paths@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0))(@deepseek-ai/dsh-launch-environment@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) + '@deepseek-ai/dsh-experimental-liveshare-protocol': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0) + '@deepseek-ai/dsh-host-webserver': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(supports-color@10.2.2) '@deepseek-ai/dsh-jobs': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(776462b0d6031c0cdecfb506b0deecf9) @@ -982,6 +985,9 @@ importers: '@deepseek-ai/dsh-session': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-scope@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) + '@deepseek-ai/dsh-session-persistence': + specifier: 0.2.2-rc.1 + version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-brand@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0))(@deepseek-ai/dsh-session@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-scope@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)))(@deepseek-ai/dsh-timeout@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)) '@deepseek-ai/dsh-session-projection': specifier: 0.2.2-rc.1 version: 0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-session@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0)(@deepseek-ai/dsh-scope@0.2.2-rc.1(@deepseek-ai/cordis@4.1.0))) diff --git a/scripts/dev-instance.mjs b/scripts/dev-instance.mjs index 7057c54a7..4f8d0cbc6 100644 --- a/scripts/dev-instance.mjs +++ b/scripts/dev-instance.mjs @@ -103,6 +103,7 @@ export function parseArgs(argv) { productArtifacts: null, dshVersion: null, json: false, + noAxModel: false, trustedHosts: [], clientDiagnostics: process.env['BOTHARNESS_CLIENT_DIAGNOSTICS'] !== '0', }; @@ -175,6 +176,9 @@ export function parseArgs(argv) { case '--json': options.json = true; break; + case '--no-ax-model': + options.noAxModel = true; + break; default: throw new Error(`unknown option: ${flag}`); } @@ -384,9 +388,9 @@ async function main() { options.copyRecoveries = []; try { const secret = resolveDevSecret(); - options.axModel = resolveAxModel(); + options.axModel = options.noAxModel ? undefined : resolveAxModel(); mkdirSync(options.home, { recursive: true }); - if (options.productArtifacts || options.axModel || options.dshVersion) { + if (options.productArtifacts || options.axModel || options.dshVersion || options.noAxModel) { if (options.productArtifacts) verifiedProductArtifacts(options.productArtifacts); const version = options.dshVersion ?? installedDshVersion(options.worktree); assertDshVersion(version); diff --git a/scripts/e2e-context-overflow.mjs b/scripts/e2e-context-overflow.mjs new file mode 100644 index 000000000..a343c8acb --- /dev/null +++ b/scripts/e2e-context-overflow.mjs @@ -0,0 +1,263 @@ +import assert from 'node:assert/strict'; +import { createServer } from 'node:http'; +import { readdirSync, readFileSync, mkdirSync, writeFileSync } from 'node:fs'; +import { zstdDecompressSync } from 'node:zlib'; +import { resolve, dirname } from 'node:path'; +import { cookieFor } from './verify-app.mjs'; + +const instance = JSON.parse(readFileSync(process.env.BH_E2E_INSTANCE, 'utf8')); +const expected = process.env.BH_E2E_EXPECT ?? 'recovered'; +assert(['recovered', 'blocked'].includes(expected)); +const evidence = resolve( + process.env.BH_E2E_EVIDENCE ?? '.humanlayer/verify/context-overflow/result.json', +); +const origin = new URL(instance.url).origin; +async function rpc(method, args = {}) { + const response = await fetch(`${origin}/api/${method}`, { + method: 'POST', + signal: AbortSignal.timeout(30_000), + headers: { + 'content-type': 'application/json', + connection: 'close', + cookie: cookieFor(instance.home), + }, + body: JSON.stringify({ + type: 'client-request', + rpcId: crypto.randomUUID(), + method, + payload: { args }, + }), + }); + assert.equal(response.status, 200, method); + const envelope = await response.json(); + assert( + !envelope.error && envelope.result?.ok !== false, + `${method}: ${envelope.result?.error?.message ?? envelope.result?.error?.code ?? 'refused'}`, + ); + return envelope.result?.ok === true ? envelope.result.value : envelope.result; +} + +let channelId; +let sessionId; +let persistedReplacement; +let summaries = 0; +let refusals = 0; +let retriedWithReplacedContext = false; +let requestOrder = 0; +let summaryOrder; +let retryOrder; +function replacementCheckpoint() { + const root = resolve(instance.home, 'sessions'); + for (const file of readdirSync(root, { recursive: true })) { + if (!file.endsWith('session.v4.jsonl.zstd')) continue; + const bytes = readFileSync(resolve(root, file)); + let text = ''; + for (let offset = 0; offset < bytes.length;) { + const frame = zstdDecompressSync(bytes.subarray(offset), { info: true }); + assert(frame.engine.bytesWritten > 0); + offset += frame.engine.bytesWritten; + text += frame.buffer.toString('utf8'); + } + const [header, ...events] = text.trim().split('\n').map(JSON.parse); + if (header.id !== sessionId) continue; + const summary = events.find((event) => event.type === 'compaction/summary'); + const replacement = events.find((event) => event.surfaceOp?.op === 'replace'); + assert(summary && replacement, 'summary and replacement must be persisted before retry'); + return { + summarySeq: summary.seq, + replacementSeq: replacement.seq, + beforeRequest: requestOrder, + }; + } + throw new Error('No persisted Session for retry'); +} +const server = createServer(async (request, response) => { + try { + let raw = ''; + for await (const chunk of request) raw += chunk; + const input = JSON.parse(raw); + requestOrder++; + const messages = JSON.stringify(input.messages ?? []); + const summary = messages.includes('acting as a compaction engine'); + const probe = messages.includes('[OVERFLOW_PROBE]'); + if (summary) { + summaries++; + summaryOrder = requestOrder; + } + if (probe && !summary && summaries > 0 && retryOrder === undefined) { + retryOrder = requestOrder; + retriedWithReplacedContext = + !messages.includes('[SEED]') && + messages.includes('Earlier seed completed. Preserve the latest recovery request.'); + assert(retriedWithReplacedContext, 'retry must use the summary instead of the old seed'); + persistedReplacement = await replacementCheckpoint(); + } + if (probe && !summary && summaries === 0) { + refusals++; + response.writeHead(400, { 'content-type': 'application/json' }); + response.end( + JSON.stringify({ + type: 'invalid_request_error', + code: 'invalid_request_error', + message: + 'Upstream request failed: [invalid_request_error] Input too long: model limit 1048576 tokens, requested 1123530 tokens', + }), + ); + return; + } + const tool = input.tools?.find((item) => item.function?.name === 'bot_send_message'); + const delta = summary + ? { content: 'Earlier seed completed. Preserve the latest recovery request.' } + : input.messages.at(-1)?.role === 'tool' || !tool + ? { content: 'Done.' } + : { + tool_calls: [ + { + index: 0, + id: `qa-${crypto.randomUUID()}`, + type: 'function', + function: { + name: tool.function.name, + arguments: JSON.stringify({ + channel_id: channelId, + body: probe ? 'OVERFLOW_RECOVERED' : 'SEED_OK', + end_turn: true, + }), + }, + }, + ], + }; + response.writeHead(200, { 'content-type': 'text/event-stream' }); + response.end( + [ + { choices: [{ delta: { role: 'assistant', ...delta }, index: 0, finish_reason: null }] }, + { + choices: [ + { delta: {}, index: 0, finish_reason: delta.tool_calls ? 'tool_calls' : 'stop' }, + ], + usage: { prompt_tokens: 3000, completion_tokens: 20 }, + }, + ] + .map((value) => `data: ${JSON.stringify(value)}\n\n`) + .join('') + 'data: [DONE]\n\n', + ); + } catch { + response.writeHead(500); + response.end('fixture failed'); + } +}); + +async function until(read, accept) { + const deadline = Date.now() + 60_000; + while (Date.now() < deadline) { + const value = await read(); + if (accept(value)) return value; + await new Promise((done) => setTimeout(done, 200)); + } + throw new Error('Context-overflow journey timed out'); +} + +try { + await new Promise((done) => server.listen(0, '127.0.0.1', done)); + const described = await rpc('settings/describe'); + const setting = described.namespaces.find((entry) => entry.ns.includes('llm-pi-ai')); + assert(setting, 'native pi-ai settings available'); + await rpc('settings/update', { + ns: setting.ns, + patch: { + providers: { + 'overflow-fixture': { + api: 'openai-completions', + baseURL: `http://127.0.0.1:${server.address().port}`, + apiKeyEnv: 'DSH_OVERFLOW_FIXTURE_KEY', + models: [{ id: 'fixture', contextWindow: 1_000_000 }], + }, + }, + }, + }); + const route = { provider: 'overflow-fixture', model: 'fixture' }; + await until( + () => rpc('botharness/modelCatalog'), + (result) => result.models.some((model) => model.provider === route.provider), + ); + const { bot } = await rpc('botharness/create', { displayName: 'Context overflow QA' }); + channelId = (await rpc('botharness/channelDm', { slug: bot.slug })).channel.id; + const { preset } = await rpc('botharness/modelPresetCreate', { + name: `Overflow replay ${Date.now()}`, + orchestrator: route, + assignmentDefault: route, + }); + await rpc('botharness/modelPresetApply', { slug: bot.slug, presetId: preset.id }); + await rpc('botharness/channelSend', { + channelId, + body: '[SEED] ' + 'Earlier task context. '.repeat(500), + }); + const messages = () => rpc('botharness/channelMessages', { channelId }); + await until(messages, (result) => result.messages.some((message) => message.body === 'SEED_OK')); + await until( + () => rpc('botharness/list'), + (result) => + result.bots.some((item) => item.slug === bot.slug && item.aggregateState === 'idle'), + ); + for (let index = 0; index < 2; index++) { + await rpc('botharness/channelSend', { + channelId, + body: `[RECENT_${index}] Finish a newer task.`, + }); + await until( + messages, + (value) => + value.messages.filter((message) => message.body === 'SEED_OK').length === index + 2, + ); + await until( + () => rpc('botharness/list'), + (value) => + value.bots.some((item) => item.slug === bot.slug && item.aggregateState === 'idle'), + ); + } + const beforeContext = await rpc('botharness/botContext', { slug: bot.slug }); + assert(beforeContext.sessionId); + sessionId = beforeContext.sessionId; + await rpc('botharness/channelSend', { + channelId, + body: '[OVERFLOW_PROBE] Reply after recovery.', + }); + const result = await until(messages, (value) => + value.messages.some((message) => + expected === 'recovered' + ? message.body === 'OVERFLOW_RECOVERED' + : message.sessionFailure?.code === 'INVALID_REQUEST', + ), + ); + assert.equal(summaries, expected === 'recovered' ? 1 : 0); + assert(refusals > 0); + if (expected === 'recovered') { + assert(retriedWithReplacedContext); + assert(retryOrder > summaryOrder); + assert(persistedReplacement); + } + const afterContext = await rpc('botharness/botContext', { slug: bot.slug }); + assert.equal(afterContext.sessionId, beforeContext.sessionId); + const record = { + expected, + verdict: 'pass', + dshVersion: instance.dshVersion, + summaries, + refusals, + sameSession: true, + retriedWithReplacedContext, + summaryOrder: summaryOrder ?? null, + retryOrder: retryOrder ?? null, + persistedReplacement: persistedReplacement ?? null, + reply: result.messages.find((message) => message.body === 'OVERFLOW_RECOVERED')?.body ?? null, + failureCodes: result.messages.flatMap((message) => + message.sessionFailure ? [message.sessionFailure.code] : [], + ), + }; + mkdirSync(dirname(evidence), { recursive: true }); + writeFileSync(evidence, JSON.stringify(record, null, 2) + '\n'); + console.log(JSON.stringify(record)); +} finally { + server.closeAllConnections(); + await new Promise((done) => server.close(done)); +} diff --git a/scripts/test/dev-instance.test.mjs b/scripts/test/dev-instance.test.mjs index a45af3856..e2b256d52 100644 --- a/scripts/test/dev-instance.test.mjs +++ b/scripts/test/dev-instance.test.mjs @@ -26,6 +26,10 @@ import { assertDshVersion } from '../dsh-compatibility.mjs'; import { qualifiedImProvider } from '../dev-im-provider.mjs'; describe('development CLI admission', () => { + it('can leave machine-local AX model configuration out of a fixture profile', () => { + expect(parseArgs([]).noAxModel).toBe(false); + expect(parseArgs(['--no-ax-model']).noAxModel).toBe(true); + }); it('requires an explicit option for interrupted copy preparation recovery', () => { expect(parseArgs([]).recoverCopyInstall).toBe(false); expect(parseArgs(['--recover-copy-install']).recoverCopyInstall).toBe(true); diff --git a/scripts/test/verify-app.test.mjs b/scripts/test/verify-app.test.mjs index bec29683f..146fd11f7 100644 --- a/scripts/test/verify-app.test.mjs +++ b/scripts/test/verify-app.test.mjs @@ -11,6 +11,9 @@ import { } from '../verify-app.mjs'; describe('verify-app arguments', () => { + it('accepts a fixture launch without the machine-local AX model overlay', () => { + expect(parseVerifyArgs(['launch', '--no-ax-model']).noAxModel).toBe(true); + }); it('forwards explicit runtime selection independently of the build pin', () => { expect(parseVerifyArgs(['launch', '--dsh-version', '0.2.1-rc.99']).dshVersion).toBe( '0.2.1-rc.99', diff --git a/scripts/verification-inventory.json b/scripts/verification-inventory.json index d57e815f7..f66fec7f9 100644 --- a/scripts/verification-inventory.json +++ b/scripts/verification-inventory.json @@ -3,6 +3,14 @@ "auditedBase": "1d1f8ac0cfeb26a8f3bf3a04550e6b92cee52bd9", "boundary": "All scripts/e2e-*.mjs files and all current feature-map documents, augmented by implemented Client/CLI/mobile source entry points. Behavior rows group related acceptance paths; they are not an exhaustive product behavior denominator. Planned unmerged #1954 work and packaged/restart identity promises are excluded.", "behaviors": [ + { + "id": "bot.context-overflow-recovery", + "label": "Native compaction and bounded retry after a gateway token-limit rejection", + "source": "packages/core/src/runtime/context-overflow-recovery.ts", + "guide": ".agents/skills/verify-deepseekbot/features/conversation-timeline.md", + "task": 4253, + "gap": "Local HTTP replay on a real Host; production provider and IM delivery require separate qualification." + }, { "id": "telemetry.bot-tokens", "label": "Anonymous Bot Session token snapshots", @@ -2619,6 +2627,14 @@ "task": 4226, "sha256": "a60b02ee56b57a840535969cefcdf28d1cdfa0bfe5d07047483289aeacbc89a8" }, + { + "path": "scripts/e2e-context-overflow.mjs", + "category": "mapped", + "behaviors": ["bot.context-overflow-recovery"], + "reason": "Same gateway rejection on base and fixed Tobs builds with the native isolated compaction service.", + "task": 4253, + "sha256": "72ee8205df3a84eef1c6ccfc4edfe323d22b02ff418e7b491bc8ecef072c1e62" + }, { "path": "scripts/e2e-token-telemetry.mjs", "category": "mapped", diff --git a/scripts/verify-app.mjs b/scripts/verify-app.mjs index eee4d1e70..a42e2022c 100644 --- a/scripts/verify-app.mjs +++ b/scripts/verify-app.mjs @@ -18,6 +18,7 @@ Drive an isolated DeepSeekBot instance the way a user does and keep the evidence Commands: launch [--port ] [--build] [--dsh-version ] [--genui-preview-browser ] Start an isolated DSH Host through dev-instance.mjs. + --no-ax-model skips machine-local AX model configuration for fixture providers. doctor Check that the recorded instance is ours, alive and authenticated. observe Save Host health and browser-reported Client diagnostics as JSON. rpc [json] Call one authenticated /api method with JSON args. @@ -60,6 +61,7 @@ export function parseVerifyArgs(argv) { botMode: false, keepOverlays: false, build: false, + noAxModel: false, port: undefined, }; for (let index = 0; index < rest.length; index += 1) { @@ -85,6 +87,9 @@ export function parseVerifyArgs(argv) { case '--build': options.build = true; break; + case '--no-ax-model': + options.noAxModel = true; + break; case '--genui-preview-browser': options.genuiPreviewBrowser = value(); break; @@ -236,6 +241,7 @@ async function launch(options, paths) { const port = options.port ?? (await freePort()); const args = ['scripts/dev-instance.mjs', '--home', paths.home, '--port', String(port), '--json']; if (options.build) args.push('--build'); + if (options.noAxModel) args.push('--no-ax-model'); if (options.dshVersion) args.push('--dsh-version', options.dshVersion); if (options.genuiPreviewBrowser) args.push('--genui-preview-browser', options.genuiPreviewBrowser);