Skip to content

Per-turn ISO timestamp in early system prompt busts prompt cache #46

Closed

Description

@prdai

Problem

renderMemoryMetadata injects wall-clock time into the system prompt on every turn:

// src/prompt.ts
return `<memory_metadata>
- The current system date is: ${now.toISOString()}
- Memory blocks were last modified: ${lastModified.toISOString()}
...
`;

That chunk is then spliced early in the system array (index 1, right after the provider header):

// src/plugin.ts
const insertAt = output.system.length > 0 ? 1 : 0;
output.system.splice(insertAt, 0, xml);

Prompt caching (OpenAI / Anthropic-style) is prefix-based. Changing one character early invalidates everything after it. A fresh toISOString() every request means the conversation/tool history never gets a cache hit past that point.

Observed impact

On OpenCode Go with this plugin enabled, token-tracker logs showed:

  • cacheRead stuck around ~22–23k tokens (stable prefix before the memory inject)
  • uncached input growing to 300k–450k/turn in long sessions
  • worst session: ~92% uncached over 376 turns

When the system prefix stayed stable (no per-turn mutation), the same setup hit ~97% cache read. So the provider cache path works; the early volatile metadata is what breaks it.

Empty seeded blocks (human / persona) still emit metadata + timestamp, so this hits even when memory values are unused.

Suggested fix

  1. Drop now.toISOString() from the system prompt (OpenCode already exposes calendar date elsewhere). Keep lastModified only, or omit metadata entirely until a block changes.
  2. Append memory after the stable system prefix (output.system.push(xml)) instead of splicing at index 1, so memory edits only invalidate the trailing chunk — not tools + core instructions.

Happy to send a PR if useful. For now I removed the plugin from my config to stop burning subscription quota.

Refs

Activity

  1. prdai commented on Oct 18, 2026

    @prdai
    Author

    Draft PR up that addresses the remaining half of this: append memory after the stable system prefix instead of splicing at index 1 → #47

    Note: the per-turn toISOString() metadata is already gone on main / 0.3.0. The early splice is what this PR removes.

  2. prdai commented on Oct 18, 2026

    @prdai
    Author

    Closing — the per-turn toISOString() bust is already fixed on main / 0.3.0. Keeping memory early in the system prompt is expected for salience; not pursuing the append follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions