Add caret token healing and restorable prompt caches

[mc-hamster]

Cotabby's pending autocomplete changes need to continue unfinished words without changing already-typed bytes and reuse prompt state after speculative generation or cancellation. This contributes the native patch currently used by Cotabby's hamster-integration branch.

• Add byte-level token pieces and completion-prefix constraints, including UTF-8 fragments and single-line filtering.
• Restore prompt state for recurrent, hybrid, and sliding-window models using a bounded on-device checkpoint; reset sampler state and rearm cancellation only after successful restoration. Failed native decodes invalidate reuse.
• Expose content-free cache diagnostics and add cold/restored equivalence, cancellation, replay, and failed-decode regression tests.
• Run deterministic token-healing tests in CI without a GGUF. Keep the existing Swift package, llama.cpp pin, and release workflow.

The README, source, and test changes match the app's checked-in patch exactly. The only additional change is the CI step. After this merges, the app can adopt the merged package revision and retire its patch/workspace override.

Validation:

• Standalone C++ token-healing tests passed.
• swift test --build-system native with Qwen3.5-0.8B-Base Q6_K: 20 tests passed, no skips or failures.
• Same suite with Gemma-4-E2B Q6_K: 20 tests passed, no skips or failures.
• CI YAML and diff checks passed.
• The default Xcode-backed swift test runner failed locally at code signing because Finder metadata was attached to the test bundle; the documented native SwiftPM runner compiled and executed the complete suites successfully.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: a5fc7052-4a78-4ef5-bf66-e62273f0747f

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}