A harness for small-context LLMs — exploring how far a 32k-window model can be pushed on real software engineering tasks when wrapped in an intelligent harness built on the GitHub Copilot SDK.
A 32k-context model wrapped in a sufficiently intelligent harness can complete multi-step software engineering tasks that "naively" require a 200k+ context window, by treating the harness — not the model — as the working memory.
- Qwen3.5 27B via Ollama (local).
- Driven through the Copilot SDK using BYOK (
provider.type = "openai",base_url = http://localhost:11434/v1/).
- Context is a budgeted resource — tracked live via streaming
assistant.usageevents; enforced by the harness, not the model. - Checkpoint-and-reset loop — at a budget threshold, force a structured handoff, end the session, spawn a fresh one. The model is unaware it was reset.
- Sub-agents as context firewalls — heavy reads / exploration happen in ephemeral sub-sessions that return only small structured summaries.
- External memory via tools — plan, notes, todos, decisions live in SQLite. The model edits via narrow tools.
- Hooks enforce discipline — pre-hooks reject wide queries; post-hooks cap/summarize/persist every tool output.
- Skills load on demand — no bloated system prompt.
- Telemetry is product — headline metric: useful actions per 1k tokens.
See plan.md for the full design, open questions, decision log, and experiment roadmap.
| Project | Purpose |
|---|---|
src/SmallContext.Harness |
Core runtime: budget tracker, reset loop, handoff schema, sub-agent pool, hooks, telemetry. |
src/SmallContext.Tools |
Custom tools attached to sessions (memory, code, skill loaders). |
src/SmallContext.Cli |
Console host for running experiments. |
tests/SmallContext.Harness.Tests |
xUnit tests for harness. |
tests/SmallContext.Tools.Tests |
xUnit tests for tools. |
tests/SmallContext.Eval |
Benchmark task ladder runner. |
- .NET 10 SDK
- Ollama with a model pulled (e.g.
ollama pull qwen2.5:32b— substitute the actual Qwen3.5 27B tag once available). - GitHub Copilot CLI (the .NET SDK bundles it; see SDK setup).
git clone https://github.com/MSBart2/smallcontext.git
cd smallcontext
dotnet build
dotnet run --project src/SmallContext.Cli/SmallContext.Cli.csprojThe CLI reads provider connection values from src/SmallContext.Cli/appsettings.json and also supports environment overrides such as:
To run the first real BYOK/Ollama smoke test:
dotnet run --project src/SmallContext.Cli/SmallContext.Cli.csproj -- --smoke-test$env:SMALLCONTEXT_OLLAMA__MODEL = "qwen3.5:27b"
$env:SMALLCONTEXT_OLLAMA__BASEURL = "http://localhost:11434/v1/"Design / pre-implementation. See plan.md for the active experiment queue.