Project Apollo — Knowledge-Work Agent Evaluation Environment

A self-contained, locally-running simulation of two knowledge-work tools — a Slack-like messaging service and a task manager — plus one or more unsaturated agentic tasks authored in the Harbor framework and an analysis of where a weak model (Gemma) fails them.

This is an agent evaluation environment, not a model-training loop. The environment holds its own state and exposes agent-callable tools; tasks pair a seeded world with a deterministic verifier whose score is the reward signal.

See plan.md for the full phased implementation plan and stand_out.md for the architectural differentiators.

Quickstart

# 1. Install uv (https://docs.astral.sh/uv/) if you don't have it.
# 2. Sync the environment (creates .venv, installs deps, pins Python 3.12):
uv sync --extra dev

# 3. Run the correctness gate:
uv run pytest

# 4. Explore the environment interactively:
uv run python -c "from apollo_env import Workspace; from apollo_env.seeds import SCENARIOS; \
ws = Workspace().seed(SCENARIOS['incident_triage']()); \
print([c.name for c in ws.slack.list_channels()])"

Layout

src/apollo_env/        Core environment
  slack/               Simulated Slack: models, errors, SlackService engine
  tasks/               Simulated Task Manager: models, errors, service engine
  grading/             Reusable code-based graders (string/outcome/tool-call/...)
  seeds/scenarios/     Named, JSON-serializable seed worlds (one per task)
  workspace.py         Composes both services; deterministic seed() / reset()
  mcp_server.py        FastMCP server exposing every tool to an agent
harbor_tasks/          Harbor task dirs: smoke-test, incident-triage,
                       client-update, priority-conflict (each with oracle + verifier)
experiments/           ReAct agent + sweep runner across the model ladder
analysis/              Statistics (stats.py), loss taxonomy, report.html
tests/                 pytest correctness gate
docs/                  Client update, loss analysis, design-decision log

What makes it a reliable RL harness

• Deterministic rollouts — reset() deep-copies the seed and rebuilds the entire object graph, so trial N can never see trial N−1's mutations.
• Self-correcting tool layer — invalid ids return structured, actionable hints ("Task T999 not found. Use list_tasks or search_tasks...") instead of crashing the agent loop.
• Partial-credit verification — the flagship task scores four independent rewards into reward.json, pinpointing exactly which hop a model dropped.
• Trajectory auditing — the verifier checks the agent actually read before it wrote, defeating reward-hacking shortcuts.
• Reusable code-based graders — verifiers compose objective checks (string/ regex/fuzzy, outcome, tool-call, transcript, binary) from apollo_env.grading, and the priority-conflict task's precision check turns reward hacking into a measured, deterministic signal.

Running the experiment sweep

cp .env.example .env            # add OPENROUTER_API_KEY (+ ANTHROPIC_API_KEY for the baseline)
# optional: a local Ollama rung — `ollama pull qwen2.5:7b-instruct && ollama serve`
uv run --extra experiments python experiments/run.py --task incident-triage
uv run --extra experiments python experiments/run.py --task client-update
uv run --extra experiments python experiments/run.py --task priority-conflict
uv run --extra analysis     python analysis/report.py     # -> analysis/report.html

The sweep runs each task through a thin LiteLLM + MCP-client ReAct agent (experiments/agent.py) across the model ladder defined in experiments/configs/sweep.yaml, scores every trial with the task's real grader-composed verifier, and persists trajectories (plus per-trial tool-trace diagnostics, turns, and tokens) under experiments/results/<run>/.

Results (pass@1, 7 models × 10 trials/task)

Model	incident-triage	client-update	priority-conflict
qwen2.5-7b (local)	0%	100%	0%
llama-3.1-8b	70%	60%	90%
gpt-4o-mini	90%	10%	100%
mistral-small-24b	60%	100%	0% (reward-hacks)
gemma-2-27b	0%	0%	100%
gemma-3-27b	100%	80%	100%
claude-sonnet (baseline)	100%	100%	100%

The tasks are unsaturated and discriminating, and no model solves all three — each isolates a different failure: gpt-4o-mini forgets the cross-tool task update on client-update; gemma-2 never posts the final threaded reply on incident-triage; mistral-small reward-hacks priority-conflict (caught by the precision check). The partial-credit verifiers and tool-trace diagnostics localize each one. Full writeups: docs/client_update.md, docs/loss_analysis.md, and the interactive hosted report generated from analysis/report.html.

Status

Environment, 15-tool MCP layer, reusable grader library, four Harbor tasks (oracles pass), a seven-model sweep with statistics, loss analysis, and writeups — all complete. See plan.md and docs/design_decisions.md.