Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions oxa/docs/work-state.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Runner checkpoint integration

The runner is the sole writer of task state. `run_task_with_observer` accepts a
`CheckpointObserver: Sync` with `checkpoint(&Record) -> Result<()>`. Observers
consume public typed records; they never read or rewrite private `State` files.
The CLI currently supplies a no-op while the separately owned facts store is
integrated. The hook is not itself an event journal or situation implementation.

Checkpoints live outside writable Git binds in the protected per-repository
runner root. `<id>.json` stores a versioned private state with its task spec,
frozen execution inputs, offline policy, deadline and public record. `<id>.lock`
is a stable inherited kernel lock. Status is read-only and does not acquire it.
Facts storage receives a separate `facts/` child of this protected root, avoiding
collisions with task files. Standing data also remains outside Git.

`Record.checkpoint_revision` is repository-wide and strictly increases. The event
producer domain is the fixed string `runner`, not one source per task. Event
sequence numbers belong to the facts store and are distinct from checkpoint
revisions. Reservations that did not reach the task rename can leave revision
gaps; they must not create event-sequence gaps.

A short repository kernel lock serializes revision allocation, task-file
replacement, observation and acknowledgement. `.checkpoint-counter` contains
the last reserved revision and at most one pending task ID. A new writer must
recover that pending durable checkpoint before reserving another revision:

1. Reserve the revision and pending identity atomically, then sync the directory.
2. Write and sync the complete task checkpoint, replace its canonical name
atomically, and sync the directory. The first task publication is create-only.
3. Call the observer with the durable public record.
4. Atomically acknowledge that observation in the repository counter.

A failure stops the batch and poisons that writer. Recovery replays the same
durable revision, including a lost acknowledgement after an append. The observer
must deduplicate that identity and reject different contents under it. Older
acknowledged task records are not replayed: the source table needs one bounded
runner high-water mark, without completed-task tombstones. A missing/corrupt
initialized counter fails closed rather than resetting revisions.
`recover_checkpoints(root, observer)` exposes that pending-observation recovery
before a lead reads its facts view; it never acquires a task lock, changes a task
file or starts a worker. The plain `status` command remains entirely read-only.

`checkpoint_kind` distinguishes `task_transition` from `worker_finished`. The
worker result is saved at acceptance/recheck with the latter kind; a separate
revision enters that phase before running checks. A crash between the two does
not restart the worker. A preparing transition can establish a `TaskStarted`
location; subsequent transitions update the closed step and optional session.
Worker completion, local acceptance, publication, external acceptance and merge
remain separate observations. Only a merge is landed work.

Records expose fixed outcomes and typed deadline interruption. Their `detail`
is bounded diagnostic context for the owner, not a facts payload: event adapters
select typed fields and fixed reason codes and never emit prompts or raw logs.
Saved head/tree IDs describe the actual saved commit. They do not prove that an
ignored helper or build artifact was absent while local checks ran; publication
must verify commit-bound acceptance separately.
79 changes: 57 additions & 22 deletions oxa/docs/work.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,14 @@ Run it inside the project repository. The JSON file is an array:
[{"id":"docs-fix","model":"your-model","branch":"agent/docs-fix","base":"main","prompt":"Fix documentation links."}]
```

`base` defaults to `main`. IDs contain letters, digits, hyphens or underscores.
`base` defaults to `main`. IDs contain 1–128 letters, digits, hyphens or underscores.
Optional `max_time` is a positive integer number of seconds, defaulting to 2700
(45 minutes). The task's saved deadline is shared by its worker, repair and crash
recovery; a repair does not get a fresh budget. Each child receives its remaining
allowance as `--max-time` for one graceful wrap-up turn. If it remains alive 60
seconds after that allowance, the runner sends SIGTERM, then SIGKILL after another
10 seconds. The result is interrupted and its worktree is retained. `kill` must be
available as a trusted executable outside the project; status reads do not need it.
IDs and branches must be unique within one file. Existing task branches can be
continued. For a GitHub issue use `oxa work --issue 123 --model your-model`.
This explicitly invokes your installed,
Expand Down Expand Up @@ -36,7 +43,7 @@ creates detached Git metadata with callbacks suppressed on the host, then create
the task branch and populates the fresh worktree inside the normal sandbox.
Reference hooks, checkout hooks and smudge filters stay inside it; a reference hook
that rejects the task branch still blocks the task.
missing bwrap blocks before checkout or a worker starts. Configured hook directories
Missing bwrap blocks before checkout or a worker starts. Configured hook directories
keep their normal read-only protection: a checkout that needs to populate a tracked
protected hook directory blocks with the worktree retained.
Acceptance commands are frozen from the captured base commit before checkout
Expand All @@ -53,41 +60,69 @@ error blocks the task rather than producing a clean verdict.
Agent-internal done checks may also run when the project is trusted.

The runner reads the child's exit code, terminal `oxa_result` and session header,
without retaining its event stream. A blocked, failed or interrupted worker remains
blocked and does not consume a repair attempt. Failed acceptance after a successful
without retaining its event stream. It checkpoints the session ID as soon as the
header arrives. A blocked or failed worker remains blocked; an interrupted worker
remains interrupted. Neither consumes a repair attempt. Failed acceptance after a successful
worker continues that same saved session once, then acceptance runs once more.
JSON lines include the worker result, session ID and captured `base_sha` and report `done`, `nothing_to_do` (no
commits or file changes), `red` (checks still fail), or `blocked` (setup or agent
execution failed). Exit codes are 0 for success, 1 if any task is red or blocked,
and 2 for invalid input or output errors. A task setup failure produces a blocked
JSON lines include the worker result, session ID, captured `base_sha`, saved
`head_sha`/`tree_sha`, step, checkpoint revision and time. Outcomes are `done`,
`nothing_to_do` (no worker commits or file changes), `red` (checks still fail),
`blocked` (setup or execution failed), or `interrupted`. Deadline interruptions
carry a typed `interruption: "deadline"`. Exit codes are 0 for success, 1 if any
task is red, blocked or interrupted, and 2 for invalid input, output or checkpoint
errors. A checkpoint/event-append error stops the entire batch before another
action. An ordinary task setup failure produces a blocked
JSON line and does not skip later tasks. If a task cannot be locked or has an
incompatible receipt, its existing state is preserved; a pre-worktree failure has
an empty `worktree` in the result. Locks use kernel ownership, so an abandoned lock file
after process death does not permanently block a task.
an empty `worktree` in the result. Locks use kernel ownership and are inherited
by the worker and check/save supervisors. Killing the runner does not allow a
second runner to start while its child still holds that lock.

Worktrees and uncommitted changes remain in `.oxa-worktrees/<repository>/<id>`
Worktrees remain in `.oxa-worktrees/<repository>/<id>`
beside the repository (or the separate git directory). Private result receipts and
locks live under `~/.oxa/agent/work/<repository-hash>/` (or the configured agent
directory), outside the worker's writable Git bind. New receipts are published
atomically without overwriting an existing receipt; a validated matching receipt
is replaced atomically under the task lock when continuing. Worktrees stay outside that git
directory so the normal sandbox can bind git objects and refs for commits. No automatic
cleanup, commit, push or PR creation is implemented yet. Inspect
retained work before removing it with `git worktree remove`. Review task prompts
and bases before running them.
directory so the normal sandbox can bind git objects and refs for commits.
Before finishing, the runner saves staged, unstaged and untracked work with an
ordinary sandboxed commit, including red, blocked and interrupted work. Normal
hooks remain active. Unresolved merges are kept in progress; a rejecting hook or
remaining dirty files block completion. No automatic cleanup, push or PR creation
is implemented. Keep retained work until its saved commit has been published and
inspected. Review task prompts and bases before running them.

`done` here describes accepted local work, which may be uncommitted. This slice
does not publish it. The saving/publication slices must bind acceptance to a clean
committed tree and verify its recorded commit before pushing.
`done` here describes locally checked and saved work. It does not mean published,
externally accepted or merged. Publication must verify a clean committed export,
bind checks to that commit and verify base-object integrity before pushing;
ignored files in a retained working directory are not commit-bound evidence.

To continue a task, run the same task file again. A matching retained receipt and
linked worktree are reused. The source checkout is never used as the task worktree.
An existing branch checked out elsewhere is blocked. Dirty retained work is kept
and blocked until committed. A clean retained branch gets an ordinary sandboxed
To recover a task, run the same task file again. An unfinished checkpoint resumes
its saved step with the original base, commands, session and offline policy.
Recovery loads those inputs before resolving the current base branch or settings,
so a deleted base ref or invalid new checks cannot strand an existing attempt.
Finished successful work is returned without another worker or check run; add
`--rerun` to the task-file or issue command to start another attempt. A task spec
change requires a new ID. Unsupported or corrupt checkpoints are preserved and
refused. A matching linked worktree is reused; the
source checkout is never used as the task worktree. An existing branch checked
out elsewhere is blocked. Dirty retained work is preserved and saved before
retrying. A clean retained branch gets an ordinary sandboxed
merge of the captured base; no reset, stash, force checkout or automatic fetch runs.

An unfinished merge is passed to the worker with its old pending commit, the
captured current base and at most twenty conflicted paths. The worker must finish
that merge and merge the captured base too. Aborted or unresolved merges and branch
switches remain blocked even if acceptance passes. Worker, merge and acceptance
failures are retained together. Receipt replacement keeps private modes.

`oxa work status --json` (also `--status`) reads checkpoints without acquiring a
task lock, changing state, constructing a model or calling GitHub. Its separate
`liveness` is `running`, `finished`, `interrupted`, or `unknown` when the kernel
cannot establish ownership. A missing original parent is not proof of a stopped
child: status also checks the inherited stdin lock. Without `--json` it prints
one short line per task.

The [checkpoint integration contract](work-state.md) defines the runner's public
event hook. Event/situation/standing stores and their CLI adapters are a separate
integration; this slice does not infer facts from worker prose or raw logs.
39 changes: 39 additions & 0 deletions oxa/oxa-cli/src/work_checks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,45 @@ pub(crate) fn run(
}
return Ok(i32::from(result.exit != Some(0)));
}
SupervisorOperation::Save { task_id } => {
let sandbox = crate::netguard::with_net(Arc::new(b), &settings);
let message = format!("oxa work: save task {task_id}").replace('\'', "'\\''");
let spec = ExecSpec {
command: format!(
r#"git --no-replace-objects diff --cached --quiet --no-ext-diff --no-textconv --diff-filter=U
work_index_status=$?
if test "$work_index_status" -ne 0; then exit 2; fi
git --no-replace-objects add --all || exit 2
git --no-replace-objects rev-parse --verify --quiet MERGE_HEAD >/dev/null
work_merge_status=$?
if test "$work_merge_status" -ne 1; then exit 2; fi
git --no-replace-objects diff --cached --quiet --no-ext-diff --no-textconv
work_changes=$?
if test "$work_changes" -eq 0; then exit 0; fi
if test "$work_changes" -ne 1; then exit 2; fi
git --no-replace-objects commit -m '{message}'
"#
),
cwd: root,
writable: true,
net: sandbox.network(),
timeout_s: Some(600),
env: git_environment(),
};
let mut tail = Vec::new();
let result = sandbox.run_streaming(&spec, &CancelToken::default(), &mut |bytes| {
keep_tail(&mut tail, bytes, 8192);
});
if result.exit != Some(0) || result.timed_out {
return Err(format!(
"save exit {:?}, timeout {}: {}",
result.exit,
result.timed_out,
String::from_utf8_lossy(&tail)
));
}
return Ok(0);
}
SupervisorOperation::Checks { commands } => commands,
};
let shell = oxa_tools::ShellConfig {
Expand Down
Loading
Loading