From 47577618e06de4bb41305ceb6a873542cdec4225 Mon Sep 17 00:00:00 2001 From: Rody Vilchez <102562850+R0SEWT@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:26:50 -0500 Subject: [PATCH 1/2] docs: add an agent workflow laboratory and experiment playbook Refs #6 Co-Authored-By: Claude Opus 5.5 --- README.md | 7 ++++ docs/agent-workflow-lab.md | 71 ++++++++++++++++++++++++++++++++++++ docs/experiments/TEMPLATE.md | 44 ++++++++++++++++++++++ 3 files changed, 122 insertions(+) create mode 100644 docs/agent-workflow-lab.md create mode 100644 docs/experiments/TEMPLATE.md diff --git a/README.md b/README.md index 027c2e6..126755a 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,13 @@ identified (only 3 of 83 repos had active CI). | scaffold | `/project-kit:scaffold` | Lay down the standard scaffold into a new project, then help fill `CLAUDE.md`. | | research-setup | `/project-kit:research-setup ` | Research, score, and record a candidate tool; wire it in if adopted. | +## Agent workflow laboratory + +Use a dedicated lab branch to trial skills, playbooks and runbooks, then promote +validated changes through small PRs. See the [laboratory playbook](docs/agent-workflow-lab.md) +and its [experiment record](docs/experiments/TEMPLATE.md). This is an opt-in +manual workflow; scaffolding does not create branches or install capabilities. + ## The scaffold `scripts/scaffold.sh [options]` does the deterministic work (the skill drives it): diff --git a/docs/agent-workflow-lab.md b/docs/agent-workflow-lab.md new file mode 100644 index 0000000..8987053 --- /dev/null +++ b/docs/agent-workflow-lab.md @@ -0,0 +1,71 @@ +# Agent workflow laboratory + +Use `lab/agent-workflows` as an opt-in experiment branch for skills, playbooks +and runbooks. Approved capabilities live on `main` alongside the code they +support. This guide defines a manual workflow; it does not add a new CLI command. + +## Start an experiment + +Start from a clean checkout and fetch the current base: + +```bash +git status --short +git fetch origin +git worktree add -b lab/agent-workflows ../project-kit-agent-lab origin/main +cd ../project-kit-agent-lab +``` + +Stop if the status output contains changes: commit or preserve them deliberately. +If the branch or worktree already exists, reuse it after inspecting its status; +do not reset, delete or recreate it. For independent experiments, use separate +`lab/` branches and worktrees. Do not use `lab/agent-workflows` both as a +branch and as a prefix for other branches. + +Run experiments against disposable fixtures or a dedicated consumer worktree. +An agent may read the checked-out documentation immediately, while an installed +plugin may still be loading its cached release. Record the actual source path +and commit used. Follow the host's reload procedure in a fresh session and +verify that the experimental skill is loaded before evaluating it. + +## Keep an experiment record + +Copy [the record template](experiments/TEMPLATE.md) into +`docs/experiments/.md`. It is evidence, not a second task tracker: link the +existing bead or GitHub issue instead of maintaining another backlog. + +Declare the problem, hypothesis, affected files, baseline, success criteria and +rollback before testing. Record failures and manual interventions as well as +successful runs. Do not commit transcripts containing credentials or private +project data; retain minimal redacted evidence. + +## Integrate a validated change + +1. Keep the experiment branch current with `origin/main`; merge the base into + the branch and resolve conflicts explicitly. Avoid rewriting shared history. +2. Repeat the relevant checks after updating the base. Verify the actual agent + behavior as well as deterministic scripts and generated files. +3. Open a PR to `main` containing one coherent capability change, its evidence + and its rollback procedure. If the lab contains unrelated trials, create a + clean promotion branch from `origin/main` and cherry-pick only the intended + commits, then revalidate their dependencies. +4. State affected consumers and compatibility changes. A candidate is not a + released capability until its approved commit is available to consumers. +5. Wait for required CI and review resolution. Merge through the repository's + normal process; do not bypass protection or automatically merge. + +If a trial fails, record the rejection or next hypothesis. Do not merge it just +to clear the branch. Keep the worktree until useful work is committed and its +retention is decided. Once a trial is integrated, prefer a fresh branch for the +next trial, especially after a squash merge. + +## First pilot + +Use a notebook-environment capability, grounded in issue #6. Test the same +candidate in two disposable uv projects: one at the repository root and one +nested under `labs/`. Verify interpreter and dependencies, notebook launch, +behavior after environment synchronization, and kernel removal. Do not claim +VS Code discovery works without observing it in VS Code. + +Success means another developer can execute the documented procedure and +recover from a broken kernel without relying on the author's chat history. +This pilot is proposed; no successful execution is claimed by this guide. diff --git a/docs/experiments/TEMPLATE.md b/docs/experiments/TEMPLATE.md new file mode 100644 index 0000000..293f850 --- /dev/null +++ b/docs/experiments/TEMPLATE.md @@ -0,0 +1,44 @@ +# + +- Status: proposed | running | accepted | rejected +- Owner and date: +- Tracking issue or bead: +- Base commit and candidate commit: +- Agent host/version and loaded skill source path: +- Consumer repo/commit and environment: + +## Problem and hypothesis + +What fails today, and what observable outcome should improve? + +## Scope and compatibility + +List changed capabilities, files, dependencies, permissions and affected +consumers. Identify project-specific configuration that must be preserved. + +## Baseline and acceptance criteria + +Describe the baseline behavior and measurable pass/fail conditions. Include +one failure/recovery case and any platform-specific observation required. + +## Evidence + +| Case | Exact command or agent task | Expected | Observed | Evidence | +| --- | --- | --- | --- | --- | +| Baseline | | | Not run | | +| Candidate | | | Not run | | +| Recovery | | | Not run | | + +Record manual interventions and redacted logs. Never substitute an expected +result for an observed result. + +## Decision and integration + +Explain accept/reject, remaining limits, release version if applicable and PR. +Link follow-up issues rather than maintaining an additional task backlog here. + +## Rollback + +Specify the previous capability commit, configuration restoration, and how to +undo external effects such as a registered kernel. Code reversion alone may +not restore the environment. From 1643e9562ee8cca297ab1bc64969485e5881c380 Mon Sep 17 00:00:00 2001 From: Rody Vilchez <102562850+R0SEWT@users.noreply.github.com> Date: Sat, 26 Sep 2026 23:11:42 -0500 Subject: [PATCH 2/2] docs: keep the lab branch off main and load the lab plugin copy Co-Authored-By: Claude Opus 5.5 --- docs/agent-workflow-lab.md | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/agent-workflow-lab.md b/docs/agent-workflow-lab.md index 8987053..0a50470 100644 --- a/docs/agent-workflow-lab.md +++ b/docs/agent-workflow-lab.md @@ -11,10 +11,15 @@ Start from a clean checkout and fetch the current base: ```bash git status --short git fetch origin -git worktree add -b lab/agent-workflows ../project-kit-agent-lab origin/main +git worktree add --no-track -b lab/agent-workflows ../project-kit-agent-lab origin/main cd ../project-kit-agent-lab +git push -u origin lab/agent-workflows ``` +`--no-track` keeps the lab branch from tracking `main`: otherwise a routine +`git pull --rebase` rebases the lab history onto `main`, and `git push` targets +the wrong branch. + Stop if the status output contains changes: commit or preserve them deliberately. If the branch or worktree already exists, reuse it after inspecting its status; do not reset, delete or recreate it. For independent experiments, use separate @@ -23,9 +28,17 @@ branch and as a prefix for other branches. Run experiments against disposable fixtures or a dedicated consumer worktree. An agent may read the checked-out documentation immediately, while an installed -plugin may still be loading its cached release. Record the actual source path -and commit used. Follow the host's reload procedure in a fresh session and -verify that the experimental skill is loaded before evaluating it. +plugin may still be loading its cached release. The installed plugin is +served from its cache (`~/.claude/plugins/cache/...//`), never from the +lab worktree, so reloading does not pick up lab changes. Start a fresh session +that loads the lab copy for that session only: + +```bash +claude --plugin-dir ../project-kit-agent-lab +``` + +Record the actual source path and commit used, and verify that the experimental +skill is loaded before evaluating it. ## Keep an experiment record