cooper (czxtm) · Sep 23, 2026 · Sep 23, 2026
diff --git a/docs/adr/0004-agent-api-on-connect.md b/docs/adr/0004-agent-api-on-connect.md
@@ -0,0 +1,164 @@
+# 0004 — The local agent API is Connect, organized as versioned domain services
+
+- **Status**: Accepted
+- **Date**: 2026-09-23
+- **Related**: [ADR 0005](./0005-per-request-project-selection.md),
+  darkmatter org ADR-0003 (protobuf is the source of truth; Connect is the
+  default transport)
+
+## Context
+
+The Studio talks to the local Go agent (`stack agent`, port 9876) over two
+transports at once:
+
+- **REST/JSON** — ~60 hand-written `/api/*` handlers, called through
+  `AgentHttpClient` (`apps/web/src/lib/agent.ts`) from 28 files. Most
+  local traffic uses it.
+- **Connect-RPC** — one `AgentService` with 48 RPCs
+  (`packages/proto/proto/agent.proto`). Only 13 are reached by live
+  components (project, apps, variables, users, secrets, processes, Nix
+  config, shell status/rebuild, exec, installed packages, module outputs,
+  and `PatchNixData`, which is how the Studio saves config edits). The other
+  35 duplicate REST endpoints and have no caller.
+
+Keeping both has cost correctness, not just code size. The September 2026
+review found the twins drifting apart: the Connect `ReadFile`/`WriteFile`
+handlers skip the project-root containment the REST versions enforce; the
+Connect handler is mounted without the project guard; search results are
+sorted on one side only; a custom `protoc-gen-connect-handlers` plugin
+generates entity getters that ignore request fields. The web side carries
+71 exported hooks in `use-agent.ts` (37 unused), and demo fixtures exist
+twice, once per wire shape.
+
+A separate tRPC API (`packages/api`) serves the cloud features (waitlist,
+GitHub, hosted state). tRPC gets its types from the client importing the
+server's TypeScript router type, so it cannot serve a Go agent. The
+realistic choice for the agent was always REST or Connect.
+
+The Studio is deployed continuously from Cloudflare while each user's agent
+is pinned by their flake, so every Studio build talks to agents of many
+versions. Wire compatibility has to be checkable, not hoped for.
+
+## Decision
+
+The agent exposes one API: **Connect**, defined in protobuf, organized as
+small domain services under a versioned package. REST handlers are retired
+as their callers move. tRPC remains the transport for the TypeScript cloud
+API only.
+
+**Schema**
+
+- Service definitions are hand-written in `proto/stackpanel/agent/v1/`, one
+  file per domain: `AgentService` (`GetAgentInfo` with version and
+  capabilities, `Pair`), `ProjectService`, `ConfigService`,
+  `SecretService`, `ProcessService`, `ShellService`, `CommandService`,
+  `PackageService`, `ModuleService`, `HealthService`, `DeployService`,
+  and `EventService`.
+- Config types are generated from the proto-nix schemas into
+  `proto/stackpanel/config/v1/`. The generator owns the mapping between
+  proto field names and kebab-case Nix attribute names, so a `FieldMask`
+  path is also a `config.nix` attribute path.
+- Messages are typed end to end: no JSON strings or `google.protobuf.Struct`
+  payloads. Singleton settings change through `UpdateConfig` with a
+  `FieldMask`; keyed collections (apps, variables, users, envs) change
+  through per-entry `Upsert`/`Delete` RPCs.
+- Side-effect-free reads set `idempotency_level = NO_SIDE_EFFECTS` so
+  clients may use HTTP GET.
+- `buf lint` and `buf breaking` (against `main`) run in CI, and generated
+  code is committed with a `buf generate` drift check, per org ADR-0003.
+  Field numbers are never reused.
+
+**Server**
+
+- Cross-cutting concerns live in one interceptor chain applied to every
+  service: authentication (`Authorization: Bearer` only; `Pair` is
+  exempt), project resolution ([ADR 0005](./0005-per-request-project-selection.md)),
+  protovalidate (`connectrpc.com/validate`), panic recovery
+  (`connect.WithRecover`), and logging. Handlers contain domain logic only.
+- Each domain lives in `internal/agent/<domain>/`, which owns its logic and
+  a thin Connect adapter. `internal/agent/server` only composes the mux,
+  the interceptors, and the per-project runtime registry.
+- Long-running or live work uses server streaming (logs, shell rebuilds,
+  commands, deploys, health runs). Browsers support only unary and
+  server-streaming calls, so no client or bidirectional streams. The
+  browser reaches the agent over plain `http://localhost`, which means
+  HTTP/1.1 and roughly six connections per origin, so all live updates are
+  multiplexed onto a single `EventService.Watch` stream.
+- Failures use Connect error codes with typed details (for example a
+  `NixEvalError` with file, line, and message) instead of
+  `{success: false}` envelopes.
+- `grpcreflect` and `grpchealth` are served so `buf curl` and `grpcurl` can
+  discover the API.
+
+**Clients**
+
+- The Studio creates one transport in `AgentProvider`, with interceptors
+  that add the auth and project headers, and uses Connect-Query for every
+  read and write. The demo mode implements the services in memory with
+  `createRouterTransport` and one set of typed fixtures.
+- `packages/agent-client` is the published client (generated code plus the
+  transport factory) for the Studio, the TUI, scripts, and third parties.
+- The Studio calls `GetAgentInfo` on connect and gates features on the
+  reported capabilities, showing "update your agent" rather than failing
+  on `Unimplemented`.
+- The CLI keeps calling the Go domain packages directly, so it works
+  without a running agent, and shares that code with the handlers.
+
+## Consequences
+
+**Pros**
+
+- One contract for every client, checked for breaking changes in CI. The
+  version skew between the Cloudflare Studio and local agents becomes a
+  tooling problem instead of a runtime surprise.
+- Cross-cutting behavior (auth, project, validation) cannot be forgotten on
+  one route, which is how the REST/Connect twins drifted.
+- The web data layer becomes generated code: the hand-written hooks, the
+  52 copy-pasted fetch blocks, and the duplicated demo fixtures go away.
+- The typed `FieldMask` path removes the hand-written kebab/snake
+  conversions that caused saved Studio fields to revert.
+
+**Cons and risks**
+
+- Protobuf's type system is narrower than TypeScript's; sum types need
+  `oneof` and optionality needs care.
+- The migration touches most Studio screens. It must move one area at a
+  time, deleting each REST route when its last caller moves, and never
+  keep a REST mirror "for compatibility".
+- Contributors need to learn enough `buf` to change the schema safely.
+
+**Follow-ups** (tracked under stackpanel-thq.8)
+
+1. Delete the 35 unused `AgentService` RPCs, the unused `use-agent.ts`
+   hooks, and the tRPC `agent` router, which proxies to `localhost:9876`
+   from the web server and so cannot reach a user's agent from Cloudflare.
+2. Stand up the v1 services, interceptors, and runtime registry beside the
+   current server.
+3. Move the Studio one area at a time.
+4. Delete the old `AgentService`, the REST routes, the SSE and WebSocket
+   endpoints, `protoc-gen-connect-handlers` and its committed binary, the
+   protobuf-ts output, and the unused JSON-schema output.
+
+## Alternatives considered
+
+- **REST as the single transport.** Carries most traffic today, but has no
+  schema, no generated clients, and no breaking-change tooling; the Go and
+  TypeScript types would stay hand-maintained on both sides.
+- **Adopt the existing `AgentService` as is.** Its shape is the problem: a
+  48-method service with generic entity getters/setters and a custom
+  handler generator. Migrating callers onto it would carry the drift over.
+- **tRPC for the agent.** Not possible: tRPC's types come from a TypeScript
+  server router, and the agent is Go. (Org ADR-0003 also rejects
+  single-language RPC for services with polyglot consumers.)
+- **Serve the Studio from the agent (same origin).** Would remove CORS,
+  pairing tokens across origins, browser local-network prompts, and version
+  skew, but gives up instant UI updates and the hosted cloud features. Not
+  adopted; the capability handshake handles skew instead.
+
+## References
+
+- stackpanel-thq (simplification epic), stackpanel-thq.8 (Agent API v1),
+  stackpanel-0h5 (public agent API), stackpanel-382 (demo fixtures from
+  proto-nix examples)
+- `packages/proto/proto/agent.proto`, `apps/stackpanel-go/internal/agent/server/`,
+  `apps/web/src/lib/{agent.ts,use-agent.ts}`, `packages/api/src/routers/agent.ts`
diff --git a/docs/adr/0005-per-request-project-selection.md b/docs/adr/0005-per-request-project-selection.md
@@ -0,0 +1,101 @@
+# 0005 — Agent requests select their project per request
+
+- **Status**: Accepted
+- **Date**: 2026-09-23
+- **Related**: [ADR 0004](./0004-agent-api-on-connect.md)
+
+## Context
+
+One agent process serves every client on the machine: Studio tabs, the TUI,
+scripts, and (with the planned global service, stackpanel-f1n) the whole
+user session. Today it holds a single project:
+
+- `s.config.ProjectRoot` is chosen at startup (flag or env, then the working
+  directory, then the saved "current project" in
+  `~/.config/stackpanel/stackpanel.yaml`).
+- `/api/project/open` overwrites that field (`project_handlers.go:173`)
+  with no lock while roughly 99 other sites read it, then
+  `reinitializeExecutor()` (`server.go:559`) swaps the singletons: executor,
+  data store, shell manager, and flake watcher.
+- Every client therefore shares one current project. Two Studio tabs, or
+  the Studio and the CLI, on different projects switch each other.
+
+A per-request design has existed since the initial import (2026-01-19):
+`project_context.go` resolves the project from an `X-Stackpanel-Project`
+header, then a query parameter, then the default and current projects. It
+was never mounted on a route. The surrounding code assumed it worked: the
+web app sends the header, CORS allows it, and `stack project` help and the
+docs tell users to send it.
+
+stackpanel-7hn asked for exactly this ("support multiple projects at once
+... by passing the project in every request"). The stackpanel-f1n spec
+(`docs/superpowers/specs/2026-06-15-global-agent-service.md`) instead kept
+a single current project with open/switch endpoints and listed replacing
+them as a non-goal. The two plans conflicted.
+
+## Decision
+
+Each agent request names its project. The agent keeps no global
+"current project".
+
+- A Connect interceptor ([ADR 0004](./0004-agent-api-on-connect.md))
+  resolves the project from the `Stackpanel-Project` request header. When
+  the header is absent it falls back to the registry's default project and
+  then its current project, so interactive CLI use stays convenient.
+- The interceptor attaches that project's **runtime** to the request
+  context. A runtime owns everything project-scoped: the command executor,
+  the data store, the shell manager, the flake watcher, caches, and an
+  `os.Root` opened on the project directory. File access goes through that
+  root, so escaping the project is impossible by construction.
+- Runtimes live in a registry keyed by project ID. They start on first use
+  and are shut down after an idle period.
+- Handlers read the runtime from the context; nothing reads a global
+  project field.
+- Event streams are scoped to a project.
+- The project registry in `~/.config/stackpanel/stackpanel.yaml` stays the
+  source of known projects. "Open" becomes "register or select", not
+  "replace the agent's global state".
+
+This supersedes the single-current-project model in the stackpanel-f1n
+spec; that spec's non-goal is updated accordingly.
+
+## Consequences
+
+**Pros**
+
+- Clients stop interfering with each other; one global agent can serve
+  several projects at once, which the global service needs.
+- The data race on `s.config.ProjectRoot` and the goroutine started on every
+  project switch disappear with the mutable field.
+- Path containment is structural (`os.Root`) instead of a helper each file
+  handler has to remember.
+
+**Cons and risks**
+
+- Memory grows with the number of active projects; idle eviction must be
+  tuned.
+- Caches keyed by project must not leak across runtimes (today a
+  process-wide config cache survives project switches).
+
+**Follow-ups** (tracked under stackpanel-thq.8.1)
+
+- Until the interceptor lands, delete `project_context.go` and correct the
+  CLI help and docs that advertise a header the agent ignores. The
+  resolution logic is small enough to rewrite as the interceptor.
+
+## Alternatives considered
+
+- **Keep one current project with open/switch (f1n spec as written).**
+  Simple, but a global agent shared by several clients makes interference
+  the normal case, and the switch path is racy.
+- **Mount the existing `project_context.go` middleware.** It only resolves
+  a path; every handler still uses per-agent singletons bound to one
+  project, so it would not deliver multi-project behavior.
+- **One agent process per project.** Rejected by stackpanel-f1n: it
+  multiplies services, ports, and pairing state.
+
+## References
+
+- stackpanel-thq.8.1, stackpanel-7hn, stackpanel-f1n
+- `apps/stackpanel-go/internal/agent/server/{project_context.go,project_handlers.go,server.go}`
+- `apps/stackpanel-go/cmd/cli/project.go`, `apps/docs/content/docs/cli/project/`
diff --git a/docs/adr/0006-single-stack-directory.md b/docs/adr/0006-single-stack-directory.md
@@ -0,0 +1,123 @@
+# 0006 — `.stack/` is the single stackpanel directory, with one config file and JSON machine data
+
+- **Status**: Accepted
+- **Date**: 2026-09-23
+- **Related**: [ADR 0004](./0004-agent-api-on-connect.md)
+
+## Context
+
+Stackpanel is aimed at developers who are not Nix natives. For them the
+most important property of the project directory is that every setting has
+one obvious home, and that the file the Studio edits is the file Nix reads.
+
+The September 2026 review found about ten places a config value can live,
+added one after another without the previous one being removed:
+
+| Location | State |
+| --- | --- |
+| `.stack/config.nix` | Canonical; edited by humans and by the Studio (syntax-tree patching) |
+| Split imports (`import ./config.apps.nix`) | Supported; one agent write path inlines them back |
+| `.stack/config/` (haumea tree) | Optional; shallow-overrides `config.nix`; needs haumea in the user's flake; its map-entity list is hand-synced with Go and has drifted |
+| `.stack/data/<entity>.nix` | Go reads and writes it when present; Nix reads only `data/packages.nix`, so Studio edits can land in files Nix never reads |
+| `.stack/data.nix` | Scaffolded by the `default`, `minimal`, and `haumea` templates, documented as "merged with config.nix"; nothing reads it |
+| `.stack/_internal.nix` | The loader's first choice; nothing writes it |
+| `.stack/config.local.nix` | Personal overrides (gitignored) |
+| `.stack/modules/` | Full NixOS-module escape hatch |
+| `.stackpanel/{config.nix,_internal.nix,config/}` | Legacy directory name, still probed (~100 references across Go, Nix, and TypeScript) |
+
+Runtime files were split between `state/` and `profile/` (plus `keys/`,
+`gen/`, `bin/`, `.token`, `secrets/state/`, `secrets/bin/`), taking 14
+`.gitignore` lines; the `state/`/`profile/` split already broke the TUI
+certificate check. Machine-synced data (GitHub collaborators, the SSH-age
+cache) was written as Nix source by Go, which is where several quoting and
+formatting bugs lived, and one writer and its readers disagreed about the
+directory.
+
+## Decision
+
+`.stack/` holds every stackpanel source of truth and all stackpanel state,
+in one layout:
+
+```
+.stack/
+  config.nix        the one config file: plain data, edited by humans and the Studio
+  config.local.nix  personal overrides (gitignored)
+  modules/          Nix escape hatch; the agent never writes here
+  data/             machine-synced, committed, never hand-edited (JSON)
+  secrets/          SOPS-encrypted values and recipients
+  machines/         deploy targets
+  gen/              generated files (gitignored)
+  state/            everything machine-local (gitignored): keys, profile, binaries, tokens, caches
+```
+
+1. **`config.nix` is Nix, but plain data.** No `let` blocks or logic; logic
+   moves to `modules/`. It reads like JSON with `=` and `;`, and the
+   Studio's syntax-tree edits stay safe. Splitting it with plain
+   `import ./<file>.nix` is allowed and understood by the patcher.
+2. **Machine data is JSON under `data/`.** It is lockfile-like: synced from
+   outside and committed so Nix evaluation stays pure. Go writes JSON; Nix
+   reads it with `builtins.fromJSON`. Go no longer emits Nix source for
+   data. Studio-installed packages become an entry in `config.nix`.
+3. **Two ignored directories:** `gen/` for anything regenerable, `state/`
+   for anything machine-local. What is safe to delete is obvious.
+4. **The project root is the nearest directory containing
+   `.stack/config.nix`.** The `.stackpanel-root` marker goes away, and the
+   four root-finding implementations become one rule.
+5. **Removed:** the `.stackpanel/` fallbacks, `_internal.nix`, `data.nix`
+   (from templates and the proto-nix schema options), Go's
+   `data/<entity>.nix` fallback, and the haumea tree layout.
+6. **Files outside `.stack/` are generated projections.** Some files must
+   live where their tool looks: `.envrc`, `turbo.json`, the workspace
+   packages under `packages/gen/`, per-app integration outputs such as
+   `apps/api/fly.toml`. Each is fully regenerated, carries a header
+   pointing back to its `.stack/` source, and is recorded in the CLI's file
+   ledger so `stack doctor` can list and clean them.
+7. **One cutover, no compatibility window.** Go, Nix, TypeScript,
+   templates, and docs change together. `stack doctor` fails with exact
+   move instructions if it finds a legacy path, instead of silently
+   supporting it.
+
+## Consequences
+
+**Pros**
+
+- One home per value; the Studio writes exactly what Nix reads.
+- Removes a class of bugs: Nix source emitted by Go for data, a key encoding
+  shared by hand between Go and Nix, and writers and readers using different
+  directories.
+- Smaller loader and writer in both Go and Nix; no extra flake input for
+  users.
+
+**Cons and risks**
+
+- Projects using the haumea tree or the legacy directory must migrate; the
+  doctor check makes that explicit.
+- Dropping the tree layout gives up per-entry files. If merge conflicts in
+  `config.nix` become a problem, split it with imports rather than bringing
+  back a second layout.
+
+**Follow-ups** (tracked under stackpanel-thq.9)
+
+- Decide whether `.stack/gen/` is committed (AGENTS.md says yes,
+  `.gitignore` says no) and make the docs match.
+
+## Alternatives considered
+
+- **Keep the haumea tree as an option.** Solves per-entry diffs and merge
+  conflicts, but it is a second layout both Go and Nix must support, adds a
+  flake input, has surprising precedence (tree wins, shallowly), and its
+  hand-synced key encoding had already drifted.
+- **A separate agent-written overlay (`data.nix`).** Tried already: two
+  files per setting with precedence rules, and it fell out of use without
+  being removed.
+- **Keep machine data as Nix files.** Requires Go to emit Nix source safely,
+  which is where the quoting and formatter bugs came from.
+- **Keep a compatibility window for `.stackpanel/`.** Rejected: silent
+  fallbacks are how the current ten locations accumulated.
+
+## References
+
+- stackpanel-thq.9 (consolidation), stackpanel-thq.7 (writer/reader
+  directory mismatch)
+- `nix/flake/load-config.nix`, `apps/stackpanel-go/pkg/nixdata/{paths.go,entities.go,store.go}`,
+  `nix/stackpanel/db/lib/proto.nix`, `nix/flake/templates/`