Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 58 additions & 56 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Development

Flow is an ESM TypeScript package built and tested with Git and the versions
Flow is an ESM TypeScript package built and tested with Git and versions
pinned in `package.json`.

## Local setup
Expand All @@ -24,36 +24,39 @@ bun run package:smoke
```

The opt-in real-host check launches the pinned `opencode-ai` package through
`bunx`. It therefore requires registry access or an already populated Bun
cache; it does not use a separately installed OpenCode binary:
`bunx`. It requires registry access or a populated Bun cache rather than a
separately installed OpenCode binary:

```bash
bun run smoke:live
```

### Bumping the pinned host

`@opencode-ai/plugin` and `zod` are on the ignore list in
`.github/dependabot.yml`, so they are raised by hand. The plugin pin is also the
host version `smoke:live` launches, so bumping it is what puts a new host under
test:
`@opencode-ai/plugin` and `zod` are on `.github/dependabot.yml`'s ignore list,
so they are raised by hand. The plugin pin is also the host version
`smoke:live` launches, so bumping it puts a new host under test:

1. Move the `devDependencies` pin, and `peerDependencies` too if the new version
falls outside the declared range.
2. Run `bun run check`, then `bun run smoke:live` for the real host.

Widen the peer range only for a host that has been smoke-tested; a range that
admits versions no check has launched is a compatibility claim with nothing
behind it.
Widen the peer range only for a smoke-tested host; a wider range makes a
compatibility claim no check has run.

## Source layout

- `src/domain/` owns Session v5 values, invariants, and transitions that use
only JavaScript/Node standard-library primitives.
- `src/application/` owns use cases and repository ports.
- `src/infrastructure/` owns filesystem persistence and source fingerprinting.
- `src/infrastructure/` owns filesystem persistence and source fingerprinting:
`fs/workspace-paths.ts` (workspace root validation, `.flow` layout),
`fs/managed-fs.ts` (managed filesystem primitives), `fs/session-lock.ts`
(cross-process session lock), `fs/workspace.ts` (session file protocol).
- `src/platform/opencode/` owns OpenCode hooks, host schemas, commands, tools,
validation capture, and the duplicate-runtime guard.
validation capture, and the duplicate-runtime guard: `command-hook.ts`
(slash-command hook), `tool-guard.ts` (leadership and auto-drive guard
around the tools), `plugin.ts` (wiring only).
- `src/guidance/`, `skills/`, and prompt surfaces own concise workflow judgment.
- `tests/` prove state-machine, persistence, platform, package, and host
contracts.
Expand All @@ -62,67 +65,65 @@ Dependencies point inward. Domain code does not import filesystem or host APIs;
application code depends on domain; infrastructure implements application
ports; the OpenCode platform composes the outer layers.

There is no distribution/activation subsystem, cache inventory, repair journal,
or Flow-owned installer. OpenCode installs and loads the npm package from its
native plugin command and normal plugin configuration.
There is no distribution/activation subsystem, cache inventory, repair
journal, or Flow-owned installer: OpenCode installs and loads the npm package
from its native plugin command and normal configuration.

## Change discipline

- Keep Session v5 as one canonical run aggregate. Derive status and progress
instead of adding parallel ledgers or cached counters.
- Keep Session v5 as one canonical run aggregate: derive status and progress
instead of parallel ledgers or cached counters.
- Every mutation needs a revision guard and stable operation ID. Exact replay is
safe; conflicting reuse fails.
- Only the reserved reviewer may create a new completion. While the Session v5
workflow remains active, every caller receives an exact accepted completion
replay through a read-only path that does not cancel validation or write
- Only the reserved reviewer may create a new completion; while the Session v5
workflow remains active, every other caller gets an exact accepted-completion
replay through a read-only path that neither cancels validation nor writes
session state.
- Keep validation host-observed and session-native. Do not add caller-authored
- Keep validation host-observed and session-native: no caller-authored
success, detached receipt stores, or clock requirements.
- Treat validation scope as a coverage claim. `broad` means the canonical
repository gate, byte for byte. Do not promote a narrow command by
relabeling it.
- Validation commands are persisted. Never inline secrets. Raw output is
intentionally reduced to completeness and a digest rather than stored or
projected.
- Treat validation scope as a coverage claim: `broad` means the canonical
repository gate, byte for byte, not a narrow command relabeled.
- Validation commands are persisted, never with inline secrets. Raw output is
reduced to completeness and a digest rather than stored or projected.
- Keep one review per run. A final review requires broad validation and is not a
second pass. The reviewer submits through `flow_feature_complete`; the
manager never proxies its verdict.
- Prefer deletion when a test or document exists only for a removed concept.
Do not preserve a dual stack for pre-v6 active state.
- Use table-driven lifecycle and persistence tests. Avoid registries that test
- Prefer deletion when a test or document exists only for a removed concept,
not a dual stack for pre-v6 active state.
- Use table-driven lifecycle and persistence tests, not registries that test
the presence of other tests.

## Documentation

Update the README, maintainer contract, ADR, and changelog when a public
lifecycle or installation contract changes. Documentation must describe only
the current product; Git history owns superseded plans and experiments.
lifecycle or installation contract changes; documentation describes only the
current product, and Git history owns superseded plans and experiments.

## Model-driven wave evidence

Deterministic CI validates schemas, permissions, prompts, and host integration
without provider credentials. It does not claim that a model actually overlaps
workers. Changes to wave behavior should therefore be exercised manually with a
real provider when available and accompanied by sanitized evidence of:
without provider credentials. It does not claim a model overlaps workers, so
changes to wave behavior should be exercised manually with a real provider
when available, with sanitized evidence of:

- worker start/end times with a positive common overlap;
- assigned versus changed paths and any scope drift;
- permission prompts or denials and worker Bash calls; and
- reviewer-owned `flow_feature_complete` submission.

Every wave-behavior change must include this evidence when marked verified, but
it is not a deterministic release gate. When a provider is unavailable, mark
the behavior unverified, record the review risk, and avoid performance or
Every wave-behavior change needs this evidence when marked verified, but it is
not a deterministic release gate. Without a provider, mark the behavior
unverified, record the review risk, and avoid performance or
reliability claims. Do not persist prompts, secrets, raw provider payloads, or a
wave ledger, and do not add provider credentials, a scheduler, or telemetry to
CI.

## Model-driven auto-continuation evidence

Deterministic tests exercise the coordinator through the real plugin hooks and
the `promptAsync` client boundary. They do not prove how a configured model
the `promptAsync` client boundary, but do not prove how a configured model
behaves after delivery. When auto-continuation behavior changes and a provider
is available, run one packed-plugin canary that records sanitized evidence of:
is available, run one packed-plugin canary recording sanitized evidence of:

- idle `ready` delivery with the Flow token and compact revision;
- recommendation or clarification at a checkpoint remaining waiting, followed
Expand All @@ -144,27 +145,28 @@ deterministic hook and lifecycle gates.

## Release

Follow the [frozen-candidate sequence](release-qualification.md#running-it): finish
fixes and dependency updates, pass deterministic checks, then approve paid evals.
`bun run qualify -- --campaign-dir <dir> --canary <record>` seals the complete
two-provider campaign, exact-artifact canary and grader evidence. Commit that bundle
before tagging; never substitute interrupted results for qualification.
Follow the [frozen-candidate sequence](release-qualification.md#running-it):
finish fixes and dependency updates, pass deterministic checks, approve paid
evals.
`bun run qualify -- --campaign-dir <dir> --canary <record>` seals the
two-provider campaign, exact-artifact canary and grader evidence. Commit that
bundle before tagging; never substitute interrupted results for qualification.

Release tags use `v<package-version>`. Blocking release checks include the
normal repository gate, package smoke, packed live OpenCode smoke, package
integrity generation, npm publication, and GitHub release assets. There is no
cross-version active-session gate because v6 is an explicit hard cutover.
Release tags use `v<package-version>`. Blocking release checks: the normal
repository gate, package smoke, packed live OpenCode smoke, package integrity
generation, npm publication, and GitHub release assets. There is no
cross-version active-session gate; v6 is an explicit hard cutover.

Publication accepts both annotated and lightweight tags, but the freshly fetched
tag, workflow event, checkout, and current remote `main` tip must identify the
same commit immediately before npm publication. Network calls have explicit
deadlines. npm publication reconciles the immutable package integrity after every
result, including timeouts. GitHub publication first builds an exact draft under
the same ref proof. That draft is the recovery marker if npm succeeds and `main`
then advances. Finalization rechecks the remote tag, refuses conflicting metadata
same commit before npm publication. Network calls have explicit deadlines. npm
publication reconciles the immutable package integrity after every result,
including timeouts. GitHub publication first builds an exact draft under the
same ref proof. That draft recovers if npm succeeds and `main` then advances.
Finalization rechecks the remote tag, refuses conflicting metadata
or assets, and publishes only after every asset digest matches. Reruns converge
after partial success without replacing published bytes.

Preparing an already-published release is read-only and requires exact assets.
Missing or pending assets fail preparation; use the explicit `github-publish`
recovery path with the original inputs and tag proof to restore a missing asset.
Missing or pending assets fail preparation; use the `github-publish` recovery
path with the original inputs and tag proof to restore a missing asset.
162 changes: 162 additions & 0 deletions src/infrastructure/fs/managed-fs.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
import { randomUUID } from "node:crypto";
import { constants } from "node:fs";
import { lstat, mkdir, open, rename, rm, writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { setTimeout as sleep } from "node:timers/promises";
import { UnreadableFlowSessionError } from "../../application/errors.js";
import {
MAX_SESSION_BYTES,
SESSION_CLOSE_RESERVE_BYTES,
} from "../../domain/limits.js";
import { flowDir, historyDir } from "./workspace-paths.js";

export class UnsafeFlowWorkspaceLayoutError extends Error {
readonly code = "UNSAFE_FLOW_WORKSPACE_LAYOUT";
}

export async function pathKind(
path: string,
expected: "file" | "directory",
description: string,
): Promise<"missing" | "present"> {
try {
const info = await lstat(path);
if (info.isSymbolicLink()) {
throw new UnsafeFlowWorkspaceLayoutError(
`Flow refuses a symbolic link for ${description}: ${path}.`,
);
}
if (
(expected === "file" && !info.isFile()) ||
(expected === "directory" && !info.isDirectory())
) {
throw new UnsafeFlowWorkspaceLayoutError(
`Flow requires ${description} to be a ${expected}: ${path}.`,
);
}
return "present";
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return "missing";
throw error;
}
}

async function ensureDirectory(
path: string,
description: string,
): Promise<void> {
if ((await pathKind(path, "directory", description)) === "present") return;
try {
await mkdir(path, { mode: 0o700 });
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error;
}
await pathKind(path, "directory", description);
}

export async function ensureFlowDirectory(workspace: string): Promise<void> {
const root = flowDir(workspace);
await ensureDirectory(root, "the Flow state directory");
const ignore = join(root, ".gitignore");
if ((await pathKind(ignore, "file", "the Flow ignore file")) === "missing") {
try {
await writeFile(ignore, "*\n", {
encoding: "utf8",
flag: "wx",
mode: 0o600,
});
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "EEXIST") throw error;
}
}
}

export async function ensureHistoryDirectory(workspace: string): Promise<void> {
await ensureFlowDirectory(workspace);
await ensureDirectory(historyDir(workspace), "the Flow history directory");
}

export async function readManaged(
path: string,
description: string,
synchronizeFile = false,
): Promise<string> {
await pathKind(path, "file", description);
const noFollow = process.platform === "win32" ? 0 : constants.O_NOFOLLOW;
const access = synchronizeFile ? constants.O_RDWR : constants.O_RDONLY;
const handle = await open(path, access | noFollow);
try {
const stat = await handle.stat();
if (
!stat.isFile() ||
stat.size > MAX_SESSION_BYTES + SESSION_CLOSE_RESERVE_BYTES
) {
throw new UnreadableFlowSessionError(
`${description} is not a bounded regular file.`,
"state exceeds the supported session size",
);
}
const contents = await handle.readFile("utf8");
if (synchronizeFile) await handle.sync();
return contents;
} finally {
await handle.close();
}
}

export async function syncDirectory(path: string): Promise<void> {
if (process.platform === "win32") return;
const handle = await open(path, "r");
try {
await handle.sync();
} finally {
await handle.close();
}
}

async function renameReplacing(temporary: string, path: string): Promise<void> {
let retry = 0;
while (true) {
try {
await rename(temporary, path);
return;
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
const transientWindowsError =
process.platform === "win32" &&
(code === "EACCES" || code === "EBUSY" || code === "EPERM");
if (!transientWindowsError || retry >= 20) throw error;
retry += 1;
// Preserve atomic replacement: wait for short-lived readers instead of
// unlinking the destination and exposing missing or partial state.
await sleep(retry * 5);
}
}
}

export async function writeAtomically(
path: string,
contents: string,
): Promise<void> {
const temporary = join(
dirname(path),
`.flow-write-${process.pid}-${randomUUID()}.tmp`,
);
const handle = await open(temporary, "wx", 0o600);
try {
await handle.writeFile(contents, "utf8");
await handle.sync();
} catch (error) {
await handle.close();
await rm(temporary, { force: true });
throw error;
}
await handle.close();
try {
await renameReplacing(temporary, path);
await syncDirectory(dirname(path));
} catch (error) {
await rm(temporary, { force: true });
throw error;
}
}
Loading
Loading