From 56cc36a92119a2b0d608c493acf6a53d7201d772 Mon Sep 17 00:00:00 2001 From: Aizat Hawari Date: Mon, 28 Sep 2026 10:07:16 +0100 Subject: [PATCH 1/4] docs: settings page guide and plain-language docs audit - Add docs/web-ui-settings.md, which explains every section of the Settings page. docs/web-ui.md links to it. - Check the other guides against the code, fix what was wrong or missing, and rewrite them in plain British English. - ADR-010: save a pasted external API key to a private key file. - ADR-011: shared code never imports a host adapter. - Point ADR links at archived OpenSpec changes. --- CONTRIBUTING.md | 2 +- README.md | 28 ++- .../adr/007-edit-global-config-from-web-ui.md | 2 +- docs/adr/008-session-first-web-import.md | 4 +- ...9-default-on-backfill-and-login-web-app.md | 2 +- .../010-private-key-file-for-external-api.md | 36 +++ .../011-shared-code-never-imports-adapters.md | 47 ++++ docs/adr/ADR_README.md | 2 + docs/ci.md | 184 +++++++------- docs/cli.md | 192 +++++++++------ docs/configuration.md | 166 +++++++++---- docs/developers.md | 63 ++--- docs/moving-projects.md | 144 ++++++----- docs/omms-migration.md | 232 ++++++++++-------- docs/opencode-adapter.md | 154 ++++++------ docs/opencode-history-import.md | 186 ++++++++------ docs/pi-adapter.md | 122 ++++----- docs/pi-history-import.md | 192 ++++++++------- docs/shared-core.md | 203 +++++++++------ docs/upgrading.md | 98 ++++++-- docs/using-memory.md | 95 ++++--- docs/web-ui-settings.md | 225 +++++++++++++++++ docs/web-ui.md | 104 ++++---- 23 files changed, 1601 insertions(+), 882 deletions(-) create mode 100644 docs/adr/010-private-key-file-for-external-api.md create mode 100644 docs/adr/011-shared-code-never-imports-adapters.md create mode 100644 docs/web-ui-settings.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 37c4c376..78abb9bd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -18,7 +18,7 @@ agent. Bug reports, fixes, documentation, and new features are all welcome. ## Set up -You need [Bun](https://bun.sh) and Node.js 24. The package itself supports +You need [Bun](https://bun.sh) and Node.js 24. The published package supports Node.js 22.14 or later. ```bash diff --git a/README.md b/README.md index d1a1b493..8815b456 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# OMMS — Opinionated Modular Memory System +# OMMS: Opinionated Modular Memory System [![npm version](https://img.shields.io/npm/v/om-memory-system.svg)](https://www.npmjs.com/package/om-memory-system) [![npm downloads](https://img.shields.io/npm/dm/om-memory-system.svg)](https://www.npmjs.com/package/om-memory-system) @@ -26,7 +26,8 @@ the other. Everything is stored locally on your machine. OpenCode v1. - **Learns your preferences.** A user profile of your habits builds up over time and follows you across projects. -- **Imports your past sessions.** Pi and OpenCode import older history after startup. A command provides a preview and manual control. +- **Imports your past sessions.** Pi and OpenCode import older history after + startup. A command gives you a preview and manual control. - **Lets you look and edit.** A local web page shows every memory and your profile. - **Keeps private things private.** Text inside `` tags is never @@ -74,6 +75,14 @@ pi install npm:om-memory-system Restart Pi. You can install OMMS in both agents; they share the same memory. +**Terminal command (optional, recommended).** A global install lets the login +web app and the `om-memory-system` terminal commands run without `npx`: + +```bash +npm i -g om-memory-system # or: bun add -g om-memory-system +om-memory-system --version +``` + ### 2. Choose which model writes memories (optional) With no settings, OMMS uses the model of the session you are working in. To @@ -91,18 +100,22 @@ config file yet, OMMS creates this one with comments on first start. } ``` -You can also use any OpenAI-compatible or Anthropic API with your own key. -See [Configuration](docs/configuration.md#choosing-the-model). +You can also use any OpenAI-compatible or Anthropic API with your own key, set +up on the Settings page's **External API** card, and choose `"external"` as a +host's model. See [Configuration](docs/configuration.md#choosing-the-model). ### 3. Check it works Work normally for a few turns, then open `http://127.0.0.1:4747` in your -browser. OpenCode or the login web app serves this page. New memories appear on the timeline. -You can also ask the agent: "search memory for what we changed today". +browser. OpenCode or the login web app serves this page. New memories appear +on the timeline. You can also ask the agent: "search memory for what we changed today". ## Import your past history -Older sessions import automatically after a host starts. This makes model calls. To opt out, set `"autoBackfill": false` in `~/.config/omms/omms.jsonc` before starting. For a manual import or custom source, preview inside the agent: +Older sessions import automatically after a host starts. This makes model +calls. To turn it off, set `"autoBackfill": false` in +`~/.config/omms/omms.jsonc` before you start the agent. For a manual import or +a custom source, preview it inside the agent: ```text /memory-import-opencode-history --dry-run @@ -134,6 +147,7 @@ backup first. See [Updating and upgrading](docs/upgrading.md) and | [Using memory day to day](docs/using-memory.md) | How capture and recall work, the `memory` tool, the user profile | | [Configuration](docs/configuration.md) | Settings, choosing the model, embeddings, troubleshooting | | [Web UI](docs/web-ui.md) | The memory explorer, opening it on a network safely | +| [Settings page](docs/web-ui-settings.md) | Every card and control on the web Settings page | | [Moving projects](docs/moving-projects.md) | Nested repositories, moved folders, backup and restore | | [Updating and upgrading](docs/upgrading.md) | Updates, pinning a version, older stores | | [OpenCode adapter](docs/opencode-adapter.md) | How the OpenCode plugin hooks in | diff --git a/docs/adr/007-edit-global-config-from-web-ui.md b/docs/adr/007-edit-global-config-from-web-ui.md index 293521ca..16188b34 100644 --- a/docs/adr/007-edit-global-config-from-web-ui.md +++ b/docs/adr/007-edit-global-config-from-web-ui.md @@ -44,4 +44,4 @@ If OMMS reads only the legacy `opencode-mem.jsonc`, the first page save copies t - [Settings page](../web-ui.md#settings-page) - [Config writer](../../src/services/global-config-writer.ts) -- [OpenSpec design](../../openspec/changes/web-settings/design.md) +- [OpenSpec design](../../openspec/changes/archive/2026-09-27-web-settings/design.md) diff --git a/docs/adr/008-session-first-web-import.md b/docs/adr/008-session-first-web-import.md index 1658d6cc..88dd101b 100644 --- a/docs/adr/008-session-first-web-import.md +++ b/docs/adr/008-session-first-web-import.md @@ -56,8 +56,8 @@ The Settings page first offered the history importer as a form of CLI flags. Use ## References -- [Importing from the page](../web-ui.md#importing-from-the-page) +- [Importing from the page](../web-ui-settings.md#import-and-backfill) - [ADR-005](./005-history-import-surfaces-and-model.md), [ADR-007](./007-edit-global-config-from-web-ui.md) - [TDR-007](../tdr/007-directory-maps-take-precedence.md), [TDR-008](../tdr/008-shared-async-opencode-snapshot.md) - `src/importer/import-sessions.ts`, `src/importer/import-sources.ts`, `src/importer/import-readiness.ts`, `src/importer/web-import-jobs.ts` -- [OpenSpec design](../../openspec/changes/web-settings/design.md) +- [OpenSpec design](../../openspec/changes/archive/2026-09-27-web-settings/design.md) diff --git a/docs/adr/009-default-on-backfill-and-login-web-app.md b/docs/adr/009-default-on-backfill-and-login-web-app.md index 5f595de3..24f8407e 100644 --- a/docs/adr/009-default-on-backfill-and-login-web-app.md +++ b/docs/adr/009-default-on-backfill-and-login-web-app.md @@ -43,7 +43,7 @@ Enable a per-user login item for the standalone web app by default. The item sta ## References -- [Change design](../../openspec/changes/auto-backfill-and-web-autostart/design.md) +- [Change design](../../openspec/changes/archive/2026-09-28-auto-backfill-and-web-autostart/design.md) - [Configuration](../configuration.md#automatic-history-import-and-login-web-app) - `src/importer/auto-backfill.ts` - `src/services/web-autostart.ts` diff --git a/docs/adr/010-private-key-file-for-external-api.md b/docs/adr/010-private-key-file-for-external-api.md new file mode 100644 index 00000000..0e8541c2 --- /dev/null +++ b/docs/adr/010-private-key-file-for-external-api.md @@ -0,0 +1,36 @@ +# ADR-010: Save a pasted external API key to a private key file + +**Date:** 2026-09-28 +**Status:** Proposed +**Deciders:** OMMS maintainers + +## Context + +The Settings page gains an External API card so that users can set up an external model (for example a Z.ai GLM endpoint) without editing `omms.jsonc` by hand, and choose it as a host's capture or backfill model with the value `external`. `memoryApiKey` already accepts a literal value, `env://NAME`, or `file://path`. The login web app is started by launchd, systemd, or the Windows Startup folder, which do not load a shell profile, so an `env://` variable set only in `~/.zshrc` does not resolve there. Users still need a way to give the page a key that works in every OMMS process, without the key ending up in the config file, the log, or an API response. + +## Decision + +The card offers three key sources: an environment variable name (saved as `env://NAME`), the path of an existing key file (saved as `file://path`), and a pasted key. A pasted key is written by the web server to `~/.config/omms/secrets/.key`. The folder is created with mode `700` and the file with mode `600` on macOS and Linux; on Windows both get a user-only access list, using the same code that protects capture traces (moved to `src/services/private-path.ts`). The config then gets `file://` with that path. An existing key file is replaced only after the user confirms. The page never writes a literal key to the config, never logs the request body, and never returns the key. Saving a pasted key is refused on a non-loopback bind without Basic Auth. + +An `env://` or `file://` key that does not resolve in a process is treated as not set there, so the card and readiness can report "does not resolve in the web app" instead of the config failing to load. + +## Alternatives considered + +- **OS keychain.** Needs native code or helper tools on three platforms, and every OMMS process (hosts, CLI, login web app) would need to read it. Rejected for now. +- **Only `env://`.** Does not work in the login web app, which is the main place a user without an open host would configure and run a backfill. +- **Store the literal key in `omms.jsonc`.** The file is often shared or synced, and the page would then write secrets into it. Rejected. + +## Consequences + +### Positive + +- One key source works in every OMMS process, including the login web app. +- The config file and every response hold only a reference. + +### Negative + +- A key lands on disk in plain text, with the same trust model as the existing `file://` support. File permissions are the only protection. + +### Neutral + +- Older OMMS versions ignore the key file and read the `file://` reference as before. diff --git a/docs/adr/011-shared-code-never-imports-adapters.md b/docs/adr/011-shared-code-never-imports-adapters.md new file mode 100644 index 00000000..ba88c79f --- /dev/null +++ b/docs/adr/011-shared-code-never-imports-adapters.md @@ -0,0 +1,47 @@ +# ADR-011: Shared code never imports a host adapter + +**Date:** 2026-09-28 +**Status:** Proposed +**Deciders:** OMMS maintainers + +## Context + +OMMS stands for Opinionated Modular Memory System. One shared engine runs behind two hosts, OpenCode and Pi, and each host has a thin adapter in `src/adapters//`. + +Some shared code had started to reach into adapter code: + +- `src/importer/profile-import.ts` imported `isInternalPrompt` from the OpenCode adapter. +- `src/importer/importer.ts` and `src/importer/session-loader.ts` imported Pi's conversation parser from the Pi adapter. + +Each import worked, but it tied shared code to one host. A new host, or a change inside one adapter, could then break the other host or the shared importer. The boundary tests covered `src/core/` and `src/services/`, not `src/importer/`, so nothing caught it. + +## Decision + +Dependencies point one way. Adapters may import shared code. Shared code (`src/core/`, `src/services/`, `src/importer/`, `src/types/`) never imports an adapter. + +- Code that knows a host's history format is shared, because the importer, the CLI, and the web app all read history without that host running. It lives in `src/importer/`: `opencode-reader.ts` for OpenCode, and `pi-conversation.ts` with `session-loader.ts` for Pi. The Pi adapter imports `pi-conversation.ts` from there for live capture. +- Code that is the same on every host lives in `src/core/`. `src/core/internal-prompt.ts` recognises omms's own summary and profile prompts. +- Host SDKs load only through dynamic `import()`, only when that host's code runs. +- `tests/pi-adapter-boundary.test.ts` fails when any file in `src/core/`, `src/services/`, `src/types/`, or `src/importer/` refers to `adapters/pi` or `adapters/opencode`. + +Adding a host means adding `src/adapters//` and, when it has importable history, a reader in `src/importer/`. The other hosts do not change. + +## Consequences + +### Positive + +- Each adapter can change, or be removed, without breaking shared code or the other host. +- A test enforces the rule, so it cannot quietly erode again. + +### Negative + +- Host-format readers sit in the importer rather than beside their adapter, so a host's code is in two places. + +### Neutral + +- `src/services/` still holds OpenCode's own model code (`opencode-provider.ts`, `opencode-sdk-client.ts`, `profile-llm-client.ts`, `user-memory-learning.ts`), which uses OpenCode SDK types. Moving it into the OpenCode adapter is planned as the OpenSpec change `move-opencode-model-code-to-adapter`. + +## Alternatives considered + +- **Keep host readers in each adapter and let the importer import them.** This is the dependency this ADR removes. +- **A registry that adapters fill at start-up.** The CLI and the login web app read history with no host running, so nothing would register the readers. diff --git a/docs/adr/ADR_README.md b/docs/adr/ADR_README.md index 2ffb2cb5..1c1e289a 100644 --- a/docs/adr/ADR_README.md +++ b/docs/adr/ADR_README.md @@ -13,3 +13,5 @@ Local decision records for OMMS maintainers. | [007](./007-edit-global-config-from-web-ui.md) | Edit the global config from the web UI | 2026-09-27 | Proposed | | [008](./008-session-first-web-import.md) | Session-first imports from the web UI with pinned selections | 2026-09-27 | Proposed | | [009](./009-default-on-backfill-and-login-web-app.md) | Default-on history backfill and login web app | 2026-09-27 | Proposed | +| [010](./010-private-key-file-for-external-api.md) | Save a pasted external API key to a private key file | 2026-09-28 | Proposed | +| [011](./011-shared-code-never-imports-adapters.md) | Shared code never imports a host adapter | 2026-09-28 | Proposed | diff --git a/docs/ci.md b/docs/ci.md index 30107345..62f94a01 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1,10 +1,13 @@ # Continuous Integration -OMMS validates changes locally on macOS first. GitHub Actions then runs -quality and test checks on every pull request, a native embedding matrix on -pull requests that touch native paths, and a full platform matrix before each -release. The repository is public, so hosted runners cost nothing. The only -limit is job concurrency: 20 jobs in total, 5 of them macOS. +OMMS checks changes locally on macOS first. GitHub Actions then runs: + +- quality and test checks on every pull request +- a native embedding matrix on pull requests that touch native paths +- a full platform matrix before each release + +The repository is public, so hosted runners are free. The only limit is job +concurrency: 20 jobs in total, 5 of them macOS. ## Where each check runs @@ -29,9 +32,9 @@ bun install --frozen-lockfile (cd web && bun install --frozen-lockfile) ``` -You need Bun and Node 24. Node ships with npm, which the package smoke tests -use. The default embedding model downloads once from Hugging Face and is -cached afterwards. +- You need Bun and Node 24. The package supports Node 22.14 or later. +- Node includes npm, which the package smoke tests use. +- The default embedding model downloads once from Hugging Face. After that it comes from the cache. ## Local commands @@ -43,36 +46,40 @@ cached afterwards. | `bun run check:package` | Published-package shape: entry points and web UI present (`verify:package`), `publint`, and `attw`. Needs a build. | | `bun run test` | Whole suite in one Bun process. Not reliable for gating. | -`ci:local` runs tests through `scripts/run-tests-isolated.sh`. That script -starts one Bun process per test file. The suite shares module and storage -state across files, so a single-process run fails non-deterministically -depending on file order. One process per file is deterministic. +`ci:local` (`scripts/local-ci.sh`) runs tests through `scripts/run-tests-isolated.sh`: + +- The script starts one Bun process for each test file. +- The suite shares module and storage state across files. In one process, results change with file order. +- One process for each file gives the same result every time. +- Do not use `bun test` for the whole suite. About 48 tests fail from shared module state. Those failures are not regressions. -Tests never write to the real `~/.omms`. `.env.test`, which Bun loads for -every test process and its children, points `OMMS_LOG_FILE` (and so the -traces directory) at a temp path and turns off the one-time migrations. The -isolated runner also gives each full run its own log directory. +Tests never write to the real `~/.omms`. Bun loads `.env.test` for every test process and its children. It: + +- points `OMMS_LOG_FILE` (and so the traces directory) at a temporary path +- turns off the one-time migrations (`OMMS_SKIP_LEGACY_MIGRATION`, `OMMS_SKIP_TAG_PREFIX_MIGRATION`) +- turns off automatic backfill (`OMMS_DISABLE_AUTO_BACKFILL`) and web login item changes (`OMMS_DISABLE_WEB_AUTOSTART`) + +The isolated runner also gives each full run its own log directory. ## Git hooks Husky installs two hooks: - **pre-commit**: `bun run typecheck && bunx lint-staged`. -- **pre-push**: `bun run check`. Fast and deterministic, about 11 seconds. +- **pre-push**: `bun run check`. It is fast and stable, about 11 seconds. -The full suite is deliberately outside the pre-push hook. Run -`bun run ci:local` before merging. +The full suite is not in the pre-push hook on purpose. Run `bun run ci:local` +before a push to a pull request and before a merge. ## Known test caveats -- `tests/plugin-bundle-boundary.test.ts` bundles `dist/` entries through the - `bun build` CLI in a child process. Bun 1.3.14 resolves in-process - `Bun.build` imports against the test file's directory when the file lives - under `tests/`, which breaks every relative import in `dist/index.js`. The - comment in the test file records this. Revisit after a Bun upgrade. -- Tests depend on a built `dist/`. `ci:local` builds before testing. If you run - a single test file without building first, build first: - `bun run build && bun test tests/.test.ts`. +- `tests/plugin-bundle-boundary.test.ts` bundles `dist/` entries with the + `bun build` CLI in a child process. In Bun 1.3.14, in-process `Bun.build` + resolves imports against the test file's folder. That breaks every relative + import in `dist/index.js`. A comment in the test file records this. Check it + again after a Bun upgrade. +- Some tests import `dist/`. `ci:local` builds before it tests. For one test + file, build first: `bun run build && bun test tests/.test.ts`. ## GitHub workflows @@ -88,14 +95,13 @@ Runs on every pull request and on every push to `main`. Three jobs: `scripts/run-tests-isolated.sh`. Skipped when a pull request changes only Markdown files or files under `docs/`. -A skipped `test` job still satisfies the required status check, so docs-only -pull requests can merge. Do not add `paths-ignore` to this workflow: if it does -not start, the required checks never report and the pull request stays -blocked. If the `changes` job fails, `test` runs anyway. +- A skipped `test` job still passes the required status check. So docs-only pull requests can merge. +- Do not add `paths-ignore` to this workflow. If it does not start, the required checks never report and the pull request stays blocked. +- If the `changes` job fails, `test` runs anyway. -Quality is the baseline gate for every pull request, including those that -skip the local hooks, such as Dependabot updates. Pull requests that touch -native paths also run Embedding Backend Verification. +Quality is the minimum gate for every pull request. This includes pull +requests that skip the local hooks, such as Dependabot updates. Pull requests +that touch native paths also run Embedding Backend Verification. ### Embedding Backend Verification (automatic on native changes, manual on demand) @@ -103,20 +109,19 @@ Runs when a pull request touches `package.json`, `bun.lock`, `bunfig.toml`, `.npmrc`, `src/services/embedding.ts`, `src/services/onnxruntime-resolve.ts`, `scripts/verify-embedding-backend.mjs`, `scripts/verify-nested-onnxruntime-fixture.mjs`, -`scripts/fixtures/compiled-host-entry.mjs`, or this workflow file. It can -also be dispatched at any time. - -onnxruntime-node and sharp ship a separate native binary for each platform, so -the `verify` job runs on `macos-15`, `macos-15-intel`, `windows-latest`, and -`ubuntu-latest`. `macos-15` is the supported floor; the Quality `test` job -already covers the newest macOS. Each job installs without lifecycle scripts and produces -real embeddings under Bun and Node 24. +`scripts/fixtures/compiled-host-entry.mjs`, or this workflow file. You can +also start it by hand at any time. -The `nested-intel-regression` job reproduces the OpenCode nested install on -Intel macOS with Bun 1.3.14 and Node 22. The compiled host must run inference -and exit 0 without a SIGILL (#210, #225). +- onnxruntime-node and sharp ship a separate native binary for each platform. + So the `verify` job runs on `macos-15`, `macos-15-intel`, `windows-latest`, + and `ubuntu-latest`. +- `macos-15` is the oldest supported macOS. The Quality `test` job already covers the newest macOS. +- Each job installs without lifecycle scripts and makes real embeddings under Bun and Node 24. +- The `nested-intel-regression` job copies the OpenCode nested install on + Intel macOS with Bun 1.3.14 and Node 22. The compiled host must run + inference and exit 0 without a SIGILL (#210, #225). -Dispatch it manually for any other native or toolchain change: +Start it by hand for any other native or toolchain change: ```bash gh workflow run "Embedding Backend Verification" --ref main @@ -125,22 +130,24 @@ gh workflow run "Embedding Backend Verification" --ref main ### Platform Package Smoke (release, weekly, manual) Runs on `macos-15`, `macos-26`, `macos-15-intel`, `macos-26-intel`, -`windows-latest`, and `ubuntu-latest`. Each job installs dependencies, runs -the full local gate, packs the npm tarball, installs it into a scratch -project, and runs the native dependency, libSQL vector, and package smoke -scripts. +`windows-latest`, and `ubuntu-latest`. Each job: + +1. Installs dependencies. +2. Runs the full local gate. +3. Packs the npm tarball and installs it into a scratch project. +4. Runs the native dependency, libSQL vector, and package smoke scripts. It runs: - Before every release. The Release workflow calls it and waits for it. -- Every Monday at 06:00 UTC, to catch runner image and upstream drift. +- Every Monday at 06:00 UTC, to find changes in runner images and upstream packages. - On demand, after any packaging change: ```bash gh workflow run "Platform Package Smoke" --ref main ``` -It stays off pull requests so its four macOS jobs do not queue behind the +It does not run on pull requests. Its four macOS jobs would queue behind the 5-job macOS limit. ### Release (push to `main`) @@ -153,23 +160,25 @@ Runs on every push to `main` once the repository variable version, and updates `CHANGELOG.md`. It signs in as the private `omms-release` GitHub App, so its pull requests run the Quality checks. Merging that pull request tags `vX.Y.Z` and creates the GitHub Release. -- `smoke` runs only for a release: it calls Platform Package Smoke on the +- `smoke` runs only for a release. It calls Platform Package Smoke on the release commit. - `publish` runs only after `smoke` passes. It builds, verifies the package contents, checks the version, and runs `npm stage publish` with no token (npm trusted publishing). npm holds the version until the maintainer approves it, and the job adds the approval steps to the GitHub Release. -Publishing happens in this run, not on a tag-push workflow, because tags -created by release-please do not start other workflows. +Publishing happens in this run, not in a tag-push workflow. Tags that +release-please creates do not start other workflows. ### Publish next (after Quality on `main`) Runs when Quality succeeds for a push to `main`, once the repository variable `NPM_NEXT_ENABLED` is `true`. It builds that commit and publishes it as -`X.(Y+1).0-next.` under the npm `next` tag, without approval, so the -maintainer can try it with `om-memory-system@next`. It skips release commits (those that -change `.release-please-manifest.json`) and fails if `latest` moves. +`X.(Y+1).0-next.` under the npm `next` tag, without approval. The +maintainer can then try it with `om-memory-system@next`. + +- It skips release commits (commits that change `.release-please-manifest.json`). +- It fails if `latest` moves. ## Release runbook @@ -180,26 +189,28 @@ Versions come from commit messages. Use `feat:` (minor), `fix:` (patch), 1. Merge work into `main` as usual. Each merge also appears as `om-memory-system@next`. 2. When you want to ship, merge the open release pull request. 3. Wait for the Release workflow: six-platform smoke, then `publish`. -4. Approve the staged version with 2FA, in the Staged tab at - or with `npm stage list om-memory-system`, then - `npm stage approve `. To try it first, run - `npm stage download ` and install the tarball. -5. Users on an unpinned install are told about the update. - -If the smoke gate fails, nothing is staged, but the tag and GitHub Release -already exist. Fix forward with a `fix:` commit, and release-please proposes -the next patch. Edit the failed GitHub Release to say it was not published to -npm. To reject a staged version instead of approving it, run -`npm stage reject `. +4. Optional: to try the staged version, run `npm stage download ` and install the tarball. +5. Approve the staged version with 2FA (two-factor authentication). Use the + Staged tab at , or run + `npm stage list om-memory-system`, then `npm stage approve `. +6. Users on an unpinned install get an update notice. + +To reject a staged version, run `npm stage reject `. + +If the smoke gate fails, nothing is staged. The tag and GitHub Release +already exist. To fix it: + +1. Push a `fix:` commit. release-please then proposes the next patch. +2. Edit the failed GitHub Release to say it was not published to npm. ## First publish (one time) -The npm package is `om-memory-system`: npm rejects the plain name `omms` as too similar -to `ms` and `os`. The product, plugin id, config folder and data folder are -still `omms`. +The npm package is `om-memory-system`. npm rejects the name `omms` because it +is too similar to `ms` and `os`. The product, plugin id, config folder and data +folder are still `omms`. -npm only allows a trusted publisher on a package that already exists, so the -first version is published by hand. Do these steps in order after merging the +npm allows a trusted publisher only on a package that already exists. So you +publish the first version by hand. Do these steps in order after you merge the release-publishing change. 1. GitHub settings: @@ -223,7 +234,7 @@ release-publishing change. npm publish --access public ``` -3. Tag that commit and create its GitHub Release, so release-please counts +3. Tag that commit and create its GitHub Release. release-please then counts later commits from it: ```bash @@ -241,7 +252,7 @@ release-publishing change. ``` The first is **stage-only**, so releases wait for approval. The same - settings are under npmjs.com → om-memory-system → Settings → Trusted publishing. + settings are in npmjs.com, om-memory-system, Settings, Trusted publishing. 5. Set the repository variables `RELEASE_PLEASE_ENABLED=true` and `NPM_NEXT_ENABLED=true`. @@ -251,23 +262,24 @@ release-publishing change. repository secret. If `publish` fails with `ENEEDAUTH`, the workflow file name, environment, or -repository on npmjs.com does not match exactly. Nothing is published; fix the -setting and re-run the job. +repository on npmjs.com does not match exactly. Nothing is published. Fix the +setting and run the job again. ## Platform scope -Supported platforms: macOS 15 and above on Apple Silicon and Intel, Windows, +Supported platforms: macOS 15 and later on Apple Silicon and Intel, Windows, and Linux. OpenCode users install the plugin on all of them. -Pull requests get the cheapest useful coverage: one Linux quality job and one -macOS test job. The native matrix runs only when native paths change. The full -six-platform matrix runs before release and weekly. +- Pull requests get the smallest useful coverage: one Linux quality job and one macOS test job. +- The native matrix runs only when native paths change. +- The full six-platform matrix runs before a release and every week. + +Keep the `onnxruntime-node@1.20.1` pin: -The `onnxruntime-node@1.20.1` pin stays in place: newer releases can SIGILL on -macOS process exit (#225), and OpenCode's nested installs ignore package -overrides (#184). +- Newer releases can SIGILL when a macOS process exits (#225). +- OpenCode's nested installs ignore package overrides (#184). -Costs: the repository is public, so hosted runners are free, macOS included. +Job counts for each event (hosted runners are free, macOS included): | Event | Ubuntu | Windows | macOS | | ------------------------ | ------ | ------- | ----- | diff --git a/docs/cli.md b/docs/cli.md index 21b32d0a..de32124f 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -1,51 +1,84 @@ # omms CLI -The npm package ships one command, `om-memory-system`. It imports past OpenCode and Pi history, starts the web app, and manages its login item without an agent session open. +The npm package ships one terminal command, `om-memory-system`. Use it to import past OpenCode and Pi history, run the web app, and manage the web app's login item. You do not need an agent session open. -```text -om-memory-system import-opencode-history [options] -om-memory-system import-pi-history [options] -om-memory-system web [install|uninstall|status] +## Command reference + +| Command | What it does | +| -------------------------------------------------- | --------------------------------------------------------------------- | +| `om-memory-system import-opencode-history [flags]` | Import OpenCode history with the external API. | +| `om-memory-system import-pi-history [flags]` | Import Pi history with the external API. | +| `om-memory-system web` | Start the web app in the foreground. | +| `om-memory-system web install` | Set `webServerAutoStart` to `true` and install the login item. | +| `om-memory-system web uninstall` | Set `webServerAutoStart` to `false` and remove the login item. | +| `om-memory-system web status` | Print the setting, the login item state, and whether a web app is up. | +| `om-memory-system --version`, `-v` | Print the installed version and exit with code `0`. | +| `om-memory-system --help`, `-h`, or no arguments | Print the command list. | +| `om-memory-system --help` | Print the flags for that import command. | + +Slash commands inside a session: + +| Host | Command | What it does | +| -------- | ----------------------------------------- | ------------------------------------------------- | +| OpenCode | `/memory-import-opencode-history [flags]` | Import OpenCode history with the session's model. | +| Pi | `/memory-import-pi-history [flags]` | Import Pi history with the session's model. | + +- Both slash commands take the [import flags](#import-options), except `--provider`, `--api-url`, and `--api-key-env`. +- `--help` on a slash command shows its flags. +- You can also run imports, backfills, and directory maps from the web Settings page. See [Web UI settings](web-ui-settings.md). + +## Global install (optional, recommended) + +`npx om-memory-system` works without an install. A global install has two benefits: + +- The login item and the terminal commands run without `npx`. +- One known version stays on your `PATH`. + +```bash +npm i -g om-memory-system # or: bun add -g om-memory-system +om-memory-system --version ``` -The two import commands take the same options and run the same importer as the in-session -slash commands `/memory-import-opencode-history` and `/memory-import-pi-history`. -The difference is the model: +Upgrade with `npm i -g om-memory-system@latest` or `bun add -g om-memory-system@latest`. The Settings page shows the running version next to the global command's version. It warns when they differ. + +## Which model an import uses + +The terminal import commands and the slash commands run the same importer with the same options. Only the model differs. | Where you run it | Model used | API key needed | | ---------------- | --------------------------------------------------------------------------------------------------------------- | -------------- | | Slash command | This session's model, or `--model provider/id` from the host's signed-in models | No | -| CLI | The saved external API (`memoryProvider`, `memoryModel`, `memoryApiUrl`, `memoryApiKey`), or flags for this run | Yes | +| Terminal | The saved external API (`memoryProvider`, `memoryModel`, `memoryApiUrl`, `memoryApiKey`), or flags for this run | Yes | +| Settings page | See [Web UI settings](web-ui-settings.md) | Depends | -Use the slash command when you can: it uses the model you are already signed -in to. Use the CLI for scripts, or when no session is open. You can also run the same importer from the [web Settings page](web-ui.md#settings-page) while OpenCode serves it. Preview there before a real import. The page uses a connected OpenCode model or the saved external API, with the same ledger and report. +- Use the slash command when you can. It uses the model you are already signed in to. +- Use the terminal for scripts, or when no session is open. +- An import never saves its model choice to the configuration. ## Requirements -- Node.js 22.14 or later (the OpenCode reader uses `node:sqlite`). -- `import-pi-history` loads sessions through `@earendil-works/pi-coding-agent`, - a peer dependency that npm installs with the package. -- The same omms configuration and store as the plugins. The CLI reads the - global config and the project config of the directory you run it from. +- Node.js 22.14 or later. The OpenCode reader uses `node:sqlite`. +- `import-pi-history` loads sessions through `@earendil-works/pi-coding-agent`. This is a peer dependency that npm installs with the package. +- The same omms configuration and store as the plugins. The CLI reads the global config and the project config of the directory you run it from. ## Quick start -Preview first. A dry run makes no model calls and writes no store files: +1. Go to your project folder. +2. Preview first. A dry run makes no model calls and writes no store files. -```bash -cd ~/code/my-project -npx om-memory-system import-opencode-history --dry-run -npx om-memory-system import-pi-history --dry-run -``` + ```bash + cd ~/code/my-project + npx om-memory-system import-opencode-history --dry-run + npx om-memory-system import-pi-history --dry-run + ``` -Import with the saved external model: +3. Import with the saved external model. -```bash -npx om-memory-system import-opencode-history -``` + ```bash + npx om-memory-system import-opencode-history + ``` -Or pick a different external model for this run only. Keep the key in an -environment variable, never in the command line: +To use a different external model for one run, pass its settings. Keep the key in an environment variable, never on the command line: ```bash export OMMS_IMPORT_KEY='your-key' @@ -55,35 +88,34 @@ npx om-memory-system import-pi-history --scope all-projects \ ``` Before a real import starts, the CLI prints `Import model: provider/model`. -Nothing is saved to the configuration. - -## Options - -| Flag | Effect | -| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `--dry-run` | Preview counts with no model calls or writes. | -| `--provider ` | External provider type: `openai-chat`, `openai-responses`, `anthropic`, `minimax` or `orcarouter`. Default: saved `memoryProvider`. | -| `--model ` | External model id. Default: saved `memoryModel`. | -| `--api-url ` | Endpoint. Required when `--provider` differs from the saved provider (except `orcarouter`). | -| `--api-key-env ` | Read the API key from this environment variable. Default: saved `memoryApiKey`. | -| `--scope ` | `current-project` (default) or `all-projects`. | -| `--project ` | Project for `current-project` scope. Default: the working directory. | -| `--session ` | Import one session (Pi also accepts a session file path). | -| `--since `, `--until ` | Inclusive date range. A bare date in `--until` covers that whole day. | -| `--max-sessions ` | Read at most this many sessions, oldest first. | -| `--map =` | Map a recorded directory to another one. A map wins over the recorded directory on both hosts. Repeat as needed. | -| `--db ` | OpenCode only: database. Default `~/.local/share/opencode/opencode.db`. | -| `--root ` | Pi only: a session folder or one `.jsonl` session file. Default `~/.pi/agent/sessions`. | -| `--skip-memories` | Record profile prompts only. | -| `--skip-profile` | Import memories only. | -| `--profile-batch ` | Prompts per profile analysis batch. Default: 50. | -| `--force` | Reprocess memory work units that already finished. | -| `--help` | Show help for the command. | - -A value may follow its flag or use `--flag=value`. Dates accept ISO 8601 or -epoch milliseconds. `--provider`, `--api-url` and `--api-key-env` exist only -on the CLI; inside a session they are rejected, and `--model` takes -`provider/id` instead of a bare id. + +## Import options + +| Flag | Effect | +| ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--dry-run` | Preview counts. No model calls and no writes. | +| `--provider ` | Terminal only. External provider type: `openai-chat`, `openai-responses`, `anthropic`, `minimax`, `google-gemini`, or `orcarouter`. Default: saved `memoryProvider`. | +| `--model ` | Terminal: external model id. Default: saved `memoryModel`. Slash command: `provider/id` of a signed-in model. Default: the session's model. | +| `--api-url ` | Terminal only. Endpoint. Required when `--provider` differs from the saved provider, except for `orcarouter`. | +| `--api-key-env ` | Terminal only. Read the API key from this environment variable. Default: saved `memoryApiKey`. | +| `--scope ` | `current-project` (default) or `all-projects`. | +| `--project ` | Project for `current-project` scope. Default: the working directory. Cannot be used with `--scope all-projects`. | +| `--session ` | Import one session. Pi also accepts the session file path. | +| `--since `, `--until ` | Inclusive date range. A bare date in `--until` covers that whole day. `--since` must be before `--until`. | +| `--max-sessions ` | Read at most this many sessions, oldest first. Must be a positive whole number. | +| `--map =` | Map a recorded directory to another one for this run. Adds to the saved `importPathMaps` and wins for the same ``. Repeat as needed. | +| `--db ` | OpenCode only. The database. Default: `~/.local/share/opencode/opencode.db`. | +| `--root ` | Pi only. A session folder or one `.jsonl` session file. Default: `~/.pi/agent/sessions`. | +| `--skip-memories` | Record profile prompts only. | +| `--skip-profile` | Import memories only. | +| `--profile-batch ` | Prompts per profile analysis batch. Default: 50. Must be a positive whole number. | +| `--force` | Reprocess memory work units that already finished. | +| `--help`, `-h` | Show help for the command. | + +- A value may follow its flag or use `--flag=value`. +- Dates accept ISO 8601 or epoch milliseconds. +- Inside a session, `--provider`, `--api-url`, and `--api-key-env` are rejected. +- An unknown flag, or `--db` on Pi or `--root` on OpenCode, is an error. ## Output and exit codes @@ -98,35 +130,47 @@ OpenCode history import (dry-run) profile prompts: 2422 pending, 0 recorded, 0 already done; 0 batches, 0 remaining ``` -| Exit code | Meaning | -| --------- | ----------------------------------------------------------------------------------- | -| `0` | Finished, or help shown. | -| `1` | Bad options, missing model settings, a failed work unit, or a failed profile batch. | +| Exit code | Meaning | +| --------- | --------------------------------------------------------------------------------------------------------- | +| `0` | Finished, help shown, or version shown. | +| `1` | Bad options, missing model settings, a failed work unit, a failed profile batch, or a failed web command. | -Error messages never contain the API key: the key from `--api-key-env` and the -saved `memoryApiKey` are replaced with `[redacted]`. +Error messages never contain the API key. The key from `--api-key-env` and the saved `memoryApiKey` are replaced with `[redacted]`. ## Web app commands -`om-memory-system web` runs the web app in the foreground, without Pi or OpenCode. It needs `webServerEnabled: true` and the configured port must be free. Press Ctrl+C to stop it. If an OpenCode host already owns the port, OMMS leaves that owner running. +`om-memory-system web` runs the web app in the foreground, without Pi or OpenCode. + +- It needs `webServerEnabled: true`, and the configured port must be free. +- Press Ctrl+C to stop it. +- If an OpenCode host already owns the port, OMMS leaves that owner running. + +`om-memory-system web install` sets `webServerAutoStart` to `true` in the global config and registers the login item. + +- It needs `webServerEnabled: true`, an installed Node or Bun runtime, and a package path it can find. +- It exits with code `1` if the item is not installed. + +`om-memory-system web uninstall` sets `webServerAutoStart` to `false` and removes only OMMS's own item. + +`om-memory-system web status` prints JSON with the setting, the item state, and whether a web app answers. It changes nothing. + +Any other argument after `web` prints the usage and exits with code `1`. -`om-memory-system web install` enables `webServerAutoStart` in the global config and registers the login item. It requires `webServerEnabled: true`, an installed Node or Bun runtime, and a resolvable package path. `om-memory-system web uninstall` disables the setting and removes only OMMS's own item. `om-memory-system web status` reads the item state without changing it. Supported platforms are macOS, Linux with systemd user services, and Windows. For an unsupported platform or a missing runtime, run `om-memory-system web` manually. +Supported platforms are macOS, Linux with systemd user services, and Windows. On other platforms, or without a runtime, run `om-memory-system web` yourself. -The login item starts the standalone web app after sign-in. It shares the same data and settings as the hosts. See [Web UI](web-ui.md) for port ownership and authentication. +The login item starts the standalone web app after sign-in. It uses the same data and settings as the hosts. See [Web UI](web-ui.md) for port ownership and authentication. ## Safety -- The CLI only reads history. OpenCode's database and its `-wal`/`-shm` files - and Pi's session files stay unchanged. See - [opencode-history-import.md](opencode-history-import.md) for how the WAL is - read while OpenCode is open. -- Reruns are safe. A ledger in the store skips work that already finished, and - failed work stays retryable. Run one importer at a time against a store. -- There is no undo command. Back up `~/.omms/data` before a large import if - you may want to roll back. +- The CLI only reads history. OpenCode's database, its `-wal` and `-shm` files, and Pi's session files stay unchanged. See [opencode-history-import.md](opencode-history-import.md) for how the WAL is read while OpenCode is open. +- Reruns are safe. A ledger in the store skips finished work. Failed work stays retryable. +- One import per host runs at a time. A real import (not `--dry-run`) takes the host's lock in the store. A CLI run, a slash command, a web import, and a backfill for the same host cannot overlap. The second one stops with "A Pi import is already running", or the OpenCode version of that message. +- Every real import records its progress in the store. The Settings page shows a CLI run with its percentage and time left. If the terminal closes, the page shows the run as stopped. Run the command again to continue from the ledger. +- There is no undo command. Back up `~/.omms/data` before a large import if you may want to roll back. ## See also - [OpenCode history import](opencode-history-import.md) - [Pi history import](pi-history-import.md) +- [Web UI settings](web-ui-settings.md) - [Configuration: Choosing the model](configuration.md#choosing-the-model), for how live capture chooses its model diff --git a/docs/configuration.md b/docs/configuration.md index e9b6cfc6..a99ce1cb 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -4,11 +4,12 @@ OMMS works with no configuration. This page covers the settings you may want to ## Where the settings live -Configure at `~/.config/omms/omms.jsonc`. While that file does not exist, omms still reads the legacy `~/.config/opencode/opencode-mem.jsonc` (it is never written), so existing installs keep working before you migrate settings. Per-project overrides go in `/.opencode/omms.jsonc` (the legacy `.opencode/opencode-mem.jsonc` is still read when no `omms.jsonc` exists): +- **Global file:** `~/.config/omms/omms.jsonc`. On Windows this is `%USERPROFILE%\.config\omms\omms.jsonc`, not AppData. +- **Legacy global file:** while the global file does not exist, OMMS still reads `~/.config/opencode/opencode-mem.jsonc`. It never writes to it. +- **Project file:** `/.opencode/omms.jsonc` overrides the global file for that project. OMMS still reads a legacy `.opencode/opencode-mem.jsonc` when no `omms.jsonc` exists. Some settings are [global only](#global-only-settings). +- **Default store:** `~/.omms/data` (`%USERPROFILE%\.omms\data` on Windows). A legacy `~/.opencode-mem/data` store moves to the new path at first start. -**Windows:** `%USERPROFILE%\.config\omms\omms.jsonc` (not AppData). Default storage resolves to `%USERPROFILE%\.omms\data` (the `~` form in the example below expands to your user home on Windows as well). A legacy `~/.opencode-mem/data` store migrates to the new path automatically on first start. - -The plugin creates a full commented template at this path on first startup (only when no config exists at all). The trimmed example below shows the most common settings: +On first start, if no config exists at all, the plugin creates a full commented template. This shorter example shows the most common settings: ```jsonc { @@ -36,8 +37,9 @@ The plugin creates a full commented template at this path on first startup (only "autoCaptureEnabled": true, "autoBackfill": true, - "piBackfillModel": "inherit", // or "provider/model" - "opencodeBackfillModel": "inherit", // or "provider/model" + "piBackfillModel": "inherit", // or "external", or "provider/model" + "opencodeBackfillModel": "inherit", // or "external", or "provider/model" + "importPathMaps": [{ "from": "~/code/app-feat-x", "to": "~/code/app" }], "autoCaptureLanguage": "auto", // Model for auto-capture and profile learning (see "Choosing the model"). @@ -77,31 +79,87 @@ The plugin creates a full commented template at this path on first startup (only ## Settings in the web UI -Open [Settings](web-ui.md#settings-page) in the login web app, in OpenCode, or with `om-memory-system web`. The page can change `opencodeProvider`, `opencodeModel`, `piProvider`, `piModel`, `autoBackfill`, `opencodeBackfillModel`, `piBackfillModel`, `webServerAutoStart`, `captureTrace`, `captureTraceRetentionDays`, and `captureAttemptRetentionDays` in the global file. It does not edit a project's config or any credential. `captureAttemptRetentionDays` defaults to 30; both retention fields require at least 1 day. - -Choosing **Session model** writes `inherit` to the host's model key. That choice takes priority over a configured external API. Choosing a manual model writes the selected host provider and model. The next capture or profile-learning run in OpenCode or Pi reloads changed config files; restart is not required. +Open the Settings page in the login web app, in OpenCode, or with `om-memory-system web`. [Web UI settings](web-ui-settings.md) explains each part of the page. -A legacy-only install copies its old config and comments to `~/.config/omms/omms.jsonc` on the first page save. OMMS reads the new file from then on. The old file stays unchanged. The page rejects a save if the file changed since it was loaded; review the refreshed values before saving again. +- The page writes only to the global file. It does not edit a project's config. +- It can change `opencodeProvider`, `opencodeModel`, `piProvider`, `piModel`, `autoBackfill`, `opencodeBackfillModel`, `piBackfillModel`, `importPathMaps`, `webServerAutoStart`, `captureTrace`, `captureTraceRetentionDays`, `captureAttemptRetentionDays`, `memoryProvider`, `memoryApiUrl`, `memoryModel`, and `memoryApiKey`. +- The only credential it changes is `memoryApiKey`. It accepts only an `env://` or `file://` reference and rejects a literal key. +- If you paste a key, the page saves it to a key file in `~/.config/omms/secrets/` and stores a `file://` reference to it. The folder and file are readable only by you. +- `captureAttemptRetentionDays` defaults to 30. Both retention fields need at least 1 day. +- Choosing **Session model** writes `inherit` to the host's model key. That choice takes priority over a configured external API. +- Choosing a manual model writes the selected host provider and model. +- OpenCode and Pi reload changed config files at the next capture or profile-learning run. You do not need to restart. +- On a legacy-only install, the first save copies the old config and its comments to `~/.config/omms/omms.jsonc`. OMMS reads the new file from then on. The old file stays unchanged. +- The page rejects a save if the file changed since the page loaded it. Check the refreshed values, then save again. ## Automatic history import and login web app -`autoBackfill` defaults to `true`. About 30 seconds after a Pi or OpenCode start, that host imports its own past chats in the background. It covers resolvable projects, records profile prompts, and resumes from the ledger after a restart. A fixed cutoff, saved at the first run for each host, limits the work to turns that existed then. Live capture handles newer turns. Imports skip exchanges already saved by live capture. Unresolved directories appear in the progress counts; use a manual import with `--map` to include them. +`autoBackfill` defaults to `true`. About 30 seconds after Pi or OpenCode starts, that host imports its own past chats in the background. + +- It covers projects whose directories resolve, and it records profile prompts. +- It resumes from the ledger after a restart. +- At the first run for each host, it saves a fixed cutoff. It only imports turns that existed then. Live capture handles newer turns. +- It skips exchanges that live capture already saved. +- Unresolved directories appear in the progress counts and in the Settings page's **Directory maps** list. To include them, save a map there or add it to `importPathMaps`. +- Backfill makes model calls. To avoid them, set `"autoBackfill": false` before you upgrade. If you turn it off during a run, the run stops after the current exchange. + +`importPathMaps` is a list of `{ "from": ..., "to": ... }` directory maps. + +- `~` is expanded. After that, both paths must be absolute. A bad entry is a config error. +- Automatic backfill, web imports, CLI imports, and slash-command imports all use it. +- A run's own `--map` adds to the list and wins for the same `from`. +- If the `to` directory does not exist, its sessions stay unresolved. + +On the Settings page you can **Run now**, **Pause**, and **Resume** each host's backfill. + +- A paused backfill does not start when the host starts. It waits until you resume it. +- Every real import records its progress in `import-ledger.db`. It stores numbers only, no conversation content. +- One import per host runs at a time, across the backfill, the page, the slash commands, and the CLI. + +`piBackfillModel` and `opencodeBackfillModel` choose the backfill model. They default to `"inherit"`. + +- `"inherit"` on Pi follows Pi's live-capture model rule. +- `"inherit"` on OpenCode uses the configured host model, then the saved external API, then OpenCode's configured default model. +- `"external"` sends that host's backfill to the external API. If the external API is not fully configured, the backfill stops and names the missing setting. With `"external"`, Run now also works in the login web app with no host open. +- A signed-in `provider/model` chooses another backfill model without changing live capture. +- Any other value is a config error. +- If no model is available, the backfill stops and records an error. + +`webServerAutoStart` defaults to `true`. When `webServerEnabled` is also true, OMMS registers a per-user login item for the web app. -`piBackfillModel` and `opencodeBackfillModel` default to `"inherit"`. Pi then follows its live-capture model rule. OpenCode uses its configured host model, the saved external API, or its configured default model. Set either key to a signed-in `provider/model` to choose another backfill model without changing live capture. A missing model stops the backfill and records an error. Backfill makes model calls; set `"autoBackfill": false` before upgrading if you want to avoid them. Turning it off during a run stops after the current exchange. +- On macOS it is a LaunchAgent. On Linux it is a systemd user unit. On Windows it is a Startup-folder entry. +- Each host start checks the item. If you turn either setting off, the next host start removes it. +- To apply a change at once, run `om-memory-system web install` or `om-memory-system web uninstall`. +- The item needs Node or Bun. It uses the same port and authentication settings as the OpenCode-hosted web app. -`webServerAutoStart` defaults to `true`. When `webServerEnabled` is also true, OMMS registers a per-user login item for the web app: a LaunchAgent on macOS, a systemd user unit on Linux, or a Startup-folder entry on Windows. A host start reconciles the item; turning either setting off removes it at the next host start. Use `om-memory-system web install` or `web uninstall` to apply the change immediately. The item needs Node or Bun and uses the same port and authentication settings as the OpenCode-hosted web app. +`autoBackfill` and `webServerAutoStart` must be `true` or `false`. -The four new settings and `webServerEnabled` are global-only. Values in a project's `.opencode/omms.jsonc` are ignored, so one project cannot turn off the shared web server for another project. See [Automatic import](web-ui.md#settings-page) for status and [CLI](cli.md#web-app-commands) for the login-item commands. +## Global-only settings + +Some settings are read only from the global file. + +- OMMS ignores these in a project's `.opencode/omms.jsonc`: `autoBackfill`, `piBackfillModel`, `opencodeBackfillModel`, `importPathMaps`, `webServerAutoStart`, `webServerEnabled`, `captureTraceRetentionDays`, `autoCleanupEnabled`, and `autoCleanupRetentionDays`. This stops one project from, for example, turning off the shared web server for another. +- A project config cannot turn `captureTrace` on. See [Capture traces](#capture-traces-opt-in). +- A project config that sets `embeddingApiUrl`, `embeddingApiKey`, `memoryProvider`, `memoryApiUrl`, or `memoryApiKey` is an error. Move those to the global file. + +See [Web UI settings](web-ui-settings.md) for backfill status and [CLI](cli.md#web-app-commands) for the login-item commands. ## Choosing the model -Auto-capture and profile learning run a background AI request to summarize technical work and learn your preferences. OpenCode and Pi choose that model by the same rule: +Auto-capture and profile learning send a background AI request. It summarises technical work and learns your preferences. OpenCode and Pi choose the model by the same rule: -1. **Host model:** `opencodeProvider` + `opencodeModel` in OpenCode, `piProvider` + `piModel` in Pi. Set the model to `"inherit"` to follow whatever model the session uses. -2. **External API:** if no host model is set, `memoryModel` + `memoryApiUrl` + `memoryApiKey` (below). +1. **Host model:** `opencodeProvider` and `opencodeModel` in OpenCode, `piProvider` and `piModel` in Pi. + - Set the model to `"inherit"` to follow the session's model. + - Set it to `"external"` to send every call to the external API. The provider value is then ignored. +2. **External API:** if no host model is set, `memoryModel`, `memoryApiUrl`, and `memoryApiKey` (below). 3. **Session model:** if neither is set, the session's own model. -If the host model fails and the external API is configured, the external API is used instead. A half-configured external API (for example a model without a key) disables auto-capture and reports the missing settings instead of switching silently. +How failures are handled: + +- If the host model fails and the external API is configured, OMMS uses the external API instead. +- With `"external"`, the external API is already the main call. A failure is not retried elsewhere. +- With `"external"` and an incomplete external API, auto-capture is off on that host. OMMS reports the missing settings. +- A half-configured external API (for example, a model without a key) also turns auto-capture off and reports the missing settings. OMMS does not switch models silently. ```jsonc "opencodeProvider": "openai", @@ -110,9 +168,16 @@ If the host model fails and the external API is configured, the external API is "piModel": "gpt-5.6-luna", ``` -Host models go through OpenCode's or Pi's own sign-in, so OpenCode or Pi owns the auth, token refresh, and provider routing, and no separate key is needed here. The OpenCode provider name must match an entry from `opencode providers list` and support structured JSON output; the Pi name must be in Pi's model list. +Host models use OpenCode's or Pi's own sign-in. The host handles the login, token refresh, and provider routing, so you need no separate key here. + +- The OpenCode provider name must match an entry from `opencode providers list`. The model must support structured JSON output. +- The Pi name must be in Pi's model list. -**Follow the session model:** `"inherit"` (or setting nothing at all) resolves to a concrete model at call time. In OpenCode, **auto-capture** records each prompt's model via the `chat.params` hook and reuses it. **Profile learning** and other structured-output paths are not tied to a single user message, so they use the most recent model in OpenCode's `model.json` recent list (preferring `opencodeProvider` when set). In Pi, `"inherit"` is the session's current model. +**Follow the session model:** `"inherit"`, or no setting at all, picks a concrete model at call time. + +- In OpenCode, **auto-capture** records each prompt's model through the `chat.params` hook and uses it again. +- In OpenCode, **profile learning** and other structured-output paths are not tied to one user message. They use the most recent model in OpenCode's `model.json` recent list. They prefer `opencodeProvider` when it is set. +- In Pi, `"inherit"` is the session's current model. **External API** (step 2, and the fallback when a host model fails): @@ -123,39 +188,50 @@ Host models go through OpenCode's or Pi's own sign-in, so OpenCode or Pi owns th "memoryApiKey": "sk-...", ``` -**API Key Formats:** +**API key formats:** ```jsonc "memoryApiKey": "sk-..." -"memoryApiKey": "file://~/.config/opencode/api-key.txt" +"memoryApiKey": "file://~/.config/omms/secrets/memory-api.key" "memoryApiKey": "env://OPENAI_API_KEY" ``` -Manual `memoryProvider` modes: - -- `openai-chat`: OpenAI Chat Completions compatible API with tool/function calling. This can work with compatible proxies such as LiteLLM only when the selected upstream model and proxy preserve tool calls. -- `openai-responses`: OpenAI Responses API with function-call output. -- `anthropic`: Anthropic Messages API with tool use. -- `minimax`: MiniMax Anthropic Messages-compatible endpoint. Set `memoryApiUrl` to the global endpoint (`https://api.minimax.io`) or the China endpoint (`https://api.minimaxi.com`); the `/anthropic/v1/messages` path and `x-api-key` header are applied automatically. MiniMax text models such as `MiniMax-M3` support the adaptive thinking modes used by this plugin via `memoryExtraParams`. -- `orcarouter`: OpenAI-compatible model gateway with namespaced model IDs. `memoryApiUrl` and `memoryModel` are optional — they default to `https://api.orcarouter.ai/v1` and `orcarouter/auto` (a routing alias that selects a capable model per request). If you set `memoryModel`, use a namespaced ID such as `openai/gpt-5.5` or `deepseek/deepseek-v4-flash`; OrcaRouter rejects bare model names. Example: - ```jsonc - "memoryProvider": "orcarouter", - "memoryApiKey": "", - ``` - [OrcaRouter](https://www.orcarouter.ai) also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes. +- A login web app started by the login item does not load your shell profile. An `env://` variable set only there does not resolve in it. The Settings page's External API card reports this. +- A `file://` key file works in every OMMS process. +- If an `env://` or `file://` key does not resolve, OMMS treats `memoryApiKey` as not set. It still loads the rest of the config. + +`memoryProvider` modes: + +- `openai-chat`: an OpenAI Chat Completions compatible API with tool (function) calling. It can work with compatible proxies such as LiteLLM, but only when the upstream model and the proxy keep tool calls. +- `openai-responses`: the OpenAI Responses API with function-call output. +- `anthropic`: the Anthropic Messages API with tool use. +- `google-gemini`: the Google Gemini API. +- `minimax`: a MiniMax endpoint compatible with Anthropic Messages. + - Set `memoryApiUrl` to the global endpoint (`https://api.minimax.io`) or the China endpoint (`https://api.minimaxi.com`). + - OMMS adds the `/anthropic/v1/messages` path and the `x-api-key` header. + - MiniMax text models such as `MiniMax-M3` support the adaptive thinking modes this plugin uses through `memoryExtraParams`. +- `orcarouter`: an OpenAI-compatible model gateway with namespaced model IDs. + - `memoryApiUrl` and `memoryModel` are optional. They default to `https://api.orcarouter.ai/v1` and `orcarouter/auto`. `orcarouter/auto` is a routing alias that picks a capable model for each request. + - If you set `memoryModel`, use a namespaced ID such as `openai/gpt-5.5` or `deepseek/deepseek-v4-flash`. OrcaRouter rejects bare model names. + - Only `memoryApiKey` is required: + ```jsonc + "memoryProvider": "orcarouter", + "memoryApiKey": "", + ``` + - [OrcaRouter](https://www.orcarouter.ai) also runs gateway-level security for AI agents on the same endpoint. It screens every prompt and response and controls every tool call on a default-deny basis. You do not need to change application code. ## Embeddings -Embeddings power similarity search for memories and the user profile. Configure them in the same file (`~/.config/omms/omms.jsonc`). There is **no MLX backend** — local embeddings use `@huggingface/transformers` with ONNX, not Apple MLX. +Embeddings power similarity search for memories and the user profile. Set them in the same global file. -**Local (default):** set only `embeddingModel`. On first use the model is downloaded from Hugging Face and cached under `{storagePath}/.cache` (default `~/.omms/data/.cache`). - -**Remote (OpenAI-compatible):** set both `embeddingApiUrl` and `embeddingApiKey`. The plugin then calls `{embeddingApiUrl}/embeddings` with a Bearer token. `embeddingApiKey` accepts the same secret formats as `memoryApiKey` (`literal`, `env://…`, `file://…`). +- There is **no MLX backend**. Local embeddings use `@huggingface/transformers` with ONNX, not Apple MLX. +- **Local (default):** set only `embeddingModel`. On first use, OMMS downloads the model from Hugging Face and caches it under `{storagePath}/.cache` (default `~/.omms/data/.cache`). +- **Remote (OpenAI-compatible):** set both `embeddingApiUrl` and `embeddingApiKey`. OMMS then calls `{embeddingApiUrl}/embeddings` with a Bearer token. `embeddingApiKey` accepts the same formats as `memoryApiKey`: a literal key, `env://…`, or `file://…`. | Key | Role | | --------------------- | ----------------------------------------------------------------------------------------- | | `embeddingModel` | Hugging Face id (local) or API model name (remote). Default: `Xenova/nomic-embed-text-v1` | -| `embeddingDimensions` | Optional override; usually omit — dimensions are looked up from a built-in map | +| `embeddingDimensions` | Optional override. Usually leave it out; OMMS looks up dimensions in a built-in map | | `embeddingApiUrl` | Base URL for an OpenAI-compatible embeddings API (no trailing path beyond `/v1`) | | `embeddingApiKey` | API key for that endpoint (required together with `embeddingApiUrl`) | @@ -169,7 +245,7 @@ Recommended local models: | `Xenova/all-MiniLM-L6-v2` | 384 | Very fast, 512 context | | `Xenova/all-mpnet-base-v2` | 768 | Good quality, 512 context | -Example — remote OpenAI embeddings: +Example of remote OpenAI embeddings: ```jsonc { @@ -179,9 +255,15 @@ Example — remote OpenAI embeddings: } ``` -Changing `embeddingModel` (or dimensions) can trigger re-embedding of stored memories on next startup. Prefer picking a model once and sticking with it for a given data directory. +If you change `embeddingModel` or the dimensions, OMMS can re-embed stored memories at the next start. Choose one model for each data directory and keep it. + +**Intel Mac (`darwin/x64`):** `onnxruntime-node@1.21.0` to `1.23.2` can crash OpenCode's embedded Bun `1.3.14` when the process exits after local embeddings (`Ort::Env` teardown, SIGILL). -**Intel Mac (`darwin/x64`):** `onnxruntime-node@1.21.0` through `1.23.2` can crash OpenCode's embedded Bun `1.3.14` during process exit after successful local embeddings (`Ort::Env` teardown / SIGILL). The fix shipped in `1.24.1`, but fixed releases still lack an x64 native binding. `omms` therefore pins `onnxruntime-node@1.20.1` and loads transformers through a CJS resolve shim so OpenCode nested installs keep that binding. Transformers is resolved to an absolute path before that shim is installed so OpenCode's Bun `--compile` host does not fail with `Cannot find module '@huggingface/transformers' from ''`. After upgrading, clear OpenCode's nested plugin cache (`~/.cache/opencode/packages/om-memory-system@*`, or `opencode-mem@*` on pre-migration installs) and reinstall, or use a remote endpoint via `embeddingApiUrl` + `embeddingApiKey` (example above). This pin stays until onnxruntime publishes a post-teardown-fix darwin/x64 build. +- The fix shipped in `1.24.1`, but fixed releases still have no x64 native binding. +- So `omms` pins `onnxruntime-node@1.20.1`. It loads transformers through a CJS resolve shim, so OpenCode nested installs keep that binding. +- OMMS resolves transformers to an absolute path before it installs that shim. This stops OpenCode's Bun `--compile` host failing with `Cannot find module '@huggingface/transformers' from ''`. +- After you upgrade, clear OpenCode's nested plugin cache (`~/.cache/opencode/packages/om-memory-system@*`, or `opencode-mem@*` on installs before the migration) and reinstall. Or use a remote endpoint through `embeddingApiUrl` and `embeddingApiKey` (example above). +- The pin stays until onnxruntime publishes a darwin/x64 build with the teardown fix. ## Memory scope diff --git a/docs/developers.md b/docs/developers.md index 07c3f0b6..0009d16d 100644 --- a/docs/developers.md +++ b/docs/developers.md @@ -2,25 +2,27 @@ ## Public subpath exports -In addition to the main plugin entry, `omms` exposes one stable subpath -that other opencode plugins can import directly. This avoids having to -reverse-engineer container-tag conventions when writing third-party tools that -read or write into the same memory store. +The package has one stable subpath that other OpenCode plugins can import. +Use it to read or write the same memory store without copying the +container tag rules. + +The other exports are plugin entry points: `.` and `./server` (V1 and V2 +plugin, `dist/plugin.js`) and `./v2` (`dist/v2/plugin.js`). ### `om-memory-system/tags` -Canonical container-tag helpers. The same functions omms itself uses -to scope auto-captured memories. +Container tag helpers. OMMS uses the same functions to scope the memories it +captures. ```ts import { getProjectTagInfo, getUserTagInfo, getTags } from "om-memory-system/tags"; -// Canonical project tag derived from cwd (git remote URL if present, else -// the project root path). Format: `omms_project_`; rows written by -// older versions are migrated automatically on first start. +// Project tag from cwd (git remote URL if present, else the project root +// path). Format: `omms_project_`. Rows written by older versions are +// migrated automatically on first start. const projectTag = getProjectTagInfo(process.cwd()).tag; -// Canonical user tag derived from `git config user.email`. +// User tag from `git config user.email`. // Format: `omms_user_`. const userTag = getUserTagInfo().tag; @@ -28,33 +30,38 @@ const userTag = getUserTagInfo().tag; const { user, project } = getTags(process.cwd()); ``` -Tags produced by these helpers match what auto-capture writes, so third-party -plugins that call `POST /api/memories` will land in the same shards the rest -of the system already understands. Hand-rolled tags whose substring isn't -`_project_` or `_user_` end up in shadow shards that `/api/stats` and -`/api/memories` silently filter out — using these helpers avoids that pitfall. +- These tags match the tags that automatic capture writes. +- A plugin that calls `POST /api/memories` with these tags writes to the same shards as the rest of OMMS. +- `/api/stats` and `/api/memories` ignore tags that do not contain `_project_` or `_user_`. Use the helpers to avoid this. ## Development and contributing -See [CONTRIBUTING.md](../CONTRIBUTING.md) for the full workflow: setup, checks, commit messages, and pull requests. +See [CONTRIBUTING.md](../CONTRIBUTING.md) for the full workflow: setup, +checks, commit messages, and pull requests. -Build and test locally: +To build and check locally: -```bash -bun install -bun run build -bun run typecheck -bun run format -``` +1. Install dependencies: `bun install --frozen-lockfile`. +2. Install web UI dependencies: `(cd web && bun install --frozen-lockfile)`. +3. Run format check, lint, and typecheck: `bun run check`. +4. Build `dist/` and the web UI: `bun run build`. +5. Run the full gate before a push to a pull request: `bun run ci:local`. + +See [CI](ci.md) for what each command does. -This project is actively seeking contributions to become the definitive memory plugin for AI coding agents. Whether you are fixing bugs, adding features, improving documentation, or expanding embedding model support, your contributions are critical. The codebase is well-structured and ready for enhancement. If you hit a blocker or have improvement ideas, submit a pull request - we review and merge contributions quickly. +Contributions are welcome: bug fixes, features, documentation, and more +embedding models. If you are blocked or have an idea, open a pull request. ## Platforms and storage -**CI-tested platforms:** Linux, Windows, macOS 15 and macOS 26 on both Intel (`darwin/x64`) and Apple Silicon (`darwin/arm64`). Older macOS releases are not excluded by that matrix; they are simply outside the current GitHub-hosted runner set. +**CI-tested platforms:** Linux, Windows, and macOS 15 and macOS 26 on Intel +(`darwin/x64`) and Apple Silicon (`darwin/arm64`). The matrix does not block +older macOS releases. They are only outside the current GitHub-hosted runner +set. -- Vector embeddings are stored and searched directly in Turso/libSQL; inserts update the vector index automatically. -- Vector search uses libSQL's DiskANN index via `vector_top_k` (approximate nearest neighbors). -- Auto-capture and user profile learning require an AI provider that can return structured/tool-call output. Memory search/add/list still work without auto-capture provider configuration. +- Turso/libSQL stores and searches the vector embeddings. Inserts update the vector index automatically. +- Vector search uses the libSQL DiskANN index through `vector_top_k` (approximate nearest neighbours). +- Automatic capture and user profile learning need an AI model that can return structured (tool-call) output. +- Memory search, add, and list work without a capture model. Architecture: [shared core](shared-core.md), [OpenCode adapter](opencode-adapter.md), [Pi adapter](pi-adapter.md), [CI](ci.md). diff --git a/docs/moving-projects.md b/docs/moving-projects.md index 09944a9c..25b6c732 100644 --- a/docs/moving-projects.md +++ b/docs/moving-projects.md @@ -1,17 +1,27 @@ # Moving projects and sharing memory -OMMS keeps one memory store per project. This page covers workspaces with several repositories, moved or renamed projects, and moving memories between machines. For a whole-machine move, see [Pi history import: Moving machines](pi-history-import.md#moving-machines). +OMMS keeps one memory store per project. This page covers: + +- workspaces with several repositories +- projects you moved or renamed +- moving memories between machines + +For a move to a new machine, see +[Pi history import: Moving machines](pi-history-import.md#moving-machines). ## One memory for several nested repositories -By default a project is identified by its enclosing git repository, so every -physical git repo gets its own isolated memory store. That is wrong for -multi-repo workspaces — trees managed by Google [`repo`](https://gerrit.googlesource.com/git-repo/+/HEAD/Docs/manual-repo.md), -monorepos, or any layout where several nested git repositories belong to one -logical project — because each sub-repository would be siloed. +By default, OMMS identifies a project by its enclosing git repository. Each git +repository gets its own separate memory store. -Drop an empty **`.omms-project`** marker file at the workspace root (the legacy -`.opencode-mem-project` marker is still honoured and gives the same project identity): +This does not suit a workspace where several nested git repositories form one +project. Examples are trees managed by Google +[`repo`](https://gerrit.googlesource.com/git-repo/+/HEAD/Docs/manual-repo.md) +and some monorepos. Each sub-repository would get its own memory. + +To share one memory, put an empty **`.omms-project`** marker file at the +workspace root. The legacy `.opencode-mem-project` marker still works and gives +the same project identity. ``` my-workspace/ @@ -21,90 +31,100 @@ my-workspace/ └── tools/ (own git repo) ``` -Every session started anywhere underneath the marker then resolves onto that -root and shares one memory store, regardless of which sub-repo the working -directory lives in: - ```sh touch ~/my-workspace/.omms-project ``` -The marker is looked up by walking up from the working directory that every -code path already passes in (the plugin's working directory, the web API's -`process.cwd()`), so identity is **directory-driven and process-independent**. -It does not rely on environment variables or a global config value, which -would be unreliable here: omms runs across multiple opencode processes -that share a single web server, and only some of those processes carry a -given env var. With the marker, the project root is always derived from where -the session actually runs. +Every session started anywhere under the marker then uses that root and shares +one memory store. It does not matter which sub-repository you work in. -The marker takes precedence over git detection. When it is present, the -sub-repo's own git remote is intentionally ignored (it would describe only one -nested repository). Without a marker, behavior is unchanged (git-based -identity). +How OMMS finds the marker: + +- It walks up from the working directory it already receives. That is the + plugin's working directory, or `process.cwd()` for the web API. +- So the identity depends only on the directory, not on the process. +- It does not use environment variables or a global config value. Several + OpenCode processes share one web server, and only some of them would carry + a given environment variable. + +The marker takes priority over git detection. With a marker, OMMS ignores the +sub-repository's own git remote, because it describes only one nested +repository. Without a marker, OMMS uses git-based identity as before. ## Moving or recovering project memories -omms keys project shards by a hash of the project identity. Moving a -repository (OS migration, path reorganization, switching from a Windows mount -to a native path) can therefore orphan the old shard under -`~/.omms/data/projects/` while a new empty shard is created for the -new path. +OMMS names each project shard (memory database file) by a hash of the project +identity. When you move a repository, the old shard under +`~/.omms/data/projects/` can be left behind, and OMMS creates a new empty +shard for the new path. This can happen after an OS migration, a folder +reorganisation, or a switch from a Windows mount to a native path. -These are OpenCode `memory` tool calls with JSON arguments, not commands to -run in a terminal. The issue-style `memory migrate --from ...` notation maps -to `memory({ mode: "migrate", fromPath: "..." })`. +The examples below are `memory` tool calls with JSON arguments. They are not +terminal commands. The notation `memory migrate --from ...` means +`memory({ mode: "migrate", fromPath: "..." })`. -**1. Local move when you still know the old path** +### 1. Local move when you know the old path -Open OpenCode in the **new** project directory. The target project must not -already contain memories (migration aborts unchanged on conflict). Preview the -detected source, destination, and file actions before changing anything: +1. Open OpenCode in the **new** project directory. +2. Check that the new project has no memories yet. Migration stops, with no + changes, if it does. +3. Preview the source, destination and file actions with `dryRun: true`. +4. Run the migration without `dryRun`. ```typescript memory({ mode: "migrate", fromPath: "/old/path/to/project", dryRun: true }); memory({ mode: "migrate", fromPath: "/old/path/to/project" }); ``` -For safety, migration refuses a source whose stored project directory still -exists. If you intentionally want to move an active source, inspect the dry-run -output first and then pass `allowLinkedSource: true`. Original source shard -files are retained as timestamped `*.pre-path-migrate-*.bak` backups. +- For safety, migration refuses a source whose stored project directory still + exists. +- To move an active source on purpose, check the dry-run output first. Then + pass `allowLinkedSource: true`. +- OMMS keeps the original source shard files as timestamped + `*.pre-path-migrate-*.bak` backups. + +### 2. The old path is gone -**2. Old path is gone — discover the orphaned shard first** +1. List the shards to find the orphaned one. +2. Migrate it by its hash. ```typescript memory({ mode: "list-shards" }); memory({ mode: "migrate", fromHash: "fa645294d88bbae2" }); ``` -`list-shards` reports each project hash, stored `projectPath`, memory count, -and status (`current`, `linked`, `orphaned`, `missing-file`, `empty`, or -`ambiguous`). `fromHash` is the 16-character lowercase hexadecimal `scopeHash` -returned by this call. Prefer it when the old directory no longer exists or -multiple shards contain the same stored path, because git-based identities -cannot always be recomputed from a missing path. +- `list-shards` reports each project hash, the stored `projectPath`, the + memory count, and a status. The status is `current`, `linked`, `orphaned`, + `missing-file`, `empty` or `ambiguous`. +- `fromHash` is the 16-character lowercase hexadecimal `scopeHash` from that + list. +- Use `fromHash` when the old directory no longer exists. Also use it when + several shards have the same stored path. OMMS cannot always work out a + git-based identity from a missing path. -**3. Cross-machine backup / restore** +### 3. Backup and restore across machines ```typescript -// on the source machine / old checkout +// on the source machine or old checkout memory({ mode: "export", outputPath: "./memories.json" }); -// on the destination machine / new checkout +// on the destination machine or new checkout memory({ mode: "import", inputPath: "./memories.json", dryRun: true }); memory({ mode: "import", inputPath: "./memories.json" }); ``` -Export writes a versioned JSON document without vectors. Import remaps the -memories onto the current project and recomputes embeddings with the currently -configured model. Import adds memories to an existing project, but duplicate -memory IDs abort the whole import before writing; this differs from `migrate`, -which requires an empty target. - -Export files are plaintext and can contain memory content, user names/email -addresses, repository URLs, and absolute project paths. Store them like other -sensitive backups and delete them when no longer needed. Fully private entries -are omitted, and user profiles and prompt history are not included. The -document contains `schemaVersion: 1`; imports reject newer unsupported schema -versions rather than guessing. +- Export writes a versioned JSON document without vectors. +- Import moves the memories onto the current project. It computes new + embeddings with the model that is set now. +- Import adds memories to a project that already has some. But if any memory + ID is already there, the whole import stops before it writes anything. + `migrate` is different: it needs an empty target. +- The document contains `schemaVersion: 1`. Import refuses newer schema + versions it does not support, instead of guessing. +- Fully private entries are left out. User profiles and prompt history are not + included. + +> **Keep export files safe.** They are plain text. They can contain memory +> content, user names, email addresses, repository URLs and absolute project +> paths. Store them like other sensitive backups, and delete them when you no +> longer need them. diff --git a/docs/omms-migration.md b/docs/omms-migration.md index 1e047424..fef498c4 100644 --- a/docs/omms-migration.md +++ b/docs/omms-migration.md @@ -1,11 +1,11 @@ # Migrating from opencode-mem to omms -omms is the fork's package identity (upstream owns `opencode-mem` on npm). -From version 3.0.0 the identity is fully separated: +omms is this fork's own identity. The upstream project owns `opencode-mem` on +npm. From version 3.0.0 the two are fully separate: | What | Legacy (opencode-mem) | omms | | -------------------- | ---------------------------------------- | --------------------------------------------------------- | -| Package | `opencode-mem` | `omms` | +| npm package | `opencode-mem` | `om-memory-system` | | Default store | `~/.opencode-mem/data` | `~/.omms/data` | | Primary config | `~/.config/opencode/opencode-mem.jsonc` | `~/.config/omms/omms.jsonc` | | Plugin id | `opencode-mem` | `omms` | @@ -19,40 +19,42 @@ From version 3.0.0 the identity is fully separated: ## What happens automatically -On the first start after upgrading, if `~/.opencode-mem/data` exists and -`~/.omms/data` does not, omms runs a one-time migration: - -1. **Backup.** A timestamped, checksum-verified backup of the ENTIRE - `~/.opencode-mem` directory is created at - `~/.omms/backups/opencode-mem-/`, with a `manifest.json` - recording every file's size and SHA-256. The backup is verified before - anything else happens. -2. **Copy.** The store is COPIED to `~/.omms/data`. Every copied file is - checksum-verified against its source. The legacy directory is never moved, - renamed, modified, or deleted. -3. **Marker.** `~/.omms/migration-marker.json` records the source, - destination, backup path, file count, and timestamps. Any later start sees - the marker and does nothing. - -If the backup or any file verification fails, the migration aborts -immediately, writes a marker with `status: "failed"`, and storage keeps -resolving to `~/.opencode-mem/data`. Your legacy directory is untouched -either way. - -Fresh installs (no `~/.opencode-mem` directory) start directly at the omms -paths; no marker or backup is created. +On the first start after the upgrade, omms runs a one-time migration if +`~/.opencode-mem/data` exists and `~/.omms/data` does not: + +1. **Backup.** omms makes a timestamped backup of the WHOLE `~/.opencode-mem` + directory at `~/.omms/backups/opencode-mem-/`. A + `manifest.json` records each file's size and SHA-256 checksum. omms checks + the backup before it does anything else. +2. **Copy.** omms COPIES the store to `~/.omms/data` and checks each copied + file against its source. It never moves, renames, changes or deletes the + legacy directory. +3. **Marker.** `~/.omms/migration-marker.json` records the source, the + destination, the backup path, the file count and the times. Later starts + see the marker and do nothing. + +If the backup or any file check fails: + +- the migration stops at once +- it writes a marker with `status: "failed"` +- storage stays on `~/.opencode-mem/data` + +Your legacy directory is untouched in every case. + +A fresh install (no `~/.opencode-mem` directory) starts directly on the omms +paths. It makes no marker and no backup. ## Before you upgrade -- Close OpenCode and Pi while the first omms start runs the migration. The - migration copies the store as it reads it; concurrent writes from another - process could miss the copy window. -- Make sure you have disk space for one backup of `~/.opencode-mem` plus one +- Close OpenCode and Pi while the first omms start runs the migration. The copy + reads the store as it is. Writes from another process at the same time could + be missed. +- Make sure you have disk space for one backup of `~/.opencode-mem` and one copy of the store. ## Rollback -Two options, both safe because the legacy directory is never modified: +Both options are safe, because omms never changes the legacy directory. 1. **Point storage at the original** (recommended). Set `storagePath` in `~/.config/omms/omms.jsonc`: @@ -63,12 +65,12 @@ Two options, both safe because the legacy directory is never modified: } ``` - omms then reads and writes the legacy directory exactly as before. Remove - the setting to return to `~/.omms/data`. + omms then reads and writes the legacy directory as before. Remove the + setting to go back to `~/.omms/data`. -2. **Restore the backup.** The verified backup at +2. **Restore the backup.** The backup at `~/.omms/backups/opencode-mem-/` is a full copy of the legacy - directory. Copy it back if you ever damage the original: + directory. Copy it back if the original is ever damaged: ```bash rsync -a ~/.omms/backups/opencode-mem-/ ~/.opencode-mem/ @@ -77,106 +79,118 @@ Two options, both safe because the legacy directory is never modified: ## If the migration failed A failed marker (`status: "failed"` in `~/.omms/migration-marker.json`) keeps -omms on the legacy layout, so nothing is lost. To retry: +omms on the legacy layout, so nothing is lost. To try again: 1. Close OpenCode and Pi. -2. Read the marker's `stage` and `error` fields and fix the cause (commonly - disk space or permissions on `~/.omms`). -3. Delete `~/.omms/migration-marker.json` and, if present, the partial - `~/.omms/data` directory (the migration removes it itself, but a crashed - run can leave it behind). -4. Start OpenCode or Pi once; the migration runs again. +2. Read the `stage` and `error` fields in the marker. +3. Fix the cause. It is often disk space or permissions on `~/.omms`. +4. Delete `~/.omms/migration-marker.json`. +5. Delete the partial `~/.omms/data` directory if it is there. A crashed run + can leave it behind. +6. Start OpenCode or Pi once. The migration runs again. -You can also inspect the backup's `manifest.json` and compare checksums -manually: +You can also compare checksums with the backup's `manifest.json` by hand: ```bash -shasum -a 256 ~/.opencode-mem/data/global.sqlite -jq '.files[] | select(.path == "data/global.sqlite")' \ +shasum -a 256 ~/.opencode-mem/data/metadata.db +jq '.files[] | select(.path == "data/metadata.db")' \ ~/.omms/backups/opencode-mem-/manifest.json ``` -## Configuration: dual-read +## Configuration: reading both files -`~/.config/omms/omms.jsonc` is the primary config. The legacy -`~/.config/opencode/opencode-mem.jsonc` is read only while no omms config -file exists, and it is never written. To migrate your settings by hand, copy -the legacy file to `~/.config/omms/omms.jsonc` and edit it; once the omms file -exists it takes precedence. A fresh install with no config at all gets a -commented template at `~/.config/omms/omms.jsonc`. +- `~/.config/omms/omms.jsonc` is the primary config. +- omms reads the legacy `~/.config/opencode/opencode-mem.jsonc` only while no + omms config file exists. It never writes to it. +- To move your settings by hand, copy the legacy file to + `~/.config/omms/omms.jsonc` and edit it. Once the omms file exists, it wins. +- A fresh install with no config gets a commented template at + `~/.config/omms/omms.jsonc`. -Project-level overrides live in `/.opencode/omms.jsonc`. The legacy -`/.opencode/opencode-mem.jsonc` is still read when no `omms.jsonc` -exists; when both exist, `omms.jsonc` wins. Rename the file at your own pace. +Project settings live in `/.opencode/omms.jsonc`. omms still reads the +legacy `/.opencode/opencode-mem.jsonc` when no `omms.jsonc` exists. +When both exist, `omms.jsonc` wins. Rename the file when it suits you. ## Container tag prefix -Memory rows written by older versions carry the `opencode_project_` -and `opencode_user_` container tag prefix. From the release that -includes the tag prefix migration, new memories carry `omms_` instead, and -stored rows are migrated automatically on the first start. +A container tag marks which project or user a memory belongs to. Older versions +wrote memory rows with the `opencode_project_` and +`opencode_user_` prefixes. From the release with the tag prefix +migration, new memories use `omms_`. Stored rows are migrated automatically on +the first start. ### What happens automatically -On the first start after upgrading, before any memory read or write is -served: - -1. **Backup.** A timestamped, checksum-verified copy of the ENTIRE store - directory is created at `~/.omms/backups/tag-prefix-/`, beside - the directory-migration backups, with a `manifest.json` recording every - file's size and SHA-256. The backup is verified before anything is - rewritten. It is never deleted or modified. If it cannot be created or - verified, nothing is rewritten and the start aborts with an error. -2. **Rewrite.** Every memory row's `container_tag` is rewritten from - `opencode__` to `omms__` across all project and - user shards. Each shard is rewritten by one SQL UPDATE inside that shard's - write transaction, under the existing cross-process write lock, so a - concurrent host can never interleave with the rewrite. -3. **Verification.** Per shard: the row count is unchanged, the memory id set - is unchanged, the number of rewritten rows equals the number of `opencode_` - rows seen before, and zero `opencode_` rows remain. Any mismatch rolls - that shard's transaction back and aborts the start. Vectors, metadata, and - every other column are untouched. -4. **Marker.** Completion is recorded in a `tag_prefix_migration` table in - the store's `metadata.db`; per-shard progress is recorded in each shard's - `shard_metadata` table. Later starts see the marker and do nothing. - -The migration is idempotent and resumable. An interrupted run resumes on the -remaining shards only. A crash between the last shard rewrite and the marker -write completes on the next start without rewriting anything. - -Close OpenCode and Pi while the first start after upgrade runs the migration, -for the same reason as the directory migration above. +On the first start after the upgrade, before OMMS serves any memory read or +write: + +1. **Backup.** omms makes a timestamped copy of the WHOLE store directory at + `~/.omms/backups/tag-prefix-/`. + - A `manifest.json` records each file's size and SHA-256 checksum. + - omms checks the backup before it rewrites anything. + - omms never deletes or changes the backup. + - If the backup cannot be made or checked, nothing is rewritten and the + start stops with an error. +2. **Rewrite.** omms rewrites each memory row's `container_tag` from + `opencode__` to `omms__`, in every project and + user shard. + - Each shard gets one SQL UPDATE inside that shard's write transaction. + - This runs under the existing cross-process write lock, so another host + cannot write in the middle of it. +3. **Checks.** For each shard, omms checks that: + - the row count has not changed + - the set of memory IDs has not changed + - the number of rewritten rows equals the number of `opencode_` rows before + - no `opencode_` rows remain + + Any mismatch rolls back that shard's transaction and stops the start. + Vectors, metadata and all other columns are not touched. + +4. **Marker.** A `tag_prefix_migration` table in the store's `metadata.db` + records completion. Each shard's `shard_metadata` table records progress for + that shard. Later starts see the marker and do nothing. + +You can safely run the migration more than once, and it continues after an +interruption: + +- An interrupted run continues on the remaining shards only. +- If a crash happens after the last shard rewrite but before the marker write, + the next start completes it without rewriting anything. + +Close OpenCode and Pi while the first start after the upgrade runs this +migration, for the same reason as the directory migration above. ### Rollback -Restore the verified backup over the store directory, then optionally set -`containerTagPrefix` to `opencode` so tags match the restored rows: +1. Restore the backup over the store directory: -```bash -rsync -a ~/.omms/backups/tag-prefix-/ ~/.omms/data/ -``` + ```bash + rsync -a ~/.omms/backups/tag-prefix-/ ~/.omms/data/ + ``` -```jsonc -{ - // ~/.config/omms/omms.jsonc — only while running on a restored pre-migration store - "containerTagPrefix": "opencode", -} -``` +2. Optionally, set `containerTagPrefix` to `opencode` so tags match the + restored rows: + + ```jsonc + { + // ~/.config/omms/omms.jsonc: only while running on a restored pre-migration store + "containerTagPrefix": "opencode", + } + ``` ### Config warning -If your config explicitly sets `containerTagPrefix: "opencode"`, omms warns -once at startup after the migration: stored rows carry `omms_`, so the -override matches no rows. Remove the override. The setting itself still -works for custom prefixes. +If your config sets `containerTagPrefix: "opencode"`, omms warns once at +startup after the migration. Stored rows now use `omms_`, so that setting +matches no rows. Remove it. The setting still works for custom prefixes. -Operators can disable the automatic gate with `OMMS_SKIP_TAG_PREFIX_MIGRATION=1`. -The test suite uses this; do not set it during normal use. +To turn off the automatic step, set `OMMS_SKIP_TAG_PREFIX_MIGRATION=1`. The +test suite uses this. Do not set it in normal use. ## Log files -New logs go to `~/.omms/omms.log` with `omms-.log` archives. Set -`OMMS_LOG_FILE` to override the path; the legacy `OPENCODE_MEM_LOG_FILE` is -still honoured when `OMMS_LOG_FILE` is unset. Old logs stay in -`~/.opencode-mem/` untouched. +- New logs go to `~/.omms/omms.log`, with `omms-.log` archives. +- Set `OMMS_LOG_FILE` to use a different path. +- omms still honours the legacy `OPENCODE_MEM_LOG_FILE` when `OMMS_LOG_FILE` is + not set. +- Old logs stay untouched in `~/.opencode-mem/`. diff --git a/docs/opencode-adapter.md b/docs/opencode-adapter.md index 82fb9f79..3e857de1 100644 --- a/docs/opencode-adapter.md +++ b/docs/opencode-adapter.md @@ -1,17 +1,17 @@ # OpenCode Adapter -`omms` ships an OpenCode plugin that runs the same shared memory engine as the -Pi extension. Both hosts read and write one store per project, so memories -captured in OpenCode are retrievable from Pi and vice versa. +OMMS includes an OpenCode plugin. It runs the same shared memory engine as +the Pi extension. Both hosts read and write one store for each project. So +Pi can find memories that OpenCode captured, and OpenCode can find memories +that Pi captured. -The package supports both OpenCode plugin APIs: V1 (OpenCode 1.18.29 or -later) and V2 (native `plugins` list). The installed entry point -(`dist/plugin.js`) exports both, and OpenCode loads the one it understands. -The plugin id is always `omms`, whatever the npm package name. +- The package supports both OpenCode plugin APIs: V1 (OpenCode 1.18.29 or later) and V2 (native `plugins` list). +- The installed entry point (`dist/plugin.js`) exports both. OpenCode loads the one it understands. +- The plugin id is always `omms`, whatever the npm package name. ## Installation -From npm (published package), add it to `~/.config/opencode/opencode.json` +From npm, add the package to `~/.config/opencode/opencode.json` (`%USERPROFILE%\.config\opencode\opencode.json` on Windows): ```jsonc @@ -22,11 +22,13 @@ From npm (published package), add it to `~/.config/opencode/opencode.json` { "plugin": ["om-memory-system"] } ``` -On OpenCode v2 you can instead run `opencode plugin add om-memory-system`. -Restart OpenCode after changing the configuration. +On OpenCode v2 you can run `opencode plugin add om-memory-system` instead. +Restart OpenCode after you change the configuration. -From a local checkout (development), build and pack the plugin, then install -the tarball the way OpenCode installs a published package: +From a local checkout (development): + +1. Build and pack the plugin with the command below. +2. Install the tarball the same way OpenCode installs a published package. ```bash bun install && bun run build && npm pack @@ -39,15 +41,13 @@ CI checks for a packed build. The plugin reads the same configuration files as the Pi extension: -1. `~/.config/omms/omms.jsonc` (global; the legacy - `~/.config/opencode/opencode-mem.jsonc` is still read while the omms file - does not exist) -2. `/.opencode/omms.jsonc` (project overrides; the legacy - `/.opencode/opencode-mem.jsonc` is still read when no `omms.jsonc` exists) +1. `~/.config/omms/omms.jsonc` (global). If this file does not exist, the + plugin reads the legacy `~/.config/opencode/opencode-mem.jsonc`. +2. `/.opencode/omms.jsonc` (project overrides). If this file does not + exist, the plugin reads the legacy `/.opencode/opencode-mem.jsonc`. -Storage, embedding, privacy, deduplication, scopes, and thresholds are shared. -`storagePath` defaults to `~/.omms/data`, so both hosts use the same store for -the same project unless you override it. +- Storage, embedding, privacy, deduplication, scopes, and thresholds are shared. +- `storagePath` defaults to `~/.omms/data`. So both hosts use the same store for the same project, unless you change it. OpenCode-specific options: @@ -61,60 +61,75 @@ OpenCode-specific options: } ``` -Model selection follows the same rule as Pi ([Configuration: Choosing the model](configuration.md#choosing-the-model)): `opencodeProvider`/`opencodeModel` if set, otherwise the -external API (`memoryModel`/`memoryApiUrl`/`memoryApiKey`) if configured, -otherwise the session's model. The provider name must appear in -`opencode providers list`, and the model must support structured output. If -the OpenCode model fails and the external API is configured, the external API -is used instead, with a "Using fallback provider" toast. +The plugin chooses the capture model with the same rule as Pi (see +[Configuration: Choosing the model](configuration.md#choosing-the-model)): + +1. `opencodeProvider`/`opencodeModel`, if set. +2. The external API (`memoryModel`/`memoryApiUrl`/`memoryApiKey`), if configured. +3. The session's model. + +- The provider name must appear in `opencode providers list`. +- The model must support structured output. +- If the OpenCode model fails and the external API is configured, the plugin uses the external API. It shows a "Using fallback provider" toast. + +Model calls run in short internal OpenCode sessions titled `omms capture`: + +- They use `omms-structured`, a least-privilege agent that the plugin registers. +- OpenCode owns the sign-in, so a host model needs no key. +- These sessions never start a capture and are never imported as history. -Model calls run in short internal OpenCode sessions titled `omms capture`, -under a least-privilege `omms-structured` agent that the plugin registers. -OpenCode owns the auth, so no key is needed for a host model. These sessions -never trigger capture themselves and are never imported as history. +Each capture attempt writes a metadata line to the OMMS log. It can also +write a full trace. See [Configuration: Capture diagnostics](configuration.md#capture-diagnostics). -Each capture attempt writes a metadata line to the OMMS log, and optionally a -full trace. See [Configuration: Capture diagnostics](configuration.md#capture-diagnostics). +The OpenCode plugin starts the web UI (`http://127.0.0.1:4747`). When several +OpenCode windows run, the first one owns the port. The others use it. -The web UI (`http://127.0.0.1:4747`) is started by the OpenCode plugin. When -several OpenCode windows run, the first one owns the port; the others use it. +History backfill (automatic history import) uses `opencodeBackfillModel`. See +[Configuration: Automatic history import and login web app](configuration.md#automatic-history-import-and-login-web-app). ## Lifecycle mapping -| OpenCode hook | omms behaviour | -| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | -| Plugin start | Load shared config for the project directory, warm storage and embeddings, read connected providers, start the web UI | -| `config` | Register the `omms-structured` agent and the `/memory-import-opencode-history` command | -| `chat.message` (V1) | Record the user prompt for capture and profile learning; inject recent project memories (`chatMessage.injectOn`: first or every prompt) | -| `prompt` + `context` (V2) | Record the prompt once OpenCode admits it; semantic retrieval injected as a delimited `` system section | -| `chat.params` | Record the prompt's model when capture follows the session model | -| `session.idle` | After 10 seconds of quiet, capture every uncaptured prompt in the session; the web-UI owner also runs profile learning and cleanup | -| `session.compacted` (V1) / `compaction` (V2) | Restore the session's own memories after compaction | -| `command.execute.before` (V1) / command (V2) | Run `/memory-import-opencode-history`; see [opencode-history-import.md](opencode-history-import.md) | -| `memory` tool | Shared add/search/profile/list/forget/help plus migrate/list-shards/export/import | +| OpenCode hook | omms behaviour | +| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Plugin start | Load shared config for the project directory, remove old capture traces, update the web login item when `webServerAutoStart` is set, warm storage and embeddings, read connected providers, register the OpenCode backfill model resolver, start automatic backfill when `autoBackfill` is on, start the web UI | +| `config` | Register the `omms-structured` agent and the `/memory-import-opencode-history` command | +| `chat.message` (V1) | Record the user prompt for capture and profile learning; inject recent project memories (`chatMessage.injectOn`: first or every prompt) | +| `prompt` + `context` (V2) | Record the prompt once OpenCode admits it; semantic retrieval injected as a delimited `` system section | +| `chat.params` | Record the prompt's model when capture follows the session model | +| `session.idle` | After 10 seconds of quiet, capture every uncaptured prompt in the session; the web-UI owner also runs profile learning and cleanup | +| `session.compacted` (V1) / `compaction` (V2) | Restore the session's own memories after compaction | +| `command.execute.before` (V1) / command (V2) | Run `/memory-import-opencode-history`; see [opencode-history-import.md](opencode-history-import.md) | +| `memory` tool | Shared add/search/profile/list/forget/help plus migrate/list-shards/export/import | ### Toasts With `showAutoCaptureToasts`, `showUserProfileToasts` and `showErrorToasts`, -the plugin reports captured memories, profile updates, memory restored after -compaction, fallback-provider switches, and errors as OpenCode toasts. Pi -reports the same events through its footer status and notifications. +the plugin shows OpenCode toasts for: + +- captured memories and profile updates +- memories restored after compaction +- changes to the fallback provider +- errors + +Pi reports the same events through its footer status and notifications. ### Capture boundary Each user prompt is one work unit: the prompt plus the assistant messages up -to the next user prompt. Assistant messages contribute visible text and tool -names with inputs; reasoning and tool outputs are excluded, and tool inputs -are truncated to 100 characters. A prompt is claimed before capture, so -repeated idle events or several windows never capture it twice. Failed -captures retry up to `autoCaptureMaxRetries` times. +to the next user prompt. + +- From assistant messages, capture takes visible text, and tool names with their inputs. +- It leaves out reasoning and tool outputs. It cuts tool inputs to 100 characters. +- The plugin claims a prompt before capture. So repeated idle events or several windows never capture it twice. +- A failed capture tries again up to `autoCaptureMaxRetries` times. ### Compaction -The plugin does not replace OpenCode's compaction. After it runs, up to -`compaction.memoryLimit` memories from that session are restored. On V1 they -are added as a synthetic, no-reply message for the session's own agent; on V2 -they are added to the system prompt of later turns. +The plugin does not replace OpenCode's compaction. After compaction, it +restores up to `compaction.memoryLimit` memories from that session: + +- On V1, it adds them as a synthetic, no-reply message for the session's own agent. +- On V2, it adds them to the system prompt of later turns. ## Provenance @@ -131,23 +146,20 @@ Memories captured from OpenCode carry: } ``` -Imported memories use `sourceType: "history-import"` plus `sourceFile` and -`importId`. Provenance is metadata only: it never changes retrieval -eligibility across hosts. +Imported memories use `sourceType: "history-import"`, plus `sourceFile` and +`importId`. Provenance is only metadata. It never changes which memories +either host can retrieve. ## Shared-store expectations -OpenCode and Pi can run in separate processes against the same store. Writes -go through the same shard allocation and write-lock path, and the same project -directory resolves to the same project tag from either host -(`omms_project_`; rows written by older versions under `opencode_` are -migrated automatically on first start). +OpenCode and Pi can run in separate processes against the same store. + +- Writes go through the same shard allocation and write-lock path. +- The same project folder resolves to the same project tag from either host (`omms_project_`). +- Rows that older versions wrote under `opencode_` are migrated automatically on first start. ## Limitations -- V1 injects recent memories rather than running a semantic search on every - prompt; V2 and Pi search on every prompt. -- Profile learning and cleanup run only in the OpenCode process that owns the - web UI. -- Historical session import is explicit and current-project by default; see - [opencode-history-import.md](opencode-history-import.md). +- V1 adds recent memories. It does not run a semantic search for each prompt. V2 and Pi search for each prompt. +- Profile learning and cleanup run only in the OpenCode process that owns the web UI. +- Manual history import covers the current project by default. See [opencode-history-import.md](opencode-history-import.md). diff --git a/docs/opencode-history-import.md b/docs/opencode-history-import.md index b8bca808..2f9e5458 100644 --- a/docs/opencode-history-import.md +++ b/docs/opencode-history-import.md @@ -1,47 +1,81 @@ # Import OpenCode history -Import past OpenCode V1 conversations into OMMS memories and the user profile. The importer reads OpenCode's SQLite database without changing it. OpenCode can stay open. When the database has a write-ahead log, the importer reads a temporary copy of the database and its log, so recent turns are included, and deletes the copy when it finishes. +Import past OpenCode V1 conversations into OMMS memories and the user profile. + +- The importer reads OpenCode's SQLite database without changing it. +- OpenCode can stay open. When the database has a write-ahead log (WAL), the importer reads a temporary copy of the database and its log. Recent turns are then included. It deletes the copy when it finishes. ## Quick start -Inside OpenCode, in the project you want to import, run a preview: +1. Open OpenCode in the project you want to import. +2. Run a preview. It reports sessions, work units, profile prompts, and unresolved directories. It makes no model calls and writes no store files. -```text -/memory-import-opencode-history --dry-run -``` + ```text + /memory-import-opencode-history --dry-run + ``` -The preview reports sessions, work units, profile prompts and unresolved directories. It makes no model calls and writes no store files. Check the counts and directory mappings, then import: +3. Check the counts and directory mappings. +4. Run the import. -```text -/memory-import-opencode-history -``` + ```text + /memory-import-opencode-history + ``` -The command uses this session's model through OpenCode's own sign-in, so no OMMS API key is needed. To use a different, cheaper model for the import, name one from a connected provider: +The command uses this session's model through OpenCode's own sign-in, so you need no OMMS API key. To use a different, cheaper model, name one from a connected provider: ```text /memory-import-opencode-history --model openrouter/your-smaller-model ``` -`--model` takes `provider/id`; everything after the first `/` is the model id. The session's own model stays unchanged. On the V1 plugin API, OpenCode runs one short turn with the session model to show the finished report. +- `--model` takes `provider/id`. Everything after the first `/` is the model id. +- The session's own model stays unchanged. +- On the V1 plugin API, OpenCode runs one short turn with the session model to show the finished report. -By default only sessions recorded for the current project are imported. Add `--scope all-projects` to import every project. The importer uses `~/.local/share/opencode/opencode.db` by default; use `--db ` for another OpenCode V1 database. Run only one importer against the memory store at a time. +Other defaults: + +- Only sessions recorded for the current project are imported. Add `--scope all-projects` to import every project. +- The importer reads `~/.local/share/opencode/opencode.db`. Use `--db ` for another OpenCode V1 database. +- One OpenCode import runs at a time. A second one stops with an "already running" message. ## Automatic import -By default, OMMS waits about 30 seconds after OpenCode starts. It saves a cutoff for OpenCode and imports earlier turns across projects whose directories resolve. It skips exchanges already saved by live capture, checking the stored entry IDs and Pi prompt IDs. Later starts resume pending work using the same cutoff and ledger; live capture handles newer turns. The importer reads a private database snapshot while OpenCode is running. Use a manual command for custom databases, maps or another date range. +By default, OMMS waits about 30 seconds after OpenCode starts. It then saves a cutoff for OpenCode and imports earlier turns across projects whose directories resolve. + +- It skips exchanges that live capture already saved, by checking the stored entry IDs. +- Later starts resume pending work with the same cutoff and ledger. Live capture handles newer turns. +- It reads a private database snapshot while OpenCode is running. +- Use a manual command for custom databases, maps, or another date range. +- Automatic import makes model calls. To prevent it, set `"autoBackfill": false` in the global config before you start OpenCode. + +`opencodeBackfillModel` chooses the backfill model: + +- `"inherit"` (default) uses the configured OpenCode host model, then the saved external API, then OpenCode's configured default model. +- `"external"` uses the external API. +- `provider/model` uses another signed-in model for backfill only. The provider must be connected. +- A missing model records an error. It does not change the live-capture model. + +On the Settings page you can: -`opencodeBackfillModel: "inherit"` selects the configured OpenCode host model, the saved external API, or OpenCode's configured default model. Set it to `provider/model` to use another signed-in model for backfill alone. A missing model records an error instead of changing the live-capture model. Automatic import makes model calls. To prevent it, set `"autoBackfill": false` in the global config before starting OpenCode. Check the cutoff, progress and errors in [Web Settings](web-ui.md#settings-page). +- See the cutoff, progress, and errors. +- **Run now**, **Pause**, and **Resume** the backfill, and watch its progress bar and minutes left. +- Manage saved directory maps (`importPathMaps`) in **Directory maps**. The page suggests targets for unresolved directories. Saved maps also apply to the automatic import. + +See [Web UI settings](web-ui-settings.md) for details. + +- With `opencodeBackfillModel` set to `"external"`, **Run now** works in the login web app with no host open. +- A paused backfill stays paused across OpenCode starts until you resume it. +- One OpenCode import runs at a time, whether from a backfill, the page, a slash command, or the CLI. ## From a terminal -The same import also runs outside OpenCode. It then has no session model, so it calls the external model configured in OMMS (`memoryProvider`, `memoryModel` and its credentials): +The same import runs outside OpenCode. It then has no session model, so it calls the external model saved in OMMS (`memoryProvider`, `memoryModel`, and its credentials): ```bash npx om-memory-system import-opencode-history --dry-run npx om-memory-system import-opencode-history ``` -To use a different external model for this run only, pass its settings. Keep the key in an environment variable, not in the command: +To use a different external model for one run, pass its settings. Keep the key in an environment variable, not in the command: ```bash export OMMS_IMPORT_KEY='your-key' @@ -50,54 +84,59 @@ npx om-memory-system import-opencode-history \ --api-url 'https://your-provider.example/v1' --api-key-env OMMS_IMPORT_KEY ``` -`--api-url` is required when `--provider` differs from the saved provider. The terminal command prints `Import model: provider/model` before it starts. Pi history imports the same way with `import-pi-history`. +- `--api-url` is required when `--provider` differs from the saved provider, except for `orcarouter`. +- The terminal command prints `Import model: provider/model` before it starts. +- Pi history imports the same way with `import-pi-history`. ## Options -The OpenCode and Pi commands, in a session or a terminal, take the same options. A value may follow its flag or use `--flag=value`; quote values that contain spaces. - -| Flag | Effect | -| ----------------------------------------- | -------------------------------------------------------------------- | -| `--dry-run` | Preview with no model calls or writes. | -| `--model ` | In a session: use this connected model instead of the session's. | -| `--scope ` | `current-project` (default) or `all-projects`. | -| `--project ` | Project for `current-project` scope. Default: the working directory. | -| `--session ` | Import one top-level session. | -| `--since `, `--until ` | Include work units inside these dates, inclusive. | -| `--max-sessions ` | Read at most this many sessions, oldest first. | -| `--map =` | Remap a missing session directory. Repeat as needed. | -| `--db ` | Read this V1 SQLite database. | -| `--skip-memories` | Record prompts and build the profile only. | -| `--skip-profile` | Import memories only. | -| `--profile-batch ` | Analyse this many prompts per profile batch. Default: 50. | -| `--force` | Reprocess memory work units with final ledger states. | -| `--help` | Show command help. | -| `--provider`, `--model `, `--api-url` | Terminal only: select an external model for this run. | -| `--api-key-env ` | Terminal only: read this run's API key from an environment variable. | - -Dates use ISO 8601 or epoch milliseconds. A bare date such as `2026-03-31` in `--until` covers that whole day. The importer does not save model choices to the configuration. +The OpenCode and Pi commands take the same options, in a session or a terminal. [cli.md](cli.md#import-options) has the full table. + +- A value may follow its flag or use `--flag=value`. Quote values that contain spaces. + +| Flag | Effect | +| ----------------------------------- | ------------------------------------------------------------------------------------- | +| `--dry-run` | Preview with no model calls or writes. | +| `--model ` | In a session: use this connected model instead of the session's. | +| `--scope ` | `current-project` (default) or `all-projects`. | +| `--project ` | Project for `current-project` scope. Default: the working directory. | +| `--session ` | Import one top-level session. | +| `--since `, `--until ` | Include work units inside these dates, inclusive. | +| `--max-sessions ` | Read at most this many sessions, oldest first. | +| `--map =` | Remap a missing session directory for this run. Adds to `importPathMaps`. Repeatable. | +| `--db ` | Read this V1 SQLite database. | +| `--skip-memories` | Record prompts and build the profile only. | +| `--skip-profile` | Import memories only. | +| `--profile-batch ` | Analyse this many prompts per profile batch. Default: 50. | +| `--force` | Reprocess memory work units with final ledger states. | +| `--help` | Show command help. | +| `--provider `, `--model ` | Terminal only: choose an external provider and model id for this run. | +| `--api-url ` | Terminal only: the endpoint for this run. | +| `--api-key-env ` | Terminal only: read this run's API key from an environment variable. | + +- Dates use ISO 8601 or epoch milliseconds. +- A bare date such as `2026-03-31` in `--until` covers that whole day. +- The importer does not save model choices to the configuration. ## How it works -1. The importer reads top-level sessions from the database, oldest first. If a - `-wal` file exists, it reads a temporary copy of the database and its WAL, - so turns OpenCode has not yet checkpointed are included. The copy is - deleted afterwards. OpenCode's own files stay unchanged. -2. Child sessions (subagents) are folded into their parent. Sessions that OMMS - created for its own model calls are skipped. -3. Each session's directory resolves to a project through the same project - identity as live capture, so imported memories land in the namespace you - already use (see Project resolution below). -4. Each user prompt becomes one work unit with its assistant text and tool - inputs. Reasoning, synthetic parts and tool outputs are excluded; tool - inputs are truncated. -5. Every unit flows through the live-capture pipeline: privacy filter, - extraction, dedup, embedding and storage, with provenance `host=opencode`, - `sourceType=history-import`, the session id, source file and timestamps. -6. Each past prompt is recorded once for profile learning, then analysed in - batches to create or update your user profile. - -The importer takes each session's recorded directory. If it still exists, that directory determines the project identity. For a deleted worktree, `--map` takes priority over OpenCode's recorded project worktree. A usable project worktree then provides a fallback. Unresolved directories appear in the report and are skipped. The root directory `/` is not a project fallback. Child sessions are folded into their parent session, rather than imported as separate conversations. Sessions that OMMS itself created for capture and profile calls are never imported. +1. The importer reads top-level sessions from the database, oldest first. If a `-wal` file exists, it reads a temporary copy of the database and its WAL. Turns that OpenCode has not yet checkpointed are then included. The copy is deleted afterwards. OpenCode's own files stay unchanged. +2. Child sessions (subagents) are folded into their parent. Sessions that OMMS created for its own model calls are skipped. +3. Each session's directory resolves to a project through the same project identity as live capture. Imported memories land in the namespace you already use. See [Project resolution](#project-resolution). +4. Each user prompt becomes one work unit with its assistant text and tool inputs. Reasoning, synthetic parts, and tool outputs are left out. Tool inputs are shortened. +5. Every unit goes through the live-capture pipeline: privacy filter, extraction, dedup, embedding, and storage. Provenance records `host=opencode`, `sourceType=history-import`, the session id, source file, and timestamps. +6. Each past prompt is recorded once for profile learning. It is then analysed in batches to create or update your user profile. + +### Project resolution + +The importer takes each session's recorded directory. + +1. If the directory still exists, it sets the project identity. +2. For a deleted worktree, a `--map` or saved `importPathMaps` entry comes first. +3. Next, OpenCode's recorded project worktree is used, if it is usable. +4. Otherwise the directory is unresolved. It appears in the report and its sessions are skipped. + +The root directory `/` is never a project fallback. ## What replaces the old scripts @@ -107,31 +146,38 @@ The importer takes each session's recorded directory. If it still exists, that d | `backfill-profile.py` | Record your past prompts for profile learning. | | `build-profile.py` | Analyse those prompts in batches and create or update the profile. | -The import runs all three steps by default. Each exchange can require one model call. Each profile batch can require another. The preview shows pending work units and profile prompts so you can estimate cost. `--skip-memories` and `--skip-profile` reduce the work when you only need one result. +The import runs all three steps by default. Use `--skip-memories` or `--skip-profile` when you only need one result. ## Cost -A real import makes one model call per work unit, plus one per profile batch. -Non-technical units return `skip` after one call. Embedding each stored memory -uses your embedding model (local, Ollama or remote), as live capture does. -Run `--dry-run` first: it reports units per project and pending profile -prompts before any spend. +- A real import makes one model call per work unit, plus one per profile batch. +- Non-technical units return `skip` after one call. +- Each stored memory is embedded with your embedding model (local or remote), as live capture does. +- Run `--dry-run` first. It reports units per project and pending profile prompts before any spend. ## Rerun, recover and undo -The importer records work unit outcomes in `/import-ledger.db`. Rerunning skips handled memories and deduplicates profile prompts. Failed work remains retryable. If a process stops after a memory write but before a ledger update, the next run checks the stored import identifier and avoids a duplicate. A failed profile batch keeps its prompts for the next run. +The importer records work unit outcomes in `/import-ledger.db`. + +- A rerun skips handled memories and removes duplicate profile prompts. +- Failed work stays retryable. +- If a process stops after a memory write but before the ledger update, the next run checks the stored import identifier. It does not create a duplicate. +- A failed profile batch keeps its prompts for the next run. + +There is no undo command. To be able to roll back: + +1. Back up the OMMS data directory before a real import. +2. To undo the import, restore that backup. This removes its memories, prompts, profile changes, and ledger entries. -There is no automatic undo command. Back up the OMMS data directory before a real import if you might need to roll back. Restore that backup to undo the import, including its memories, prompts, profile changes and ledger. Restoring also discards any newer OMMS data written since the backup. Do not delete ledger entries alone: a rerun can then create duplicate memories. +Restoring also removes any OMMS data written after the backup. Do not delete ledger entries on their own: a rerun can then create duplicate memories. ## Inspecting import status -The ledger lives at `/import-ledger.db`. Per-run counts appear in -the command report. For per-key inspection: +The ledger is at `/import-ledger.db`. Each run's counts appear in the command report. To inspect each key: ```bash sqlite3 ~/.omms/data/import-ledger.db \ "SELECT status, COUNT(*) FROM import_ledger WHERE key LIKE 'opencode:%' GROUP BY status" ``` -See [cli.md](cli.md) for the terminal command and [opencode-adapter.md](opencode-adapter.md) -for how the plugin captures new work. +See [cli.md](cli.md) for the terminal command and [opencode-adapter.md](opencode-adapter.md) for how the plugin captures new work. diff --git a/docs/pi-adapter.md b/docs/pi-adapter.md index 8224b023..a6edefa3 100644 --- a/docs/pi-adapter.md +++ b/docs/pi-adapter.md @@ -1,17 +1,17 @@ # Pi Adapter -`omms` ships a Pi coding-agent extension that runs the same shared -memory engine as the OpenCode plugin. Both hosts read and write one store per -project, so memories captured in OpenCode are retrievable from Pi and vice -versa. +OMMS includes a Pi coding-agent extension. It runs the same shared memory +engine as the OpenCode plugin. Both hosts read and write one store for each +project. So Pi can find memories that OpenCode captured, and OpenCode can find +memories that Pi captured. -Verified against `@earendil-works/pi-coding-agent` **0.86.1**. Re-verify the -lifecycle APIs in `openspec/changes/add-pi-adapter-shared-memory/design.md` -when upgrading the Pi dependency. +The extension is tested against `@earendil-works/pi-coding-agent` **0.86.1**. +When you upgrade the Pi dependency, check the lifecycle APIs again. They are +listed in `openspec/changes/archive/2026-09-21-add-pi-adapter-shared-memory/design.md`. ## Installation -From npm (published package): +From npm: ```bash pi install npm:om-memory-system @@ -31,26 +31,22 @@ Or try it without installing: pi -e /absolute/path/to/omms ``` -Pi discovers the extension through the `pi` manifest in `package.json` -(`dist/adapters/pi/extension.js`). Pi core packages -(`@earendil-works/pi-coding-agent`, `typebox`) are peer dependencies: the host -Pi runtime provides them and no second runtime is bundled. +- Pi finds the extension through the `pi` manifest in `package.json` (`dist/adapters/pi/extension.js`). +- The Pi core packages (`@earendil-works/pi-coding-agent`, `typebox`) are peer dependencies. +- The Pi runtime on the host provides them. The package does not include a second runtime. ## Configuration The extension reads the same configuration files as the OpenCode plugin: -1. `~/.config/omms/omms.jsonc` (global; the legacy - `~/.config/opencode/opencode-mem.jsonc` is still read while the omms file - does not exist) -2. `/.opencode/omms.jsonc` (project overrides; the legacy - `/.opencode/opencode-mem.jsonc` is still read when no `omms.jsonc` exists) +1. `~/.config/omms/omms.jsonc` (global). If this file does not exist, the + extension reads the legacy `~/.config/opencode/opencode-mem.jsonc`. +2. `/.opencode/omms.jsonc` (project overrides). If this file does not + exist, the extension reads the legacy `/.opencode/opencode-mem.jsonc`. -Storage, embedding, privacy, deduplication, scopes, and thresholds are shared. -`storagePath` defaults to `~/.omms/data` — a legacy `~/.opencode-mem/data` -store is migrated there automatically on first start with a verified backup -first (see [omms-migration.md](omms-migration.md)) — so both hosts use the -same store for the same project unless you override it. +- Storage, embedding, privacy, deduplication, scopes, and thresholds are shared. +- `storagePath` defaults to `~/.omms/data`. So both hosts use the same store for the same project, unless you change it. +- On first start, a legacy `~/.opencode-mem/data` store moves there automatically, after a verified backup. See [omms-migration.md](omms-migration.md). Pi-specific options: @@ -64,28 +60,36 @@ Pi-specific options: } ``` -Model selection follows the same rule as OpenCode ([Configuration: Choosing the model](configuration.md#choosing-the-model)): `piProvider`/`piModel` if set, otherwise the -external API (`memoryModel`/`memoryApiUrl`/`memoryApiKey`) if configured, -otherwise the session's model (`ctx.model`). If the Pi model fails or is not -in Pi's model list and the external API is configured, the external API is -used instead. If no model resolves, automatic capture fails with a log entry; -manual memory operations remain available. +The extension chooses the capture model with the same rule as OpenCode (see +[Configuration: Choosing the model](configuration.md#choosing-the-model)): -Each capture attempt writes a metadata line to the OMMS log, and optionally a -full trace. See [Configuration: Capture diagnostics](configuration.md#capture-diagnostics). +1. `piProvider`/`piModel`, if set. +2. The external API (`memoryModel`/`memoryApiUrl`/`memoryApiKey`), if configured. +3. The session's model (`ctx.model`). -The web UI is not started by the Pi adapter. When both hosts run, let OpenCode -own the web server port as before. +- If the Pi model fails or is not in Pi's model list, and the external API is configured, the extension uses the external API. +- If no model resolves, automatic capture fails and writes a log entry. Manual memory operations stay available. + +Each capture attempt writes a metadata line to the OMMS log. It can also +write a full trace. See [Configuration: Capture diagnostics](configuration.md#capture-diagnostics). + +The Pi adapter does not start the web server itself. When both hosts run, let +OpenCode own the web server port. When `webServerAutoStart` is set, the Pi +adapter updates the web app login item, the same as OpenCode. + +History backfill (automatic history import) uses `piBackfillModel`. See +[Configuration: Automatic history import and login web app](configuration.md#automatic-history-import-and-login-web-app). ## Lifecycle mapping -| Pi event | omms behaviour | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `session_start` | Load shared config for `ctx.cwd`, warm storage and embeddings in the background | -| `before_agent_start` | Semantic retrieval: search project memory with the incoming prompt, inject results as a delimited `` system-prompt section (never a fake user message) | -| `agent_settled` | Automatic capture of the settled work unit: the last user prompt plus its assistant/tool response window from the active branch | -| `session_shutdown` | Idempotent cleanup (quit, reload, new, resume, fork) | -| `memory` tool | Shared add/search/profile/list/forget/help plus migrate/list-shards/export/import | +| Pi event | omms behaviour | +| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `session_start` | Load shared config for `ctx.cwd`, remove old capture traces, update the web login item when `webServerAutoStart` is set, register the Pi backfill model resolver, start automatic backfill when `autoBackfill` is on, warm storage and embeddings in the background | +| `before_agent_start` | Semantic retrieval: search project memory with the incoming prompt, inject results as a delimited `` system-prompt section (never a fake user message) | +| `agent_settled` | Automatic capture of the settled work unit: the last user prompt plus its assistant/tool response window from the active branch | +| `session_shutdown` | Cleanup that is safe to repeat (quit, reload, new, resume, fork). Stops a running automatic backfill and closes the store | +| `/memory-import-pi-history` | Import Pi session history; see [pi-history-import.md](pi-history-import.md) | +| `memory` tool | Shared add/search/profile/list/forget/help plus migrate/list-shards/export/import | ### Footer status @@ -101,18 +105,19 @@ The adapter clears the status when the Pi session shuts down. ### Capture boundary -Capture runs only at `agent_settled`, after automatic retries, compaction, and -queued continuation finish. `agent_end` is deliberately not used. The work unit -is identified by the Pi user-entry ID: retries, compaction continuation, and -repeated settled events never capture the same unit twice. Assistant entries -contribute visible text and tool-call inputs only; hidden thinking blocks and -tool results are excluded, and tool inputs are truncated to 100 characters. +Capture runs only at `agent_settled`, after automatic retries, compaction, +and queued continuation finish. The adapter does not use `agent_end`, on +purpose. + +- The Pi user-entry ID identifies the work unit. So retries, compaction continuation, and repeated settled events never capture it twice. +- From assistant entries, capture takes only visible text and tool-call inputs. +- It leaves out hidden thinking blocks and tool results. It cuts tool inputs to 100 characters. ### Compaction -The adapter does not replace or customise Pi's native compaction. When -compaction occurs mid-run, capture waits for the settled boundary and the work -unit spans the compaction (assistant work on both sides is captured once). +The adapter does not replace or change Pi's native compaction. If compaction +happens during a run, capture waits for the settled point. The work unit then +covers both sides of the compaction, and each part is captured once. ## Provenance @@ -129,21 +134,18 @@ Memories captured from Pi carry: } ``` -Provenance is metadata only: it never changes retrieval eligibility across -hosts. +Provenance is only metadata. It never changes which memories either host can +retrieve. ## Shared-store expectations -OpenCode and Pi can run in separate processes against the same store. Writes -go through the same shard allocation and write-lock path as OpenCode, and the -same project directory resolves to the same project tag from either host -(`omms_project_`; rows written by older versions under `opencode_` are -migrated automatically on first start). +OpenCode and Pi can run in separate processes against the same store. + +- Writes go through the same shard allocation and write-lock path as OpenCode. +- The same project folder resolves to the same project tag from either host (`omms_project_`). +- Rows that older versions wrote under `opencode_` are migrated automatically on first start. ## Limitations -- Pi profile learning applies analysed batches directly. The decay, - validation-task, and conflict-retry machinery of the OpenCode idle path is - not ported. -- Historical session import is explicit and current-project by default; see - [pi-history-import.md](pi-history-import.md). +- Pi profile learning applies analysed batches directly. It does not have the decay, validation-task, and conflict-retry steps of the OpenCode idle path. +- Manual history import covers the current project by default. See [pi-history-import.md](pi-history-import.md). diff --git a/docs/pi-history-import.md b/docs/pi-history-import.md index 5a0d99c9..53e50e08 100644 --- a/docs/pi-history-import.md +++ b/docs/pi-history-import.md @@ -1,35 +1,39 @@ # Pi Historical Session Import -Import your existing Pi session history into the shared memory store. Pi session JSONL files stay unchanged. By default, OMMS also imports older sessions automatically after Pi starts; the command below gives you a manual preview and control over scope and maps. +Import your existing Pi session history into the shared memory store. Pi session JSONL files stay unchanged. + +- By default, OMMS also imports older sessions automatically after Pi starts. See [Automatic import](#automatic-import). +- The command below gives you a manual preview and control over scope and maps. Verified against `@earendil-works/pi-coding-agent` 0.86.1. ## Quick start -From inside `pi`, in the project you want to import for: +1. Open `pi` in the project you want to import for. +2. Run a dry run. It reports what would happen and writes nothing. -```text -/memory-import-pi-history --dry-run -``` + ```text + /memory-import-pi-history --dry-run + ``` -The dry-run reports what would happen and writes nothing. When the numbers -look right: +3. When the numbers look right, run the import. -```text -/memory-import-pi-history -``` + ```text + /memory-import-pi-history + ``` -By default only sessions recorded for the current project are imported, and -extraction uses this session's current model. The `piProvider`/`piModel` -settings apply to live capture only. To import with another model from Pi's -model list, without changing the session's model: +By default: + +- Only sessions recorded for the current project are imported. +- Extraction uses this session's current model. The `piProvider` and `piModel` settings apply to live capture only. + +To import with another model from Pi's model list, without changing the session's model: ```text /memory-import-pi-history --model zai/your-smaller-model ``` -The same import runs from a terminal with an external model and API key, -exactly like the OpenCode import (see [cli.md](cli.md)): +The same import runs from a terminal with an external model and API key. See [cli.md](cli.md). ```bash npx om-memory-system import-pi-history --dry-run @@ -39,17 +43,39 @@ npx om-memory-system import-pi-history --provider openai-chat --model 'your-smal ## Automatic import -On first Pi start, OMMS waits about 30 seconds, then saves a cutoff for Pi and imports earlier turns across projects whose directories resolve. It skips turns already saved by live capture, using the recorded Pi prompt ID and assistant entry IDs. Later Pi starts resume pending work using the same cutoff and ledger. Newer turns are handled by live capture; use this command for custom sources, maps, or another date range. +On the first Pi start, OMMS waits about 30 seconds. It then saves a cutoff for Pi and imports earlier turns across projects whose directories resolve. + +- It skips turns that live capture already saved. It checks the recorded Pi prompt ID and assistant entry IDs. +- Later Pi starts resume pending work with the same cutoff and ledger. +- Live capture handles newer turns. Use the command for custom sources, maps, or another date range. +- The import makes model calls. Preview by hand before a large extra import. +- To stop the automatic run, set `"autoBackfill": false` in the global config. + +`piBackfillModel` chooses the backfill model: + +- `"inherit"` (default) follows Pi's live-capture model rule. +- `"external"` uses the external API. +- `provider/model` chooses a signed-in Pi model for backfill only. +- The setting does not change the slash command's model. -Automatic import uses `piBackfillModel`: `"inherit"` follows Pi's live-capture model rule, and `provider/model` chooses a signed-in Pi model just for backfill. The setting does not change the slash command's session model. Set `"autoBackfill": false` in the global config to prevent the automatic run. For state, pending counts, cutoff and errors, open [Web Settings](web-ui.md#settings-page). The import makes model calls. Preview manually before a large additional import. +On the Settings page you can: + +- See the state, pending counts, cutoff, and errors. +- **Run now**, **Pause**, and **Resume** the backfill, and watch its progress bar and minutes left. +- Manage saved directory maps (`importPathMaps`) in **Directory maps**. The page suggests targets for unresolved directories. Saved maps also apply to the automatic import. + +See [Web UI settings](web-ui-settings.md) for details. + +- With `piBackfillModel` set to `"external"`, **Run now** works in the login web app with no host open. +- A paused backfill stays paused across Pi starts until you resume it. +- One Pi import runs at a time, whether from a backfill, the page, a slash command, or the CLI. ## Command reference -The Pi and OpenCode commands take the same options; see the full table in -[OpenCode history import](opencode-history-import.md#options). Pi reads its -sessions from `--root ` (default `~/.pi/agent/sessions`) where OpenCode -takes `--db`. A value may follow its flag or use `--flag=value`; quote values -that contain spaces. +The Pi and OpenCode commands take the same options. The full table is in [cli.md](cli.md#import-options). + +- Pi reads its sessions from `--root `. OpenCode takes `--db` instead. +- A value may follow its flag or use `--flag=value`. Quote values that contain spaces. ```text /memory-import-pi-history [options] @@ -62,81 +88,69 @@ that contain spaces. --since Only work units at/after this time --until Only work units at/before this time; a bare date covers the day --max-sessions Read at most n sessions, oldest first - --map = Remap a recorded cwd that no longer exists - --root Session root (default ~/.pi/agent/sessions) + --map = Remap a recorded cwd that no longer exists (repeatable) + --root Session folder or one .jsonl file (default ~/.pi/agent/sessions) --skip-memories Record profile prompts only --skip-profile Import memories without profile learning --profile-batch Prompts per profile analysis batch (default: 50) - --force Reprocess units with terminal ledger states + --force Reprocess units with final ledger states + --help Show this help ``` -Dates accept ISO 8601 (`2026-01-01`, `2026-01-01T10:00:00Z`) or epoch -milliseconds. `--since`/`--until` filter on each work unit's user-entry -timestamp, inclusive. +- Dates accept ISO 8601 (`2026-01-01`, `2026-01-01T10:00:00Z`) or epoch milliseconds. +- `--since` and `--until` filter on each work unit's user-entry timestamp. Both are inclusive. ## How it works -1. Sessions are discovered under the session root. Only files with a Pi - session header qualify; subagent artifacts and unrecognized formats are - counted and skipped. -2. Each session loads through Pi's own `SessionManager.open`, which migrates - legacy session versions in memory. -3. The recorded `cwd` in each session header resolves through the same project - identity as live capture, so imported memories land in the same namespace - you already use. -4. Each user prompt on the session's active branch becomes one work unit with - its assistant and tool work. Hidden thinking, images, and tool outputs are - excluded; tool inputs are truncated. -5. Every unit flows through the live-capture pipeline: privacy filter, - extraction (through this session's model unless `--model` is set), dedup, - embedding, persistence, with provenance `host=pi`, `sourceType=history-import`, - session id, source file, entry ids, and timestamps. -6. Each past user prompt is recorded once for profile learning. The importer - analyses unprocessed prompts in batches, creating or updating your user - profile. `--skip-profile` omits these steps. Dry-run reports pending prompts - without recording or analysing them. - -The model override uses Pi's model registry. Select an available model with -`--model provider/id`. The same model handles memory extraction and profile -analysis for this run; the session's model stays unchanged. An unknown model -stops the import before processing. +1. OMMS finds sessions under the session root. Only files with a Pi session header count. It counts and skips subagent files and unknown formats. +2. Each session loads through Pi's own `SessionManager.open`. This updates old session versions in memory. +3. The recorded `cwd` in each session header resolves through the same project identity as live capture. Imported memories land in the namespace you already use. +4. Each user prompt on the session's active branch becomes one work unit, with its assistant and tool work. Hidden thinking, images, and tool outputs are left out. Tool inputs are shortened. +5. Every unit goes through the live-capture pipeline: privacy filter, extraction, dedup, embedding, and storage. + - Extraction uses this session's model unless you set `--model`. + - Provenance records `host=pi`, `sourceType=history-import`, the session id, source file, entry ids, and timestamps. +6. Each past user prompt is recorded once for profile learning. The importer analyses new prompts in batches and creates or updates your user profile. + - `--skip-profile` leaves out these steps. + - A dry run reports pending prompts without recording or analysing them. + +About `--model`: + +- It uses Pi's model registry. Choose an available model as `provider/id`. +- The same model handles memory extraction and profile analysis for this run. +- The session's model stays unchanged. +- An unknown model stops the import before any processing. ## Idempotency and recovery -Each work unit gets a deterministic key: -`pi:::`. A durable -ledger inside the store (`import-ledger.db` in the storage path) records -imported, skipped, and failed states. +Each work unit gets a fixed key: `pi:::`. A ledger in the store (`import-ledger.db` in the storage path) records imported, skipped, and failed states. - Import twice: the second run processes nothing. -- Extraction skips are terminal: non-technical units never spend model calls - again. -- Failures are retryable: fix the cause and rerun. -- Crash between the memory write and the ledger update: the rerun finds the - stored `importId` and reconciles instead of duplicating. +- Extraction skips are final. Non-technical units never cost model calls again. +- Failures can be retried. Fix the cause and run again. +- If a crash happens between the memory write and the ledger update, the next run finds the stored `importId`. It reconciles instead of duplicating. ## Unresolvable directories -Sessions recorded in directories that no longer exist (deleted worktrees, temp -dirs) are skipped and listed in the report. To import them into the project -they belonged to: +Sessions recorded in directories that no longer exist (deleted worktrees, temp folders) are skipped and listed in the report. To import them into the project they belonged to: ```text --map /old/deleted-worktree-path=/current/main-repo-path ``` -The map target must exist. Repeat `--map` for multiple paths. +- The map target must exist. +- Repeat `--map` for more paths. +- To keep a map for every later import and the automatic backfill, save it in `importPathMaps` or in the Settings page's **Directory maps**. +- A `--map` for the same directory wins for that run. ## Cost -One model call per work unit, plus one per profile batch. Non-technical units -return `skip` after a single call. Run `--dry-run` first: it reports the unit -count per project and pending profile prompts before any spend. +- One model call per work unit, plus one per profile batch. +- Non-technical units return `skip` after one call. +- Run `--dry-run` first. It reports the unit count per project and pending profile prompts before any spend. ## Inspecting import status -The ledger lives at `/import-ledger.db`. Status counts per run -appear in the command summary. For per-key inspection: +The ledger is at `/import-ledger.db`. Each run's counts appear in the command summary. To inspect each key: ```bash sqlite3 ~/.omms/data/import-ledger.db \ @@ -145,34 +159,30 @@ sqlite3 ~/.omms/data/import-ledger.db \ ## Moving machines -The memory store is portable. To carry memories and import state to a new -machine: +The memory store is portable. To move memories and import state to a new machine: -1. Copy the whole data directory: +1. Copy the whole data directory. -```bash -rsync -a ~/.omms/data/ newmachine:~/.omms/data/ -``` + ```bash + rsync -a ~/.omms/data/ newmachine:~/.omms/data/ + ``` + +2. Copy the config: `~/.config/omms/omms.jsonc`. On the legacy layout, copy `~/.config/opencode/opencode-mem.jsonc`. +3. Copy any key files your config points to, such as `~/.config/omms/secrets/`. +4. Keep the embedding model the same. Stored vectors only match queries from the same embedding model. +5. Clone your repositories. -2. Copy the config: `~/.config/omms/omms.jsonc` (on machines still on the - legacy layout, `~/.config/opencode/opencode-mem.jsonc`). -3. Install the embedding runtime on the new machine and keep the model - identical (for example `ollama pull qwen3-embedding:0.6b`). Stored vectors - only match queries from the same embedding model. -4. Clone your repositories. +If the absolute project path is the same on the new machine, you are done. -If the absolute project path is identical on the new machine, you are done. -If the username or layout differs, project tags change and memories appear -orphaned. Re-associate per project from inside that project: +If the username or layout differs, project tags change and memories look orphaned. Link them again from inside each project: ```text memory list-shards memory migrate --from-path /old/machine/absolute/path ``` -or `--from-hash ` from the `list-shards` output. The old path does not -need to exist on the new machine. No re-embedding happens. +- You can use `--from-hash ` from the `list-shards` output instead. +- The old path does not need to exist on the new machine. +- No re-embedding happens. -Pi session files move separately (`~/.pi/agent/sessions`). Because the import -ledger travels with the data directory, rerunning the import on the new -machine creates zero duplicates. +Pi session files (`~/.pi/agent/sessions`) move separately. The import ledger moves with the data directory, so running the import again on the new machine creates no duplicates. diff --git a/docs/shared-core.md b/docs/shared-core.md index f336b726..4bb0d5d4 100644 --- a/docs/shared-core.md +++ b/docs/shared-core.md @@ -1,14 +1,17 @@ # Shared Memory Core Boundary -`omms` runs one memory engine behind two host adapters: the OpenCode -plugin and the Pi coding-agent extension. This document defines the boundary, -the dependency rules, and the compatibility guarantee for existing data. +OMMS runs one memory engine behind two host adapters: the OpenCode plugin and +the Pi coding-agent extension. This page sets out: -## Layout and dependency direction +- the boundary between the shared code and the adapters +- the import rules +- the compatibility promise for existing data + +## Layout and import direction ```text - src/core + src/services - (shared memory core/engine) + src/core + src/services + src/importer + (shared memory core and engine) ▲ ▲ │ │ src/adapters/opencode src/adapters/pi @@ -16,92 +19,132 @@ the dependency rules, and the compatibility guarantee for existing data. + src/v2 (compat) ``` -Mandatory direction: +Rules: -- `src/core/*` and `src/services/*` MUST NOT import host SDKs +- `src/core/*` and `src/services/*` must not import host SDKs (`@opencode-ai/*`, `@earendil-works/*`) or anything from `src/adapters/*`. - Enforced by `tests/host-neutral-capture-boundary.test.ts` and - `tests/pi-adapter-boundary.test.ts`. -- Host adapters import the core through narrow ports and own everything - host-specific: lifecycle events, session reading, host UI, host-native model - access, and tool registration. -- Adapters never import each other's host-coupled modules. - -## Ports (src/core/host.ts) - -The shared capture pipeline depends on two interfaces, nothing else: - -- `CaptureSummaryProvider.summarize(request)` — structured extraction. The - OpenCode provider path (`services/ai/*`) and the Pi model bridge - (`adapters/pi/provider.ts`) both implement it. Failure semantics: throw to - defer or skip the work unit; manual memory operations stay available. -- `AutoCaptureHost` — session conversation access, readiness, notifications. - The OpenCode adapter implements it; the Pi adapter drives - `captureConversation` directly from `agent_settled`. - -The normalised work unit (`CaptureWorkUnit` in `src/core/capture.ts`) carries -visible conversation content plus provenance and is the only shape the shared -pipeline accepts. Hidden reasoning and host SDK objects never enter it. + `tests/host-neutral-capture-boundary.test.ts` and + `tests/pi-adapter-boundary.test.ts` enforce this. +- `src/core/*`, `src/services/*`, and `src/types/*` must not import + `src/importer/*`. The one exception is `src/services/web-server.ts`. It + reaches the importer only through dynamic imports of + `src/importer/web-import-api.ts`, `src/importer/settings-health.ts`, and + `src/importer/web-import-jobs.ts`. `tests/pi-adapter-boundary.test.ts` + enforces this. +- The OpenCode entry points (`src/index.ts`, `src/v2/adapter.ts`, + `src/v2/plugin.ts`) must not import `importer/` or `@earendil-works` + directly. They load importer code through `src/adapters/opencode/*`. +- Adapters own everything that is specific to a host: lifecycle events, + session reading, host UI, host model access, and tool registration. +- An adapter must not import the other host's host-coupled modules. +- Load host SDKs and heavy modules with dynamic `import()`. + `tests/plugin-bundle-boundary.test.ts` checks the plugin bundle. + +`src/importer/` is shared by both hosts and never imports an adapter. +`tests/pi-adapter-boundary.test.ts` checks this. + +- The readers for each host's history format live in the importer: + `opencode-reader.ts` for OpenCode's database, and `pi-conversation.ts` + with `session-loader.ts` for Pi session files. +- The Pi adapter imports `pi-conversation.ts` for live capture. Adapters may + depend on shared code; shared code never depends on an adapter. +- `session-loader.ts` loads the Pi SDK. The importer loads it with dynamic + `import()`, only when it reads Pi history. +- `src/core/internal-prompt.ts` recognises omms's own summary and profile + prompts, which are the same text on both hosts. + +## Ports (`src/core/host.ts`) + +The shared capture pipeline depends on two interfaces only: + +- `CaptureSummaryProvider.summarize(request)` does structured extraction. + - The OpenCode provider path (`src/services/ai/*`) and the Pi model bridge (`src/adapters/pi/provider.ts`) both implement it. + - Throw to defer or skip the work unit. Do not return partial data. Manual memory operations stay available. +- `AutoCaptureHost` extends `CaptureSummaryProvider`. It adds session + conversation access (`getConversation`), readiness (`isCaptureReady`), and + optional notifications (`notify`). + - The OpenCode adapter implements it. + - The Pi adapter calls `captureConversation` directly from `agent_settled`. + +`CaptureWorkUnit` in `src/core/capture.ts` is the only shape the shared +pipeline accepts. It holds visible conversation content and provenance. +Hidden reasoning and host SDK objects never go into it. ## What each layer owns -| Layer | Owns | -| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| `src/core` | Ports, capture pipeline, context budgeting, shared extraction schema/parsing, memory tool operations | -| `src/services` | Storage (Turso/libSQL), embeddings, vector search, privacy, deduplication, project identity, profiles, portability, cleanup, web backend | -| `src/adapters/opencode` + `src/index.ts` + `src/v2` | OpenCode lifecycle, session reading, provider bridge, v2 compatibility | -| `src/adapters/pi` | Pi lifecycle (`before_agent_start`, `agent_settled`, `session_shutdown`), retrieval injection, model bridge, memory tool registration | +| Layer | Owns | +| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/core` | Ports, capture pipeline, context budgeting, shared extraction schema and parsing, retrieval, `memory` tool operations | +| `src/services` | Storage (Turso/libSQL), embeddings, vector search, privacy, deduplication, project identity, profiles, portability, cleanup, web backend, live-model rule (`ai/live-model-choice.ts`) | +| `src/importer` | History import and automatic backfill, import ledger, import runs and progress, backfill controls, path maps, the web import API | +| `src/adapters/opencode` + `src/index.ts` + `src/v2` | OpenCode lifecycle, session reading, provider bridge, OpenCode backfill model resolver, V2 compatibility | +| `src/adapters/pi` | Pi lifecycle (`session_start`, `before_agent_start`, `agent_settled`, `session_shutdown`), retrieval injection, model bridge, Pi backfill model resolver, `memory` tool registration | + +### Shared modules added for import and settings + +| Module | Purpose | +| ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- | +| `src/importer/import-runs.ts` | Progress for each host's import run, and one run at a time for each host across every surface (`auto`, `web`, `cli`, `slash`) | +| `src/importer/backfill-controls.ts` | Run now, Pause, and Resume. Each host registers its backfill model resolver with `registerHostBackfillModels` | +| `src/importer/external-backfill-models.ts` | Backfill models from the external API settings (`memoryModel`, `memoryApiUrl`, `memoryApiKey`) | +| `src/importer/external-api-test.ts` | A short test call to the external API | +| `src/importer/import-path-maps.ts` | Checks and normalises `importPathMaps` from the global config | +| `src/importer/map-suggestions.ts` | Suggests path maps by looking for project markers such as `.git` and `package.json` | +| `src/importer/settings-health.ts` | Settings health checks for the web UI | +| `src/importer/web-import-api.ts`, `web-import-jobs.ts` | The importer functions and jobs that the web server calls | +| `src/services/private-path.ts` | Limits a file or folder to the current user. Capture traces and `memory-key-source.ts` use it | +| `src/services/memory-key-source.ts` | How the web UI External API card supplies `memoryApiKey` (`env`, `file`, or a pasted value) | +| `src/services/global-version.ts` | The version of the global `om-memory-system` command, if installed | +| `src/services/package-version.ts` | This package's version, read from its `package.json` | ## Cross-process storage safety OpenCode and Pi can run in separate processes against the same store: - Every connection sets `busy_timeout=5000` and uses one pooled handle. -- Every write flows through `withScopeWriteLock`, which now nests a - per-scope cross-process advisory lock - (`src/services/turso/cross-process-write-lock.ts`, PID-liveness, stale - reclaim) so shard allocation, vector-count sync, insert, and increment run - as one critical section per scope across processes. -- Shard creation races resolve through the `UNIQUE(scope, scope_hash, -shard_index)` constraint with re-read adoption. -- Verified by `tests/two-process-storage.test.ts` (two spawned processes, - production write path): concurrent init, interleaved same-project writes, - read-after-write, close/reopen durability, exact counts, and rollover. - -The default rollback journal is deliberate: file-level migration copies rely -on the main database file being current after every commit. - -## Compatibility guarantee for existing OpenCode data - -Phase 1 preserved, and later phases must preserve: - -1. The default store is `~/.omms/data`, established by a one-time verified - migration from `~/.opencode-mem/data` (see - [omms-migration.md](omms-migration.md)): the legacy directory is backed up - and copied, never modified, and storage keeps resolving to the legacy - layout until the migration succeeds. The container tag prefix moved from - the historical `opencode_project_` to `omms_project_` via a - one-time verified rewrite of stored rows (same backup-first pattern). The - same project directory resolves to the same shard set from either host. -2. Memories written before provenance fields existed remain valid and +- Every write goes through `withScopeWriteLock`. It nests a cross-process + lock for each scope (`src/services/turso/cross-process-write-lock.ts`). + That lock checks if the owner PID is alive and takes over stale locks. +- So shard allocation, vector-count sync, insert, and increment run as one + critical section for each scope, across processes. +- Races to create a shard end at the `UNIQUE(scope, scope_hash, shard_index)` + constraint. The loser reads the row again and uses it. +- `tests/two-process-storage.test.ts` checks this with two spawned processes + on the production write path. It covers concurrent start, mixed + same-project writes, read-after-write, close and reopen, exact counts, and + rollover. + +The default rollback journal is on purpose. File-level migration copies need +the main database file to be current after every commit. + +## Compatibility promise for existing OpenCode data + +Phase 1 kept these, and later phases must keep them: + +1. The default store is `~/.omms/data`. + - A one-time verified migration moves it from `~/.opencode-mem/data` (see [omms-migration.md](omms-migration.md)). + - The legacy folder is backed up and copied, never changed. + - Storage uses the legacy layout until the migration succeeds. + - The container tag prefix changed from `opencode_project_` to `omms_project_`. A one-time verified rewrite of stored rows did this, with a backup first. + - The same project folder resolves to the same shard set from either host. +2. Memories written before provenance fields existed stay valid and searchable. Provenance (`host`, `hostSessionId`, `sourceType`, `sourceEntryIds`, `sourceTimestamp`, `sourceFile`, `importId`) is optional - metadata and never gates retrieval. -3. No re-embedding is required to upgrade. Schema changes must be additive - and idempotent on open. -4. The OpenCode v1 entry point (`src/index.ts`) keeps its existing behaviour. - The v2 entry (`src/v2/adapter.ts`) uses native v2 session hooks: per-prompt - retrieval through the shared `src/core/retrieval.ts` (the same code Pi - uses) and compaction restore through the `compaction` hook. The memory - tool, idle capture, and profile learning still route through the shared - operations layer. - -## Adding a new host adapter + metadata. It never controls retrieval. +3. An upgrade never needs new embeddings. Schema changes must only add, and + must be safe to run again on open. +4. The OpenCode V1 entry point (`src/index.ts`) keeps its behaviour. + - The V2 entry (`src/v2/adapter.ts`) uses native V2 session hooks. + - It retrieves for each prompt through the shared `src/core/retrieval.ts`, the same code Pi uses. + - It restores memories after compaction through the `compaction` hook. + - The `memory` tool, idle capture, and profile learning still go through the shared operations layer. + +## Add a new host adapter 1. Implement `CaptureSummaryProvider` for the host's model runtime. -2. Normalise the host's transcript into `CaptureConversation` (visible text - and bounded tool inputs only). -3. Call `captureConversation` with a `CaptureWorkUnit` carrying provenance. -4. Route the host's memory tool surface through `executeMemoryOperation`. -5. Add boundary tests: no host SDK in the shared path, type-only host imports - in the adapter. +2. Convert the host's transcript into `CaptureConversation`. Include visible text and bounded tool inputs only. +3. Call `captureConversation` with a `CaptureWorkUnit` that carries provenance. +4. Send the host's `memory` tool calls through `executeMemoryOperation`. +5. Register a backfill model resolver with `registerHostBackfillModels`. +6. Add the same features as the other hosts, so all hosts stay equal. +7. Add boundary tests. Keep host SDKs out of the shared path. Use type-only host imports in the adapter. diff --git a/docs/upgrading.md b/docs/upgrading.md index 42ea555c..baeae912 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -1,50 +1,94 @@ # Updating and upgrading +This page covers updates, pinning a version, and older stores. + ## Updating OMMS -Install OMMS without a version number, as shown above, so your agent can tell -you when a new release is out. Neither agent installs updates by itself; you -choose when to update. +Install OMMS without a version number, as the [README](../README.md#set-up) +shows. Your agent can then tell you when a new release is out. Neither agent +installs updates by itself. You choose when to update. | Agent | How you hear about a new release | Update with | | ----------- | ------------------------------------------------------------- | ----------------------------------------------------------------------- | | Pi | Pi shows an update notice while you work | `pi update npm:om-memory-system` (or `pi update --extensions` for all) | | OpenCode v2 | Run `opencode plugin check` to list plugins with new versions | `opencode plugin update om-memory-system` (or `opencode plugin update`) | -Restart the agent after updating. +Restart the agent after you update. + +### The global terminal command + +If you installed the terminal command globally (see +[CLI](cli.md#global-install-optional-recommended)), update it on its own: + +```bash +npm i -g om-memory-system@latest # or: bun add -g om-memory-system@latest +om-memory-system --version +``` + +The **Web app** card on the Settings page warns when the global command's +version is different from the OMMS version that serves the page. See +[Settings page](web-ui-settings.md). + +### Rolling back to an older version + +Versions before `external` and `importPathMaps` existed treat them like this: -To stay on one version, install it with the version number instead: -`pi install npm:om-memory-system@3.1.1` in Pi, or `opencode plugin add om-memory-system@3.1.1` in -OpenCode. A pinned install is never updated or flagged; install without the -number again to go back to receiving updates. +- They reject `"external"` as a model value. Change those values back before + you roll back. +- They ignore `importPathMaps` and saved key files. + +### Pinning a version + +To stay on one version, install it with the version number: + +- Pi: `pi install npm:om-memory-system@3.1.1` +- OpenCode: `opencode plugin add om-memory-system@3.1.1` + +A pinned install is never updated or flagged. To receive updates again, +install without the number. ### Trying unreleased changes (`next`) Every merge to `main` is published as a prerelease under the npm `next` tag, -for example `3.2.0-next.8`. It has not been through the release checks, so use -it only to try a change early. In Pi, install it with -`pi install npm:om-memory-system@next`; Pi then reports each newer `next` -build and `pi update` installs it. Whether OpenCode's `opencode plugin check` -reports newer `next` builds has not been verified. Install without `@next` to -return to full releases. +for example `3.2.0-next.8`. It has not been through the release checks. Use it +only to try a change early. -Release notes for every version are in [CHANGELOG.md](../CHANGELOG.md) and on the -[GitHub Releases](https://github.com/cmdaltctr/omms/releases) page. +- In Pi, install it with `pi install npm:om-memory-system@next`. Pi then + reports each newer `next` build, and `pi update` installs it. +- We have not checked whether OpenCode's `opencode plugin check` reports newer + `next` builds. +- To return to full releases, install without `@next`. -Upgrading from an existing `opencode-mem` install? The store migrates to -`~/.omms/data` automatically on first start, with a verified backup first. -See [docs/omms-migration.md](omms-migration.md). +Release notes for every version are in [CHANGELOG.md](../CHANGELOG.md) and on +the [GitHub Releases](https://github.com/cmdaltctr/omms/releases) page. + +Upgrading from an `opencode-mem` install? The store moves to `~/.omms/data` +automatically on first start, after a verified backup. See +[Migrating from opencode-mem](omms-migration.md). ## Upgrading from legacy SQLite shards -On first startup after upgrading, omms automatically migrates existing memory shard databases to native Turso/libSQL vector format: +A shard is one memory database file. On the first start after an upgrade, OMMS +converts old shards to the native Turso/libSQL vector format: + +- Each shard is backed up as `.db.legacy.bak` before it is rewritten. +- Progress for each shard is kept in `.db.turso-migrate.json`. +- OMMS writes the global marker `.turso-migrated` only after every shard + passes its checks. +- Do not run several OpenCode instances on the same `storagePath` during this + step. A lock file, `.turso-migrate.lock`, stops two migrations at once. +- Manual dimension migrations use `.turso-operation.lock`. Other plugin + processes refuse new memory writes until the migration finishes. + +If the migration stops part way, the next start continues from the backup. -- Each shard is backed up as `.db.legacy.bak` before rewrite -- Progress is tracked per shard in `.db.turso-migrate.json` -- A global marker `.turso-migrated` is written only after all shards verify successfully -- Do not run multiple OpenCode instances against the same `storagePath` during migration; a lock file (`.turso-migrate.lock`) prevents concurrent migration -- Manual dimension migrations use `.turso-operation.lock`; other plugin processes reject new memory writes until the migration finishes +A shard can become incompatible, for example after you change +`embeddingDimensions`. OMMS then blocks writes and leaves the original +database untouched. To fix it: -If migration is interrupted, the next startup resumes from the backup automatically. +1. Open the web UI. +2. Run the re-embed migration. It builds and checks a replacement shard. +3. OMMS swaps the replacement into place. -If a shard becomes incompatible (for example after changing `embeddingDimensions`), writes are blocked and the original database is left untouched. Use the Web UI's re-embed migration to build and verify a replacement before it is swapped into place. The previous shard remains available as `.db.pre-reembed--.bak`. +The previous shard stays available as +`.db.pre-reembed--.bak`. diff --git a/docs/using-memory.md b/docs/using-memory.md index 0aa2fc6f..8d72568d 100644 --- a/docs/using-memory.md +++ b/docs/using-memory.md @@ -1,45 +1,82 @@ # Using memory day to day -You do **not** need to ask your agent to “remember” things. With the defaults, memory builds up as you work. +You do **not** need to ask your agent to "remember" things. With the default +settings, memory builds up as you work. ## Typical daily flow 1. Install OMMS (see [Set up](../README.md#set-up)) and restart your agent. -2. Optionally choose the auto-capture model. With nothing set, auto-capture uses the session's own model. Pin one with `opencodeProvider` + `opencodeModel` (Pi: `piProvider` + `piModel`) or set an external API. Details under [Choosing the model](configuration.md#choosing-the-model). -3. Work normally. When an OpenCode session goes idle, or a Pi turn settles, auto-capture extracts memorable technical context and stores it. -4. Relevant memories are injected into context automatically. On OpenCode v2 and Pi, every prompt runs a semantic search of the project memory and adds the matches as an `` system section (never as a chat message). On OpenCode v1, the most recent memories are injected on the first message of a session (`chatMessage.injectOn`). After compaction, the session's own memories are restored. Browse or edit memories in the web UI at `http://127.0.0.1:4747`. -5. Use the `memory` tool when you want something stored or retrieved immediately (see [The memory tool](#the-memory-tool)). - -## Automatic vs manual memory - -| Approach | When it runs | What you do | -| ------------------------------------------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------------ | -| **Auto-capture** (`autoCaptureEnabled: true`, default) | After conversation turns when the session goes idle | Nothing — extraction is automatic | -| **Manual** `memory` tool / commands | On demand | `add`, `search`, `list`, `profile`, `forget`, `list-shards`, `migrate`, `export`, `import` | - -Manual search/add/list still work even if auto-capture has no provider configured. Auto-capture and user profile learning need a provider that can return structured/tool-call output. - -## Memory vs AGENTS.md / project docs - -| Store in **memory** | Store in **AGENTS.md** / static docs | -| -------------------------------------------------------------------- | ----------------------------------------------------------- | -| Project-specific decisions, bug patterns, “we tried X and it failed” | Stable rules and workflows that rarely change | -| User preferences discovered over sessions | Always-on coding conventions and process | -| Facts that should follow you across chats | Instructions every agent should see regardless of retrieval | - -Rule of thumb: if it is a lasting project instruction, put it in AGENTS.md; if it is context that grows from real work, let memory (or auto-capture) hold it. +2. Optionally, choose the model for automatic capture. See + [Choosing the model](configuration.md#choosing-the-model). + - With nothing set, capture uses the session's own model. + - To pin a model, set `opencodeProvider` and `opencodeModel` (Pi: + `piProvider` and `piModel`). + - To use an external API, set `opencodeModel` (Pi: `piModel`) to + `"external"`. +3. Work normally. Capture runs when an OpenCode session goes idle or a Pi turn + settles. It stores the useful technical context. +4. OMMS adds relevant memories to the agent's context by itself: + - On OpenCode v2 and Pi, every prompt runs a semantic search of the project + memory. The matches go in an `` system section, never in + a chat message. + - On OpenCode v1, the most recent memories go into the first message of a + session (`chatMessage.injectOn`). + - After compaction, the session's own memories are put back. +5. Browse or edit memories in the web UI at `http://127.0.0.1:4747`. +6. Use the `memory` tool to store or find something at once. See + [The memory tool](#the-memory-tool). + +## Automatic and manual memory + +| Approach | When it runs | What you do | +| ------------------------------------------------------ | -------------------------------------------------- | ------------------------------------------------------------------------------------------ | +| **Auto-capture** (`autoCaptureEnabled: true`, default) | After conversation turns, when the session is idle | Nothing. Capture is automatic. | +| **Manual** `memory` tool and commands | When you ask | `add`, `search`, `list`, `profile`, `forget`, `list-shards`, `migrate`, `export`, `import` | + +- Manual `search`, `add` and `list` work even when capture has no model set + up. +- Capture and user profile learning need a model that can return structured + or tool-call output. + +## Memory or AGENTS.md + +| Keep in **memory** | Keep in **AGENTS.md** or other fixed docs | +| ----------------------------------------------------------- | ------------------------------------------------------- | +| Project decisions, bug patterns, "we tried X and it failed" | Stable rules and workflows that rarely change | +| User preferences found over many sessions | Coding conventions and process that always apply | +| Facts that should follow you across chats | Instructions every agent must see, whatever it searches | + +A simple rule: put lasting project instructions in AGENTS.md. Let memory hold +context that grows from real work. ## How auto-capture works -After each turn, a background AI request summarizes the technical work and saves it as a memory. Greetings and chat without technical content are skipped. No special prompt from you is required. Which model it uses is set in [Choosing the model](configuration.md#choosing-the-model); with nothing configured it uses the session's own model. +- After each turn, a background model call summarises the technical work and + saves it as a memory. +- It skips greetings and chat with no technical content. +- You do not need a special prompt. +- The model it uses is set as in + [Choosing the model](configuration.md#choosing-the-model). With nothing set, + it uses the session's own model. ## User profile -The **User Profile** is a separate, cross-project summary of how you like to work (preferences, habits). It is updated on an interval (`userProfileAnalysisInterval`, default every 10 analyzed prompts), shown in the web UI’s profile view, and readable via `memory({ mode: "profile" })`. You do not populate it by hand for normal use — profile learning fills it when a provider is ready. +The **user profile** is a separate summary of how you like to work, such as +preferences and habits. It covers all your projects. + +- It updates on an interval, `userProfileAnalysisInterval`. The default is + every 10 analysed prompts. +- You can see it in the web UI's profile view, or read it with + `memory({ mode: "profile" })`. +- You do not fill it in by hand. Profile learning fills it when a model is + ready. ## Web UI -Open `http://127.0.0.1:4747` to browse the memory–prompt timeline, inspect captures, and manage the user profile. If you bind the server beyond loopback, see [Web UI HTTP Basic Auth](web-ui.md#http-basic-auth). +Open `http://127.0.0.1:4747` to browse the memory and prompt timeline, look at +captures, and manage the user profile. For the Settings page, see +[Settings page](web-ui-settings.md). If you open the server beyond loopback, +see [Web UI HTTP Basic Auth](web-ui.md#http-basic-auth). ## The memory tool @@ -57,4 +94,6 @@ memory({ mode: "export", outputPath: "./memories.json" }); memory({ mode: "import", inputPath: "./memories.json" }); ``` -See [Moving projects](moving-projects.md) for `list-shards`, `migrate`, `export` and `import`, and [Configuration](configuration.md#memory-scope) for `scope`. +See [Moving projects](moving-projects.md) for `list-shards`, `migrate`, +`export` and `import`. See [Configuration](configuration.md#memory-scope) for +`scope`. diff --git a/docs/web-ui-settings.md b/docs/web-ui-settings.md new file mode 100644 index 00000000..4e4e7c1a --- /dev/null +++ b/docs/web-ui-settings.md @@ -0,0 +1,225 @@ +# Settings page + +This guide explains each part of the web app's Settings page, in the order the page shows them. + +Open `http://127.0.0.1:4747/settings`, or select the cogwheel at the bottom of the sidebar. OpenCode serves the page while it runs. The login web app and `om-memory-system web` serve it without an agent open. See [Web UI](web-ui.md) for starting the web app, ports, and access control. + +## How saving works + +Every section saves to the global config file, `~/.config/omms/omms.jsonc`. + +- A save changes only the keys you changed. Comments, key order, and other keys stay as they are. +- The page never writes a project's `.opencode/omms.jsonc`. When a project file overrides a value, the page says so. +- If the file changed after the page loaded it, the save is refused. The page reloads the current values. Check them, then save again. +- If OMMS still reads the old `~/.config/opencode/opencode-mem.jsonc`, the first save copies it, comments included, to `~/.config/omms/omms.jsonc`. The old file is not changed. From then on OMMS reads the new file. +- Running Pi and OpenCode use the saved values from their next capture. You do not need to restart them. +- A value that fails OMMS's startup checks is refused, and the file stays unchanged. + +The page never shows a secret. For a key it shows only whether it is set and where it comes from: a literal value, an environment variable (`env://NAME`), or a file (`file://path`). + +## External API + +An external API is an OpenAI- or Anthropic-compatible endpoint that you pay for with your own key, for example a Z.ai GLM plan. Either host can use it for live capture and for importing old chats. + +The card sets four values: + +| Field | Config key | What to enter | +| -------- | ---------------- | ----------------------------------------------------------------------------------------------------------- | +| Provider | `memoryProvider` | The API style: `openai-chat`, `openai-responses`, `anthropic`, `minimax`, `orcarouter`, or `google-gemini`. | +| API URL | `memoryApiUrl` | The endpoint, for example `https://api.z.ai/api/coding/paas/v4`. Not needed for `orcarouter`. | +| Model | `memoryModel` | The model name at that endpoint, for example `glm-5.3`. Not needed for `orcarouter`. | +| API key | `memoryApiKey` | Where the key comes from. See the three key sources below. | + +Select **Save endpoint** to save the provider, URL, and model. + +### Key sources + +Choose one source, then select **Save key source**. + +- **Environment variable.** Type a variable name, such as `ZAI_API_KEY`. OMMS saves `env://ZAI_API_KEY`. +- **Key file.** Type the path of a file that holds only the key. The file must exist. OMMS saves `file://` and the path. +- **Save key to a private file.** Type a file name and paste the key. OMMS writes it to `~/.config/omms/secrets/.key` and saves its `file://` path. + +What happens to a pasted key: + +- The file can be read only by you: mode `600` in a folder with mode `700` on macOS and Linux, and a user-only access list on Windows. +- The key is never written to `omms.jsonc`, the log, or any page response. The page does not show it again. +- If a key file with that name exists, the page asks before it replaces it. +- On a server bound to a network address without Basic Auth, the page refuses to save a pasted key. + +The card also shows: + +- **Key source.** The saved source type and the variable name or file path. +- **Resolves in the web app.** Whether the key can be read by the process that serves this page. + +A login web app does not load your shell profile, such as `~/.zshrc`. A variable set only there does not reach it, and the card says the key does not resolve. Use a key file for the login web app. A key file works in every OMMS process. + +### Test + +Select **Test** to send one short request with the saved settings and a small output limit. The card shows `Test call succeeded` with the model, or `Test call failed` with the error. The error never contains the key. + +## Models + +This section chooses the model for automatic capture and profile learning, one card for each host. Automatic capture is the summary OMMS writes after each exchange. Profile learning builds your user profile from your prompts. + +Each card offers three choices: + +- **Session model.** Use the model of the session you are working in. Saves `inherit` to `opencodeModel` or `piModel`. +- **Manual model.** Use one model from the host's own sign-in. Saves the provider and model, for example `piProvider: "openai-codex"` and `piModel: "gpt-5.6-luna"`. When the page can list the host's signed-in models, pick one. Otherwise type `provider/model`. +- **External API.** Use the endpoint from the External API card. Saves `external`. You can choose it only when the external API is fully set up. Until then the card lists the missing settings, for example `External API needs: memoryApiUrl`. + +Each card also shows, without letting you change it: + +- **Effective model.** The model the capture rule would use now. +- **External API fallback.** The external model used when a manual host model fails. +- A warning when a project config overrides the host's model. + +The rule for choosing a model is the same on both hosts. See [Configuration: Choosing the model](configuration.md#choosing-the-model). + +## Capture diagnostics + +Every capture attempt writes one metadata record: host, model, sizes, stop reason, outcome, and failure reason. It never contains conversation text. This section shows those records. + +- **Time range.** Show the last 24 hours, 7 days, 30 days, or 90 days. +- **Outcomes by model.** Saved, skipped, and failed counts for each host and model. +- **Failure reasons.** The most common reasons for failed attempts. +- **Recent attempts.** One row for each attempt, with its model, stop reason, sizes, duration, and outcome. +- **Attempt retention (days).** How long OMMS keeps these records. The default is 30 days. + +### Capture traces + +A trace is the full prompt and reply of each capture attempt, saved to `~/.omms/traces/` for debugging. + +- **Save capture traces** turns tracing on or off. Tracing is off by default. +- Traces can contain conversation content. OMMS removes text inside `` tags and common API key formats first. +- Trace files can be read only by you, and are deleted after **Trace retention (days)**. The default is 7 days. +- **Trace files** lists each day's file. **View** opens one. **Delete** removes it. +- A project config can turn tracing off but never on. On a network-bound server without Basic Auth, the page refuses to turn tracing on. + +Turn tracing on only while you debug a problem. + +## Health + +This section runs checks on the parts OMMS needs, and shows a pass, warning, or fail line for each. + +- **Run checks.** Check the config files, the memory store, the embedding model, the web binding, the model selection, and the capture failure rate over the last 24 hours. +- **Run checks and test models.** Also send one short fixed prompt to each host's capture model. + +A session model can be tested only from inside an open session. The OpenCode web server cannot call a Pi session model. + +## Import and backfill + +Use this section to import past chats by hand. A backfill is an import of old chats; the next section runs one automatically. + +1. Choose the **History host**: Pi or OpenCode. +2. Select **List sessions**. The list shows each session's date, ID, project folder, and how the folder was found. It never shows prompts or replies. +3. Tick sessions, or select **Select all matching** to include every page. +4. Select **Preview (dry run)**. It counts the exchanges that would be imported. It makes no model calls and writes nothing. +5. Choose the **Import model**: a connected OpenCode model, or **Saved external API**. +6. Select **Start import**. + +While an import runs, progress updates every second. **Cancel after current unit** stops at a safe point. A later run imports the rest. The page, the terminal commands, and the automatic backfill share one record of finished work, called the ledger, so nothing is imported twice. + +How the folder was found (**Resolved by**): + +- **recorded.** The folder the session was recorded in still exists. +- **mapped.** A directory map sends the session to another folder. +- **project root.** OpenCode only. The recorded folder is gone, so OMMS uses the project folder OpenCode recorded. +- **missing.** No folder could be found. The session cannot be imported until a directory map resolves it. + +The page keeps the preview and the import on the same sessions: + +- If a session appears or disappears after you list them, the page asks you to refresh the list. +- Turns written after you listed the sessions wait for the next run. The report says how many. +- If you change the scope, project, source, or directory maps, the page asks you to refresh the list first. + +### Advanced options + +Select **Advanced options** to see these fields. + +| Option | Effect | +| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | +| Source | Pi: a sessions folder or one `.jsonl` file. OpenCode: a database file. The default is the host's own store. | +| Scope | **Current project** or **All projects**. | +| Project directory | The project for the current-project scope. | +| Prompt date from, Prompt date to | Import only turns from these whole days, in your browser's time zone. | +| Directory maps | `old=new`, one per line. They apply to this import and win over saved maps for the same folder. | +| Profile batch size | Prompts per profile analysis request. | +| Force reimport, Skip memories, Skip profile | Import finished units again, or skip one of the two import steps (memories or the user profile). | + +Other details: + +- **Browse** lists one folder at a time. It works only when the server is bound to `127.0.0.1`. On a network address, type the path instead. +- The server reads files in place and never changes them. +- While OpenCode runs, the server reads a private copy of its database. It first checks that the temporary folder has space for the copy. + +**Model readiness.** A real import needs a connected OpenCode model, when OpenCode serves the page, or a fully set up external API. A web import cannot use a Pi sign-in. `Configured, not tested.` means the settings are present. Use **Test models in Health** to try a call. + +## Automatic import + +An automatic import, or backfill, imports each host's old chats in the background. It starts about 30 seconds after Pi or OpenCode starts. It makes model calls, so it costs what your model costs. + +- **Import past chats automatically** turns backfill on or off for both hosts (`autoBackfill`). Turning it off stops a running backfill after its current exchange. + +For each host: + +- **Backfill model.** The model that imports old chats. + - **Same as live capture** uses the model from the Models section. Saves `inherit`. + - **External API** uses the External API card. Saves `external`. It can run from the web app with no host open. + - A listed or typed `provider/model` uses another model from the host's sign-in, for example a cheaper one. Live capture keeps its own model. +- **State.** Not started, running, paused, stopped, done, or failed. For a running import it also shows where it started: automatically, from the web page, from the terminal, or from a slash command. +- **Progress.** A bar, the percentage, done out of total, and the minutes left. Minutes left comes from the recent rate. It shows as unknown until about a minute of progress. +- **Counts.** Imported, skipped, failed, and pending exchanges, and sessions whose folder cannot be found. When there are any, a link goes to [Directory maps](#directory-maps). +- **Model**, **Cutoff**, and the last error. The cutoff is fixed at the first backfill. Later turns are saved by live capture instead. + +The page refreshes these values every 3 seconds while an import runs. Progress counts only exchanges that need a model call. Exchanges already in the ledger are left out. + +Only one import runs for each host at a time. This includes a backfill, a page import, a terminal import, and a slash command. A second one is refused with `A Pi import is already running`. + +### Run now, Pause, and Resume + +- **Run now** starts the host's backfill at once. It uses the same cutoff, maps, model rule, and ledger as the automatic run. +- **Pause** stops the run after its current exchange, including a run in another process. A paused backfill does not start again when the host starts. +- **Resume** clears the pause and starts the run. It continues from the ledger. + +Run now and Resume run inside the web app. Without Pi or OpenCode open, they need the host's backfill model to use the external API. Otherwise the page says: `Open Pi, or choose the external API for Pi's backfill`. When OpenCode serves the page, OpenCode's backfill can also use OpenCode's connected models. + +## Directory maps + +A directory map tells OMMS which project a folder belongs to. Use it when chats were recorded in a folder that no longer exists, such as a deleted git worktree. + +- **Saved maps** lists the maps in `importPathMaps`. **Remove** marks one for removal, and **Keep** undoes that. +- **Unresolved directories** lists, for each host, the folders the latest session listing or import could not find, with a session count. +- Where OMMS can find one, the target box holds a suggested existing folder: + 1. The main repository of a deleted worktree. For `~/code/app-feat-x` or `~/workspaces/app/feat-x`, it suggests `~/code/app`. + 2. For OpenCode, the project folder that OpenCode recorded for the session. OMMS reads OpenCode's database without writing to it. +- `No suggestion found.` means there is no candidate, for example for an old temporary folder. Type a target, or leave the folder unmapped. + +To save maps: + +1. Check or edit the target folder. +2. Tick **Use this map**. +3. Select **Save maps**. + +Maps apply to the next import or backfill run, on every surface. A terminal `--map` for the same folder wins for that run. A map to a folder that does not exist leaves its sessions unresolved. Memories already imported through a map stay when you remove it. + +The list fills when you list sessions under Import and backfill with All projects, or when an import or backfill runs. + +## Web app + +- **Start web app at login** (`webServerAutoStart`) installs or removes a login item that starts the web app when you sign in. The change applies at the next Pi or OpenCode start. To apply it now, run `om-memory-system web install` or `om-memory-system web uninstall`. +- **Login item** shows whether the item is installed, unsupported on this system, or missing a Node or Bun runtime. +- **Running version** is the OMMS version that serves this page. **Global command** is the version of `om-memory-system` on the web app's `PATH`, or `not installed globally`. +- When the two versions differ, the section warns and shows the upgrade command: `npm i -g om-memory-system@latest`. + +A global install is optional but recommended. With it, the login item and the terminal commands run without `npx`. See [CLI: Global install](cli.md#global-install-optional-recommended). + +## Log + +This section shows the latest lines of `~/.omms/omms.log`. + +- **Capture attempts only** shows only capture attempt records. +- **Refresh** reads the log again. +- **Copy log path** copies the file path. + +The log holds sizes, identifiers, and codes. It never holds prompts, replies, or keys. diff --git a/docs/web-ui.md b/docs/web-ui.md index 7f2c2009..183558d1 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -1,74 +1,94 @@ # Web UI -OMMS serves a memory explorer at `http://127.0.0.1:4747`. OpenCode can serve it while running. The per-user login item can keep it available without an open agent; `om-memory-system web` also starts it manually. Pi does not serve the page. If OpenCode starts while the standalone server owns the port, it uses the running server. Use the page to browse memories, inspect captures, edit entries and manage your profile. +OMMS serves a web app at `http://127.0.0.1:4747`. Use it to browse and edit memories, manage your user profile, and change settings. -The sidebar footer shows the current language as EN, ZH, or AR. Select the code to open the language menu, then choose English, Chinese, or Arabic. Opening or closing the menu keeps the current language. The choice is saved for the next visit. +## Starting the web app -## Settings page +Three things can serve the page. They all use the same port, settings, and memory store. + +- **OpenCode** starts it while OpenCode runs. +- **The login item** starts it when you sign in to your computer. Turn it on with `om-memory-system web install`, or on the Settings page. See [CLI: Web app commands](cli.md#web-app-commands). +- **`om-memory-system web`** starts it by hand in the terminal. Press Ctrl+C to stop it. + +Pi does not serve the page. If OpenCode starts while another OMMS process serves the page, OpenCode uses that one instead of starting a second server. -Open `http://127.0.0.1:4747/settings` or select the cogwheel in the sidebar footer. OpenCode or the standalone web app serves the page. Pi uses the same global config and store. +A global install of the terminal command is optional but recommended. With it, the login item and the commands run without `npx`: -- **Models:** Choose the session model or a signed-in model for each host. Manual mode shows a model picker when the server can list models. Type `provider/model` when it cannot. The cards show the effective model and the read-only external API fallback. Project overrides take precedence. The page shows whether credentials are configured, without showing their values. An isolated preview cannot list OpenCode models because it has no OpenCode client; the plugin-hosted page can. -- **Capture diagnostics:** Select 1, 7, 30, or 90 days to see outcomes, failure reasons, and recent attempts. You can set attempt and trace retention, enable tracing, and inspect or delete trace files. Traces can contain conversation content after redaction. Turn tracing on only when you need it. -- **Health:** Run config, store, embedding, web binding, model-selection, and capture failure checks. An optional model test sends a fixed short prompt. A session model needs an active host session to test; the OpenCode server cannot call a Pi session model. -- **Automatic import:** Turn background history import on or off for both hosts. Choose a separate `provider/model` for Pi and OpenCode, or inherit each host's model rule. The page shows each host's state, cutoff, pending count and last error. It polls while an import runs. A first run starts about 30 seconds after a host starts and may make model calls. See [Configuration](configuration.md#automatic-history-import-and-login-web-app). -- **Web app:** Turn the per-user login item on or off. See whether it is installed, unavailable or unsupported. The setting takes effect at the next host start; use [the CLI](cli.md#web-app-commands) to apply it immediately. -- **Import and backfill:** For a manual import, see [Importing from the page](#importing-from-the-page). -- **Log:** View the most recent OMMS log lines. Filter for capture attempts or copy the log path. +```bash +npm i -g om-memory-system # or: bun add -g om-memory-system +om-memory-system --version +``` + +## Language -Saves change only the global config. Existing comments and unrelated keys remain. If the file changed while the page was open, review the reloaded settings before saving again. A first save from a legacy-only install copies that file into `~/.config/omms/omms.jsonc` and leaves the old file unchanged. New capture and profile work in running hosts picks up the saved settings without a restart. +The sidebar footer shows the current language: EN, ZH, or AR. Select it to choose English, Chinese, or Arabic. The page remembers your choice for the next visit. -## Importing from the page +## Settings page -1. Choose Pi or OpenCode and select **List sessions**. The list shows each session's date, ID, project directory, and how that directory was found. It never shows prompts or replies. The default is the current project. -2. Tick sessions, or choose **Select all matching**, which covers every page. Untick sessions to leave them out. -3. Select **Preview (dry run)**. It reports the sessions and turns that would be imported and makes no model calls or writes. -4. Select **Start import**. Progress updates every second. **Cancel after current unit** stops at a safe point, and a later run imports the rest. The page and the CLI share one ledger, so nothing is imported twice. +Open `http://127.0.0.1:4747/settings`, or select the cogwheel in the sidebar footer. The page has these sections: -What the page guarantees: +- **External API.** Set up your own OpenAI- or Anthropic-compatible endpoint and its key, and test it. +- **Models.** Choose the capture model for each host. +- **Capture diagnostics.** See capture outcomes and failures, and manage debug traces. +- **Health.** Check that each part of OMMS works. +- **Import and backfill.** Import past chats by hand. +- **Automatic import.** Control the background import of past chats, watch its progress, and run, pause, or resume it. +- **Directory maps.** Tell OMMS where chats from deleted folders belong. +- **Web app.** Control the login item and check the installed version. +- **Log.** Read the latest OMMS log lines. -- **The preview and the import use the same sessions.** If a new session appears after you list, or a selected session disappears, the page asks you to refresh the list instead of changing the selection. -- **Newer turns wait for the next run.** Turns written after you listed the sessions are held back, and the report says how many. List the sessions again and import to include them. Nothing is lost or duplicated. -- **Changing the scope, project, source, or directory maps** asks you to refresh the list before you can preview or import. +[Settings page](web-ui-settings.md) explains every part in detail. -**Advanced options** hold the source, scope, project, prompt dates, directory maps, profile batch size, and the force and skip switches. +## Opening the page through a terminal proxy -- **Source.** Pi takes a sessions folder or one `.jsonl` session file. OpenCode takes a database file, which contains many sessions. Enter an absolute path, including one on a mounted volume. **Browse** lists one folder at a time, and only when the server is bound to loopback. On a network bind, enter the path instead. The browser never uploads files; the server reads them in place and never changes them. -- **Prompt dates** are inclusive whole days in your browser's time zone. They filter the turns inside each session, not the session list. Turns without a timestamp are always included and counted in the preview. -- **Directory maps** (`old=new`, one per line) take precedence over the recorded directory. Sessions whose directory no longer exists appear under **All projects** with the recorded path, and you can import them once a map resolves them. In the current-project view the page shows how many there are. -- **Large OpenCode databases.** While OpenCode is running, its database has a write-ahead log, so the server reads a private copy. It checks that the temporary folder has room for the database, its log, and a margin before copying, and it says how much space is needed if not. The listing, preview, and import share one copy, which is removed after 30 minutes idle and when OpenCode stops. A copy of a database on another volume, or over 1 GB, is tried once; if OpenCode changes the database during that copy, quit OpenCode or choose a checkpointed backup. +Some terminals send local addresses through their own proxy. For example, Orca opens links as `*.orca.localhost` addresses, not the `http://127.0.0.1:4747` address OMMS prints. Both reach the same server. If the proxied address does not load or asks for a token, open the printed `127.0.0.1` address. -**Model readiness.** A real import needs a model that OpenCode is connected to (when OpenCode hosts the page), or a complete saved external API (`memoryProvider`, `memoryModel`, `memoryApiUrl`, and `memoryApiKey`). Pi sign-ins cannot run a web import, because the web server cannot call Pi models. The page checks readiness inside its running process, so an `env://` key that exists only in another shell shows as missing. "Configured, not tested" means the settings are present; use **Health** to test a call. A preview needs no model. Pi previews and imports need the Pi SDK (`@earendil-works/pi-coding-agent`) installed in the web server's package environment; the page says so when it is missing. +## Network access -## Network binding +Keep `webServerHost` on `127.0.0.1` unless you mean to open the web app to other computers. -Keep `webServerHost` on `127.0.0.1` unless you intentionally expose the UI. Binding to `0.0.0.0` (or any non-loopback host) requires `webServerApiToken`; all `/api/*` requests must then send `Authorization: Bearer ` or `X-Omms-Token` (the legacy `X-Opencode-Mem-Token` header is still accepted). Open the UI with `?apiToken=` so the browser stores and sends it. +A non-loopback host, such as `0.0.0.0`, needs `webServerApiToken`. Loopback means your own computer only. + +- Every `/api/*` request must then send `Authorization: Bearer ` or the `X-Omms-Token` header. The old `X-Opencode-Mem-Token` header still works. +- Open the page once with `?apiToken=`. The browser stores the token and sends it. ## HTTP Basic Auth -When `webServerHost` is set to anything other than loopback (for example `0.0.0.0`), the web UI is reachable by anyone on the network. To keep your memories off the LAN, gate the web server with HTTP Basic Auth via the same config file used for everything else: +On a network address, anyone on the network can reach the web app. Add a password with HTTP Basic Auth in the global config: ```jsonc { - "webServerHost": "0.0.0.0", // optional: reach the UI from the LAN + "webServerHost": "0.0.0.0", // optional: reach the web app from your network "webServerAuthPassword": "pick-a-strong-one", - "webServerAuthUsername": "admin", // optional, defaults to the current OS user + "webServerAuthUsername": "admin", // optional, defaults to your user name } ``` -| Field | Default | Effect | -| ----------------------- | ----------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `webServerAuthPassword` | _(empty)_ | When set, the server demands HTTP Basic Auth credentials on every request. Leave empty to keep the open-by-default behavior. | -| `webServerAuthUsername` | OS user (`$USER`) | Username required by the Basic Auth challenge. | +| Field | Default | Effect | +| ----------------------- | -------------------------- | ----------------------------------------------------------------------- | +| `webServerAuthPassword` | empty | When set, every request needs the user name and password. Empty is off. | +| `webServerAuthUsername` | your operating system user | The user name the browser asks for. | + +`webServerAuthPassword` accepts the same formats as `memoryApiKey`: + +- a literal value, which is fine on a personal computer; +- `env://SOME_VARIABLE`, read from the environment when OMMS starts; +- `file:///path/to/secret`, read from a file. Use `chmod 600`; OMMS warns when others can read the file. -`webServerAuthPassword` accepts the same secret formats as `memoryApiKey`: +The browser asks for the user name and password once, and forgets them when you close all its windows. -- a literal string (simple, fine for personal machines), -- `env://SOME_ENV_VAR` to pull the value from the environment at startup, -- `file:///path/to/secret` to read it from a file (`chmod 600` recommended — the plugin will warn if the file is world-readable). +- OMMS compares the password in constant time. +- The "unauthorised" reply is never cached. +- With Basic Auth on, other tools on your network can use the API after they sign in. -The browser will pop its native Basic Auth dialog and remember the credentials for the current session; closing all browser windows discards them, so reopening the browser requires signing in again. Credentials are compared with a constant-time check, and the unauthenticated 401 response carries `Cache-Control: no-store` so no intermediate cache will replay it. CORS is also relaxed once auth is on, so other tools on the same LAN can talk to the API after authenticating. +Some Settings actions are refused on a network address without Basic Auth: turning on capture traces and saving a pasted API key. ## Re-embedding -Dimension migrations generate every new embedding first, import them into a temporary indexed shard, verify the row count, and only then replace the original file. Failed migrations leave the source shard untouched. +Changing the embedding model's size migrates each memory store file safely: + +1. OMMS makes every new embedding first. +2. It writes them into a temporary file and checks the row count. +3. Only then does it replace the original file. + +A failed migration leaves the original file unchanged. From 148658dd17a7277a834f495e57462241d8a2851b Mon Sep 17 00:00:00 2001 From: Aizat Hawari Date: Mon, 28 Sep 2026 10:07:22 +0100 Subject: [PATCH 2/4] docs(openspec): propose moving OpenCode model code into its adapter --- .../.openspec.yaml | 2 + .../design.md | 74 +++++++++++++++++++ .../proposal.md | 37 ++++++++++ .../specs/host-neutral-memory-core/spec.md | 22 ++++++ .../tasks.md | 24 ++++++ 5 files changed, 159 insertions(+) create mode 100644 openspec/changes/move-opencode-model-code-to-adapter/.openspec.yaml create mode 100644 openspec/changes/move-opencode-model-code-to-adapter/design.md create mode 100644 openspec/changes/move-opencode-model-code-to-adapter/proposal.md create mode 100644 openspec/changes/move-opencode-model-code-to-adapter/specs/host-neutral-memory-core/spec.md create mode 100644 openspec/changes/move-opencode-model-code-to-adapter/tasks.md diff --git a/openspec/changes/move-opencode-model-code-to-adapter/.openspec.yaml b/openspec/changes/move-opencode-model-code-to-adapter/.openspec.yaml new file mode 100644 index 00000000..ee7c5448 --- /dev/null +++ b/openspec/changes/move-opencode-model-code-to-adapter/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-28 diff --git a/openspec/changes/move-opencode-model-code-to-adapter/design.md b/openspec/changes/move-opencode-model-code-to-adapter/design.md new file mode 100644 index 00000000..a3512141 --- /dev/null +++ b/openspec/changes/move-opencode-model-code-to-adapter/design.md @@ -0,0 +1,74 @@ +# Design + +## Context + +What calls the OpenCode-specific modules in `src/services/ai/` today: + +| Module | Called from | +| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `opencode-provider.ts` | `src/index.ts`, `adapters/opencode/user-prompt.ts`, `services/settings-models.ts`, `opencode-provider-loader.ts` | +| `opencode-provider-loader.ts` | `src/index.ts`, `src/v2/plugin.ts`, `adapters/opencode/*`, `importer/web-import-jobs.ts`, `services/user-memory-learning.ts`, `services/user-profile/user-profile-manager.ts`, `services/user-profile/ai-cleanup.ts`, `profile-llm-client.ts`, `opencode-import-models.ts` | +| `opencode-import-models.ts` | `importer/settings-health.ts`, `importer/web-import-jobs.ts`, `adapters/opencode/import-command.ts`, `adapters/opencode/backfill-models.ts` | +| `opencode-sdk-client.ts` | `opencode-provider.ts` | +| `opencode-diagnostics.ts` | `opencode-provider.ts` | +| `opencode-host-config.ts` | `src/index.ts` | +| `internal-capture-sessions.ts` | `src/index.ts`, `importer/opencode-reader.ts`, `opencode-provider.ts` | +| `profile-llm-client.ts` | `services/user-memory-learning.ts`, `services/user-profile/user-profile-manager.ts` | + +- `src/services/user-memory-learning.ts` is OpenCode's profile learning loop. It takes OpenCode's `PluginInput`, and only `src/index.ts` calls it. +- Pi learns the profile through the shared `analyzeProfile` in `src/core/profile-analysis.ts`, with a `ModelPort` built by `adapters/pi/profile.ts`. +- `user-profile-manager.ts` and `ai-cleanup.ts` check `resolveOpencodeHostModel(CONFIG)` and then call OpenCode's `generateStructuredOutput` themselves. +- `src/importer/backfill-controls.ts` already has a registry, `registerHostBackfillModels`, that the OpenCode adapter fills at start-up. +- The boundary tests check the capture files for host SDKs, and check `src/core`, `src/services`, `src/types`, and `src/importer` for adapter paths. They do not check the rest of `src/services` for SDK imports. + +## Goals / Non-Goals + +**Goals:** + +- No file in `src/core/`, `src/services/`, or `src/types/` imports a host SDK or an adapter. +- Shared profile code calls models only through `ModelPort`. +- A boundary test enforces this for every shared file. +- Behaviour on both hosts stays exactly the same. + +**Non-Goals:** + +- Merging OpenCode's profile learning loop into the shared `analyzeProfile` path that Pi uses. It is worth doing, but it changes behaviour and needs its own change. +- Changing the live-model rule, capture diagnostics, or any config, API, or data format. +- Changing how the OpenCode V1 and V2 plugins load. + +## Decisions + +### D1. The OpenCode modules move into the OpenCode adapter unchanged + +`opencode-provider.ts`, `opencode-sdk-client.ts`, `opencode-provider-loader.ts`, `opencode-host-config.ts`, `opencode-import-models.ts`, `opencode-diagnostics.ts`, `profile-llm-client.ts`, and `user-memory-learning.ts` move to `src/adapters/opencode/` with `git mv`. Only their import paths change. This keeps the history of each file and makes the review a list of path changes. + +### D2. Internal session titles move to the importer + +`INTERNAL_CAPTURE_SESSION_TITLES` and `isInternalCaptureSessionTitle` describe OpenCode's history format: the titles omms gives its own capture sessions. The OpenCode history reader needs them without OpenCode running. They move to `src/importer/opencode-internal-sessions.ts`. The in-process tracking of live internal sessions (`isTrackedInternalCaptureSession`) moves with the adapter. + +### D3. Shared profile code takes a ModelPort + +`user-profile-manager.ts` and `ai-cleanup.ts` gain an optional `ModelPort` argument on the functions that call a model today. When the host passes one, they call `model.complete(system, prompt)` with the same prompt and schema as now. When it does not, they use the external API as today. The OpenCode adapter builds the `ModelPort` from its host model with a new `adaptOpencodeProfileModel`, in the same way as `adaptPiProfileModel`. The web server's profile cleanup endpoint passes the port registered by the host serving it (D4), or none in the standalone web app. + +Alternative: move both files into the adapter. Rejected, because Pi and the web server use them too. + +### D4. The importer receives OpenCode's models by registration + +`src/importer/backfill-controls.ts` generalises its registry. A host registers one object with its backfill model resolver, its import model factory, and its profile `ModelPort` factory. `web-import-jobs.ts` and `settings-health.ts` use the registered import model factory instead of importing `opencode-import-models.ts`. With nothing registered, as in the standalone web app, they report that OpenCode models are unavailable, exactly as they do now when no OpenCode client exists. + +Alternative: let the importer import the adapter dynamically. Rejected by ADR-011. + +### D5. Model listing for the Settings page moves to the importer + +`settings-models.ts` lists each host's signed-in models for the page. It reads host data without the host running, which is the importer's role (ADR-011). It moves to `src/importer/settings-models.ts`. It loads the Pi SDK and the OpenCode client dynamically, as today. + +### D6. One boundary test covers every shared file + +`tests/pi-adapter-boundary.test.ts` gains a check that walks every `.ts` file under `src/core`, `src/services`, and `src/types`. It fails on any `@opencode-ai/` or `@earendil-works/` specifier in an `import`, `import type`, `export … from`, or `import()` call. For `src/importer`, it allows host SDKs only as dynamic `import()` in a named list of reader modules. The failure message names each file. + +## Risks / Trade-offs + +- [OpenCode capture or profile learning breaks after the move] → The moves are path-only (D1). The existing OpenCode capture, profile learning, V2 plugin, and bundle-boundary tests must pass unchanged, apart from mock paths. A manual check in OpenCode confirms one capture and one profile run. +- [Tests mock the moved modules by path with `mock.module`] → Update each mock path in the same commit as the move. A mock of the old path fails silently, so search the tests for every old path. +- [The V1 plugin bundle grows or loads the SDK too early] → `tests/plugin-bundle-boundary.test.ts` checks it. Keep every SDK load behind dynamic `import()`. +- [Two profile learning loops stay] → This change leaves that as it is, by design (Non-Goals). Record it as a follow-up. diff --git a/openspec/changes/move-opencode-model-code-to-adapter/proposal.md b/openspec/changes/move-opencode-model-code-to-adapter/proposal.md new file mode 100644 index 00000000..13ca92a9 --- /dev/null +++ b/openspec/changes/move-opencode-model-code-to-adapter/proposal.md @@ -0,0 +1,37 @@ +# Proposal + +## Why + +OMMS's shared layers still hold OpenCode's own model code. `src/services/ai/` contains eight OpenCode-specific modules, and shared services call OpenCode models directly through them. This breaks the rule in CLAUDE.md and ADR-011 that shared code never depends on one host. It also goes against the existing requirement that the capture and profile pipeline depend on a provider-neutral port, not on OpenCode or Pi model APIs. Only the capture files are guarded by a boundary test today, which is how this went unnoticed. + +## What Changes + +- Move the OpenCode-specific modules from `src/services/ai/` into `src/adapters/opencode/`: + - `opencode-provider.ts`, `opencode-sdk-client.ts`, `opencode-provider-loader.ts`, `opencode-host-config.ts`, `opencode-import-models.ts`, `opencode-diagnostics.ts`, and `profile-llm-client.ts` + - `internal-capture-sessions.ts`, except the list of internal session titles. The OpenCode history reader needs that list, so it moves to `src/importer/`. +- Move OpenCode's profile learning loop, `src/services/user-memory-learning.ts`, which takes OpenCode's `PluginInput`, into `src/adapters/opencode/`. +- Change `user-profile-manager.ts` and `ai-cleanup.ts` in `src/services/user-profile/` to call a model through the existing `ModelPort` (`src/core/profile-analysis.ts`), which the host passes in. They no longer resolve or call OpenCode models themselves. +- Let the OpenCode adapter register its import models with the importer, the same way it registers its backfill models today (`registerHostBackfillModels`). `web-import-jobs.ts` and `settings-health.ts` then use the registered models instead of importing OpenCode code. +- Move host model listing for the Settings page (`src/services/settings-models.ts`) into `src/importer/`. The importer already holds the code that reads host data without the host running. +- Extend the boundary tests: + - No file in `src/core/` or `src/services/` imports `@opencode-ai/*` or `@earendil-works/*`, statically or dynamically, or any adapter. + - No file in `src/importer/` imports an adapter. The importer loads host SDKs only with dynamic `import()`, and only in its named reader modules. +- No user-visible behaviour changes. Capture, profile learning, AI cleanup, imports, backfills, and the Settings page work as before on both hosts. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +- `host-neutral-memory-core`: the requirement "Provider-specific extraction is behind a narrow port" gains an enforceable rule. Shared code does not import host SDKs or adapter modules, and a boundary test checks every shared file. + +## Impact + +- **Code:** `src/services/ai/` (eight modules move out), `src/services/user-memory-learning.ts` (moves), `src/services/user-profile/user-profile-manager.ts` and `ai-cleanup.ts` (take a `ModelPort`), `src/services/settings-models.ts` (moves to `src/importer/`), `src/importer/web-import-jobs.ts`, `settings-health.ts`, `opencode-reader.ts`, `src/index.ts`, `src/v2/plugin.ts`, and `src/adapters/opencode/*` (new homes and import paths). +- **Tests:** the three boundary tests, plus the existing tests that mock the moved modules by path (`mock.module`), which need their paths updated. +- **Docs:** `docs/shared-core.md`, `docs/opencode-adapter.md`, and CLAUDE.md's architecture table. +- **Risk:** OpenCode live capture, profile learning, and AI cleanup are rewired. The existing OpenCode capture, profile, and v2 plugin tests, plus a manual check in OpenCode, cover this. +- **No data, config, or API change.** No release note beyond a refactor entry. diff --git a/openspec/changes/move-opencode-model-code-to-adapter/specs/host-neutral-memory-core/spec.md b/openspec/changes/move-opencode-model-code-to-adapter/specs/host-neutral-memory-core/spec.md new file mode 100644 index 00000000..a4130d18 --- /dev/null +++ b/openspec/changes/move-opencode-model-code-to-adapter/specs/host-neutral-memory-core/spec.md @@ -0,0 +1,22 @@ +## MODIFIED Requirements + +### Requirement: Provider-specific extraction is behind a narrow port + +The shared capture/profile pipeline SHALL depend on a provider-neutral structured-extraction interface rather than directly on OpenCode or Pi model APIs. Shared code in `src/core/`, `src/services/`, `src/types/`, and `src/importer/` SHALL NOT import a host adapter module. Shared code in `src/core/`, `src/services/`, and `src/types/` SHALL NOT import a host SDK (`@opencode-ai/*` or `@earendil-works/*`), statically or dynamically. The importer MAY load a host SDK with dynamic `import()` only in its named modules that read host data without the host running. A host SHALL give shared code its model calls through the `ModelPort` and capture provider ports, or by registering them with the importer. A boundary test SHALL check every shared file for these rules. + +#### Scenario: Pi extraction fails + +- **WHEN** the Pi provider bridge cannot complete a structured extraction request +- **THEN** automatic capture MAY fail or defer for that work unit +- **AND** local manual memory search, list, add, and delete operations SHALL remain available + +#### Scenario: Shared code imports a host SDK + +- **WHEN** a file in `src/services/` imports `@opencode-ai/sdk` +- **THEN** the boundary test SHALL fail and name the file + +#### Scenario: Profile learning on OpenCode + +- **WHEN** OpenCode runs profile learning or AI profile cleanup with an OpenCode host model +- **THEN** the shared profile code SHALL call the model through a `ModelPort` that the OpenCode adapter passes in +- **AND** the result SHALL be the same as before the move diff --git a/openspec/changes/move-opencode-model-code-to-adapter/tasks.md b/openspec/changes/move-opencode-model-code-to-adapter/tasks.md new file mode 100644 index 00000000..acb1dc29 --- /dev/null +++ b/openspec/changes/move-opencode-model-code-to-adapter/tasks.md @@ -0,0 +1,24 @@ +## 1. Boundary test first + +- [ ] 1.1 Add the check from D6 to `tests/pi-adapter-boundary.test.ts`: every file in `src/core`, `src/services`, and `src/types` has no `@opencode-ai/` or `@earendil-works/` specifier in any import form, and `src/importer` loads them only by dynamic `import()` in named reader modules; confirm it fails today and lists the current offending files + +## 2. Move the OpenCode modules + +- [ ] 2.1 `git mv` the seven `src/services/ai/opencode-*.ts` and `profile-llm-client.ts` modules into `src/adapters/opencode/`, and update every import and `mock.module` path; verify with `bun run typecheck` and the OpenCode capture, V2 plugin, and bundle-boundary tests +- [ ] 2.2 Split `internal-capture-sessions.ts`: titles to `src/importer/opencode-internal-sessions.ts` (D2), live tracking to the adapter; verify with the OpenCode reader tests and the internal-session capture tests +- [ ] 2.3 `git mv` `src/services/user-memory-learning.ts` to `src/adapters/opencode/profile-learning.ts` and update `src/index.ts` and its tests; verify with the existing profile learning tests + +## 3. Ports for shared code + +- [ ] 3.1 Add `adaptOpencodeProfileModel` in `src/adapters/opencode/`, matching `adaptPiProfileModel`; verify with a unit test that it calls OpenCode's structured output with the same prompt +- [ ] 3.2 Give `user-profile-manager.ts` and `ai-cleanup.ts` an optional `ModelPort` (D3) and remove their OpenCode imports; verify with tests that the port is called when passed and the external API is used when not, with the same prompt and output as before +- [ ] 3.3 Generalise the host registry in `backfill-controls.ts` (D4) to carry the backfill resolver, the import model factory, and the profile `ModelPort` factory; register them from the OpenCode adapter and the Pi extension; verify with registry tests on both hosts +- [ ] 3.4 Switch `web-import-jobs.ts` and `settings-health.ts` to the registered import models; verify the web import and Health tests, including the standalone web app reporting OpenCode models as unavailable +- [ ] 3.5 Move `src/services/settings-models.ts` to `src/importer/settings-models.ts` (D5); verify with the Settings models tests + +## 4. Close out + +- [ ] 4.1 Confirm the boundary test from 1.1 now passes with no allowances beyond the named importer readers +- [ ] 4.2 Update `docs/shared-core.md`, `docs/opencode-adapter.md`, and CLAUDE.md's architecture table; record the remaining follow-up (one profile learning loop for both hosts) in the change's design and the ADR-011 consequences +- [ ] 4.3 Run `bun run ci:local` and `bun run check:package` +- [ ] 4.4 Manual check in OpenCode V1 and V2: one live capture, one profile learning run, and one AI profile cleanup succeed, and the capture attempt log shows the host model From e71e8f11d0d53010fa0d752d99272397e2e36a7e Mon Sep 17 00:00:00 2001 From: Aizat Hawari Date: Mon, 28 Sep 2026 10:11:51 +0100 Subject: [PATCH 3/4] docs(openspec): archive external-api-backfill-maps-progress --- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/auto-backfill/spec.md | 0 .../specs/host-neutral-memory-core/spec.md | 0 .../specs/import-directory-maps/spec.md | 0 .../specs/import-progress/spec.md | 0 .../specs/web-autostart/spec.md | 0 .../specs/web-settings/spec.md | 0 .../tasks.md | 0 openspec/specs/auto-backfill/spec.md | 26 ++++++- .../specs/host-neutral-memory-core/spec.md | 20 +++++- openspec/specs/import-directory-maps/spec.md | 56 ++++++++++++++++ openspec/specs/import-progress/spec.md | 67 +++++++++++++++++++ openspec/specs/web-autostart/spec.md | 28 +++++++- openspec/specs/web-settings/spec.md | 65 ++++++++++++++++-- 16 files changed, 250 insertions(+), 12 deletions(-) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/.openspec.yaml (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/design.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/proposal.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/auto-backfill/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/host-neutral-memory-core/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/import-directory-maps/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/import-progress/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/web-autostart/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/specs/web-settings/spec.md (100%) rename openspec/changes/{external-api-backfill-maps-progress => archive/2026-09-28-external-api-backfill-maps-progress}/tasks.md (100%) create mode 100644 openspec/specs/import-directory-maps/spec.md create mode 100644 openspec/specs/import-progress/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/.openspec.yaml b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/.openspec.yaml similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/.openspec.yaml rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/.openspec.yaml diff --git a/openspec/changes/external-api-backfill-maps-progress/design.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/design.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/design.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/design.md diff --git a/openspec/changes/external-api-backfill-maps-progress/proposal.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/proposal.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/proposal.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/proposal.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/auto-backfill/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/auto-backfill/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/auto-backfill/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/auto-backfill/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/host-neutral-memory-core/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/host-neutral-memory-core/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/host-neutral-memory-core/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/host-neutral-memory-core/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/import-directory-maps/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/import-directory-maps/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/import-directory-maps/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/import-directory-maps/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/import-progress/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/import-progress/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/import-progress/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/import-progress/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/web-autostart/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/web-autostart/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/web-autostart/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/web-autostart/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/specs/web-settings/spec.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/web-settings/spec.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/specs/web-settings/spec.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/specs/web-settings/spec.md diff --git a/openspec/changes/external-api-backfill-maps-progress/tasks.md b/openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/tasks.md similarity index 100% rename from openspec/changes/external-api-backfill-maps-progress/tasks.md rename to openspec/changes/archive/2026-09-28-external-api-backfill-maps-progress/tasks.md diff --git a/openspec/specs/auto-backfill/spec.md b/openspec/specs/auto-backfill/spec.md index 2fd8e8bd..b533ccbb 100644 --- a/openspec/specs/auto-backfill/spec.md +++ b/openspec/specs/auto-backfill/spec.md @@ -8,7 +8,7 @@ Turn past Pi and OpenCode chats into memories and profile input without a manual ### Requirement: Past chats are imported automatically when a host starts -When `autoBackfill` is `true`, which is the default, each host SHALL start a background import of its own chat history after it starts: Pi imports Pi sessions from its default sessions folder, and OpenCode imports sessions from its default database. The run SHALL cover every project whose directory can be resolved with the importer's project rules. It SHALL use the shared importer, its ledger, and the live capture pipeline, and it SHALL also record the imported prompts and build the user profile from them. The run SHALL start after a start-up delay and SHALL NOT delay session start, prompt handling, retrieval, or live capture. When `autoBackfill` is `false`, no automatic run SHALL start. `autoBackfill` SHALL be read from the global config only. +When `autoBackfill` is `true`, which is the default, each host SHALL start a background import of its own chat history after it starts: Pi imports Pi sessions from its default sessions folder, and OpenCode imports sessions from its default database. The run SHALL cover every project whose directory can be resolved with the importer's project rules, including the saved directory maps in `importPathMaps`. It SHALL use the shared importer, its ledger, and the live capture pipeline, and it SHALL also record the imported prompts and build the user profile from them. The run SHALL start after a start-up delay and SHALL NOT delay session start, prompt handling, retrieval, or live capture. When `autoBackfill` is `false`, or when the user has paused that host's backfill, no automatic run SHALL start. `autoBackfill` SHALL be read from the global config only. #### Scenario: A new machine with existing history @@ -32,6 +32,16 @@ When `autoBackfill` is `true`, which is the default, each host SHALL start a bac - **WHEN** a project config sets `autoBackfill` - **THEN** the value SHALL be ignored and the global value SHALL apply +#### Scenario: Sessions from a mapped directory + +- **WHEN** a saved directory map covers sessions recorded in a deleted worktree +- **THEN** the backfill SHALL import them into the map's target project + +#### Scenario: The backfill is paused + +- **WHEN** the user has paused Pi's backfill and Pi starts +- **THEN** no Pi backfill SHALL start + ### Requirement: Backfill covers history up to a fixed cutoff The first automatic run for a host on a memory store SHALL record a cutoff time for that host in the store. Every automatic run for that host SHALL import only user turns at or before the cutoff. Turns after the cutoff SHALL be left to live capture. The cutoff SHALL NOT move on later runs. @@ -95,7 +105,7 @@ At most one automatic backfill for a given host SHALL run at a time across all p ### Requirement: Each host's backfill model is configurable -`opencodeBackfillModel` and `piBackfillModel` SHALL choose the model for each host's automatic backfill. The value `inherit`, which is the default, SHALL use the model that the host's live capture would use under the live-model rule. A `provider/model` value SHALL use that model from the host's signed-in models. When the chosen model cannot be resolved, the run SHALL NOT start and the status SHALL say why. The setting SHALL NOT change the model of live capture or of manual imports. +`opencodeBackfillModel` and `piBackfillModel` SHALL choose the model for each host's automatic backfill. The value `inherit`, which is the default, SHALL use the model that the host's live capture would use under the live-model rule. The value `external` SHALL use the external API (`memoryProvider`, `memoryModel`, `memoryApiUrl`, `memoryApiKey`). A `provider/model` value SHALL use that model from the host's signed-in models. When the chosen model cannot be resolved, including an `external` value while the external API is not fully configured, the run SHALL NOT start and the status SHALL say why. The setting SHALL NOT change the model of live capture or of manual imports. #### Scenario: A cheaper model for Pi's backfill @@ -109,6 +119,18 @@ At most one automatic backfill for a given host SHALL run at a time across all p - **THEN** no OpenCode backfill SHALL start - **AND** the status SHALL say that the model is not available +#### Scenario: Backfill through the external API + +- **WHEN** `opencodeBackfillModel` is `external` and the external API is fully configured +- **THEN** the OpenCode backfill SHALL call the external API +- **AND** OpenCode's live capture SHALL keep using its own model rule + +#### Scenario: The external API is not configured + +- **WHEN** `piBackfillModel` is `external` and `memoryModel` is not set +- **THEN** no Pi backfill SHALL start +- **AND** the status SHALL say that `memoryModel` is missing + ### Requirement: Backfill progress is recorded and visible Each host's backfill SHALL record its state (not started, running, stopped, done, or failed), counts of imported, skipped, failed, and pending exchanges, the number of sessions whose project cannot be resolved, the model used, the cutoff, the last update time, and the last error with secrets removed. The record SHALL live in the memory store and SHALL NOT contain prompts, replies, or other conversation content. The host SHALL show one notice when a run starts with pending work, naming the number of exchanges, the model, and the setting that turns the backfill off, and one notice when the run finishes. diff --git a/openspec/specs/host-neutral-memory-core/spec.md b/openspec/specs/host-neutral-memory-core/spec.md index 61f3cc30..e93eadd9 100644 --- a/openspec/specs/host-neutral-memory-core/spec.md +++ b/openspec/specs/host-neutral-memory-core/spec.md @@ -166,11 +166,11 @@ Capture extraction SHALL accept a model reply whose type is `skip` even when `su OpenCode and Pi SHALL choose the model for automatic capture and profile learning in the same order: -1. the host model: `opencodeProvider`/`opencodeModel` on OpenCode, `piProvider`/`piModel` on Pi, where the model value `inherit` means the session's model +1. the host model: `opencodeProvider`/`opencodeModel` on OpenCode, `piProvider`/`piModel` on Pi, where the model value `inherit` means the session's model and the model value `external` means the external API 2. the external API (`memoryModel`, `memoryApiUrl`, `memoryApiKey`), when no host model is set 3. the session's own model, when neither is set -When the host model fails and the external API is fully configured, the call SHALL use the external API and the user SHALL be notified. When the external API is only partly configured and no host model is set, automatic capture SHALL be disabled and the missing settings SHALL be reported. +When the host model is `external`, the provider value SHALL be ignored and every call SHALL go to the external API. When the host model is `external` and the external API is not fully configured, automatic capture on that host SHALL be disabled and the missing settings SHALL be reported. When another host model fails and the external API is fully configured, the call SHALL use the external API and the user SHALL be notified. When the external API is only partly configured and no host model is set, automatic capture SHALL be disabled and the missing settings SHALL be reported. #### Scenario: Nothing is configured @@ -188,6 +188,22 @@ When the host model fails and the external API is fully configured, the call SHA - **WHEN** `memoryModel` is set but `memoryApiKey` is not, and no host model is set - **THEN** automatic capture SHALL be disabled and the missing settings SHALL be reported +#### Scenario: A host chooses the external API + +- **WHEN** `piModel` is `external` and the external API is fully configured +- **THEN** Pi's automatic capture and profile learning SHALL call the external API +- **AND** OpenCode SHALL keep using its own host model + +#### Scenario: The same choice on both hosts + +- **WHEN** `opencodeModel` is `external` on OpenCode and `piModel` is `external` on Pi, with the same external API settings +- **THEN** both hosts SHALL call the same external model + +#### Scenario: A host chooses an unconfigured external API + +- **WHEN** `opencodeModel` is `external` and `memoryApiUrl` is not set +- **THEN** OpenCode's automatic capture SHALL be disabled and `memoryApiUrl` SHALL be reported as missing + ### Requirement: History import has the same options on both hosts The OpenCode and Pi history imports SHALL accept one option set, parsed by one shared parser, both in a session and from the terminal. Only the history location flag SHALL differ: `--db` for OpenCode and `--root` for Pi. The default scope SHALL be the current project on both hosts. diff --git a/openspec/specs/import-directory-maps/spec.md b/openspec/specs/import-directory-maps/spec.md new file mode 100644 index 00000000..1246ce6c --- /dev/null +++ b/openspec/specs/import-directory-maps/spec.md @@ -0,0 +1,56 @@ +# import-directory-maps Specification + +## Purpose + +Let the user save directory maps once, so that history recorded in deleted or moved directories is imported into the right project by automatic backfill, web imports, and CLI and slash-command imports alike. + +## Requirements + +### Requirement: Saved directory maps apply to every import + +The global config SHALL accept `importPathMaps`, a list of maps, each with a source directory `from` and a target directory `to`. Automatic backfill, web imports, and CLI and slash-command imports on both hosts SHALL resolve project directories with these maps, using the existing resolution order: an exact directory map first, then the recorded directory, then, for OpenCode only, the project worktree. A `--map` flag or a web import's own map SHALL add to the saved maps for that run, and SHALL win over a saved map with the same source directory. A map whose target directory does not exist SHALL leave its sessions unresolved. `importPathMaps` SHALL be read from the global config only. Invalid entries SHALL be rejected by the same validation used at startup. + +#### Scenario: Automatic backfill uses a saved map + +- **WHEN** `importPathMaps` maps `/Users/me/code/app-feat-x` to `/Users/me/code/app` and Pi's backfill finds sessions recorded in `/Users/me/code/app-feat-x`, which no longer exists +- **THEN** those sessions SHALL be imported into the project of `/Users/me/code/app` + +#### Scenario: A CLI map overrides a saved map + +- **WHEN** a saved map sends `/old` to `/a` and the user runs an import with `--map /old=/b` +- **THEN** that run SHALL use `/b` for sessions recorded in `/old` +- **AND** the saved map SHALL be unchanged + +#### Scenario: The target directory is missing + +- **WHEN** a saved map points at a directory that does not exist +- **THEN** its sessions SHALL be reported as unresolved and SHALL NOT be assigned to any project + +#### Scenario: A project config sets maps + +- **WHEN** a project config sets `importPathMaps` +- **THEN** the value SHALL be ignored and the global value SHALL apply + +### Requirement: The Settings page manages directory maps + +The Settings page SHALL have a **Directory maps** section. It SHALL list the saved maps and the directories that the latest listing, preview, or backfill could not resolve, for each host, with the number of sessions in each. For each unresolved directory it SHALL offer a suggested target when one can be found, and SHALL let the user accept, edit, or reject it. Saving SHALL write `importPathMaps` to the global config with the same safe-save rules as other settings. The page SHALL say that a change applies to the next import or backfill run. It SHALL NOT show conversation content. + +A suggestion SHALL be an existing directory. The page SHALL suggest, in this order: the main repository of a deleted Git worktree, found as an existing directory whose name is the longest leading part of the missing directory's name or of one of its parent directories' names (for example `app` for `app-feat-x` or for `workspaces/app/feat-x`); then, for OpenCode sessions, the project directory that OpenCode recorded for the session's project, read without writing to OpenCode's database. When no candidate exists, the page SHALL show no suggestion and SHALL let the user type a target or leave the directory unmapped. + +#### Scenario: Accepting a suggested target + +- **WHEN** Pi sessions are recorded in `/Users/me/code/app-feat-x`, which no longer exists, and `/Users/me/code/app` is a Git repository +- **THEN** the page SHALL suggest `/Users/me/code/app` +- **AND** accepting it and saving SHALL add the map to `importPathMaps` + +#### Scenario: No candidate exists + +- **WHEN** sessions are recorded in a temporary directory that no longer exists and no candidate is found +- **THEN** the page SHALL show the directory with no suggestion +- **AND** its sessions SHALL stay unresolved unless the user types a target + +#### Scenario: Removing a map + +- **WHEN** the user removes a saved map and saves +- **THEN** `importPathMaps` SHALL no longer contain it +- **AND** memories already imported through it SHALL remain diff --git a/openspec/specs/import-progress/spec.md b/openspec/specs/import-progress/spec.md new file mode 100644 index 00000000..16827269 --- /dev/null +++ b/openspec/specs/import-progress/spec.md @@ -0,0 +1,67 @@ +# import-progress Specification + +## Purpose + +Show how far every history import has progressed and how long it has left, whether it runs as an automatic backfill, from the web page, or from the terminal, and let the user start, pause, and resume each host's backfill from the Settings page. + +## Requirements + +### Requirement: Every import run records its progress + +Each import run that makes model calls, whether an automatic backfill, a web import, or a CLI or slash-command import, SHALL record a progress entry in the memory store. The entry SHALL hold the host, the surface that started it, the state (running, paused, stopped, done, or failed), the start time, the total number of memory units and profile batches to process, the number processed so far with imported, skipped, and failed counts, the last update time, and the last error with secrets removed. The total SHALL count only units that need a model call, not units already in the ledger. For an automatic backfill, and for Run now and Resume, the total SHALL be the backfill's dry-run count, known before the first model call. For a CLI, slash-command, or web import, which has no dry run, the total SHALL start from the units found and SHALL be refined as the run passes units already in the ledger. The entry SHALL NOT contain prompts, replies, or other conversation content. Dry runs SHALL NOT record progress entries. A run that stops without updating its entry SHALL be shown as stopped once its owning process is gone. + +#### Scenario: A CLI import is visible on the page + +- **WHEN** the user runs `om-memory-system import-pi-history` in a terminal and opens the Settings page +- **THEN** the page SHALL show that Pi import as running, started from the CLI, with its counts + +#### Scenario: Most units are already imported + +- **WHEN** a Pi backfill finds 736 units, of which 729 are already in the ledger +- **THEN** the page SHALL show a total of 7 and count only those 7 as pending + +#### Scenario: The terminal is closed during a CLI import + +- **WHEN** the CLI process ends without finishing its run +- **THEN** the page SHALL show that run as stopped +- **AND** rerunning the same command SHALL continue from the ledger + +### Requirement: Progress shows percentage and time left + +For a running import, the Settings page SHALL show a progress bar, the percentage done, the number done out of the total, and the estimated minutes left. The estimate SHALL be computed from the processing rate over the recent part of the run, not from the whole run, and SHALL be shown as unknown until enough units have finished to measure a rate. The values SHALL refresh without a page reload while the run is active. + +#### Scenario: Watching a long import + +- **WHEN** 400 of 1,000 units are done and the recent rate is 5 units a minute +- **THEN** the page SHALL show 40%, 400 of 1,000, and about 120 minutes left + +#### Scenario: A run has just started + +- **WHEN** fewer units than needed to measure a rate have finished +- **THEN** the time left SHALL be shown as unknown + +### Requirement: The user can run, pause, and resume a host's backfill + +The Settings page SHALL offer **Run now**, **Pause**, and **Resume** for each host's backfill. Run now SHALL start that host's backfill at once, with the same cutoff, maps, model rule, ledger, and one-run-per-host lock as an automatic backfill. It SHALL be available in the login web app and in `om-memory-system web` without Pi or OpenCode open when the host's backfill model resolves to the external API, and SHALL otherwise say which model setting it needs. Pause SHALL stop the run after its current exchange and record the paused state. A paused backfill SHALL NOT start automatically at a host start until the user resumes it. Resume SHALL clear the paused state and start the run, continuing from the ledger. Run now SHALL be refused, with the reason, while another import for the same host runs. These controls SHALL follow the same origin and authentication rules as other Settings changes. + +#### Scenario: Running a backfill with no host open + +- **WHEN** the login web app runs, Pi is closed, `piBackfillModel` is `external`, and the user clicks Run now for Pi +- **THEN** the Pi backfill SHALL start in the web app's process and its progress SHALL be shown + +#### Scenario: Pausing across a restart + +- **WHEN** the user pauses the OpenCode backfill and later starts OpenCode +- **THEN** no OpenCode backfill SHALL start +- **AND** the page SHALL show it as paused until the user clicks Resume + +#### Scenario: A second run for the same host + +- **WHEN** a Pi CLI import is running and the user clicks Run now for Pi +- **THEN** the page SHALL refuse with the reason that a Pi import is already running + +#### Scenario: No usable model outside a host + +- **WHEN** the login web app runs and the Pi backfill model is a Pi signed-in model +- **THEN** Run now for Pi SHALL be unavailable +- **AND** the page SHALL say to open Pi or choose the external API for Pi's backfill diff --git a/openspec/specs/web-autostart/spec.md b/openspec/specs/web-autostart/spec.md index 75fb0cde..e02e5c29 100644 --- a/openspec/specs/web-autostart/spec.md +++ b/openspec/specs/web-autostart/spec.md @@ -39,7 +39,7 @@ When `webServerAutoStart` is `true`, which is the default, and `webServerEnabled ### Requirement: The web app runs without a host session -The web app started by the login item or by `om-memory-system web` SHALL serve the same pages and API as the web app inside OpenCode, with the same port, bind address, origin rules, API token, and Basic Auth settings. When another OMMS process already serves the web app on the configured port, it SHALL follow the existing port ownership and takeover rules instead of starting a second server. Features that need a host, such as OpenCode's connected-model list, SHALL report that they are unavailable, with the existing reasons. It SHALL NOT run automatic backfill. +The web app started by the login item or by `om-memory-system web` SHALL serve the same pages and API as the web app inside OpenCode, with the same port, bind address, origin rules, API token, and Basic Auth settings. When another OMMS process already serves the web app on the configured port, it SHALL follow the existing port ownership and takeover rules instead of starting a second server. Features that need a host, such as OpenCode's connected-model list, SHALL report that they are unavailable, with the existing reasons. It SHALL NOT start a backfill on its own. It SHALL run a host's backfill when the user starts or resumes it on the Settings page and that host's backfill model resolves to the external API. #### Scenario: Opening the web app with no session open @@ -51,9 +51,14 @@ The web app started by the login item or by `om-memory-system web` SHALL serve t - **WHEN** the login web app owns the port and OpenCode starts - **THEN** OpenCode SHALL use the running web app instead of starting a second one +#### Scenario: The login web app does not backfill by itself + +- **WHEN** the login web app starts with pending history and `autoBackfill` on +- **THEN** no backfill SHALL start until the user clicks Run now or Resume + ### Requirement: The terminal can start and manage the web app -The package SHALL provide `om-memory-system web`, which starts the web app in the foreground until it is stopped, and `om-memory-system web install`, `web uninstall`, and `web status`. `install` SHALL set `webServerAutoStart` to `true` in the global config and install the login item. `uninstall` SHALL set it to `false` and remove the item. `status` SHALL report the setting, whether the item is installed, what it starts, and whether a web app answers on the configured port. `web` SHALL refuse to start, with the reason, when `webServerEnabled` is `false`. +The package SHALL provide `om-memory-system web`, which starts the web app in the foreground until it is stopped, and `om-memory-system web install`, `web uninstall`, and `web status`. `install` SHALL set `webServerAutoStart` to `true` in the global config and install the login item. `uninstall` SHALL set it to `false` and remove the item. `status` SHALL report the setting, whether the item is installed, what it starts, and whether a web app answers on the configured port. `web` SHALL refuse to start, with the reason, when `webServerEnabled` is `false`. `om-memory-system --version` SHALL print the package version and exit with code `0`. #### Scenario: Starting the web app by hand @@ -70,3 +75,22 @@ The package SHALL provide `om-memory-system web`, which starts the web app in th - **WHEN** `webServerEnabled` is `false` and the user runs `om-memory-system web` - **THEN** the command SHALL exit with a message that the web server is disabled + +#### Scenario: Printing the version + +- **WHEN** the user runs `om-memory-system --version` +- **THEN** the command SHALL print the installed package version and exit with code `0` + +### Requirement: The web app reports the global command's version + +The Settings page's **Web app** section SHALL show the running OMMS version and the version of the `om-memory-system` command found on the web app's `PATH`, or that the command is not installed globally. When the two versions differ, the section SHALL warn about it and show the command that upgrades the global install. It SHALL say that a global install is optional but recommended, because it lets the login item and the terminal commands run without `npx`. + +#### Scenario: The global command is older + +- **WHEN** OMMS 3.4.0 runs and the global `om-memory-system --version` prints `3.3.1` +- **THEN** the section SHALL warn that the versions differ and show the upgrade command + +#### Scenario: No global install + +- **WHEN** `om-memory-system` is not on the web app's `PATH` +- **THEN** the section SHALL say that it is not installed globally and show the install command diff --git a/openspec/specs/web-settings/spec.md b/openspec/specs/web-settings/spec.md index 823c3aea..9e5dc65d 100644 --- a/openspec/specs/web-settings/spec.md +++ b/openspec/specs/web-settings/spec.md @@ -23,7 +23,7 @@ The sidebar footer SHALL show a cogwheel button next to the language, theme, and ### Requirement: Each host's capture model can be chosen on the page -The Settings page SHALL show one model card for OpenCode and one for Pi. Each card SHALL offer **Session model** and **Manual model**. Choosing Session model SHALL save the host's model as `inherit`. Choosing Manual model SHALL save the selected provider and model to that host's settings (`opencodeProvider`/`opencodeModel` or `piProvider`/`piModel`). The manual picker SHALL list the host's signed-in models when the server can read them. Otherwise it SHALL accept a typed `provider/model` value and say that the list is not available. Each card SHALL show, read-only, the external API fallback and which model the live-model rule would choose now. The page SHALL NOT change the order of the live-model rule. +The Settings page SHALL show one model card for OpenCode and one for Pi. Each card SHALL offer **Session model**, **Manual model**, and **External API**. Choosing Session model SHALL save the host's model as `inherit`. Choosing External API SHALL save the host's model as `external`. Choosing Manual model SHALL save the selected provider and model to that host's settings (`opencodeProvider`/`opencodeModel` or `piProvider`/`piModel`). The manual picker SHALL list the host's signed-in models when the server can read them. Otherwise it SHALL accept a typed `provider/model` value and say that the list is not available. External API SHALL be selectable only when the external API is fully configured, and otherwise SHALL say which setting is missing. Each card SHALL show, read-only, the external API fallback and which model the live-model rule would choose now. The page SHALL NOT change the order of the live-model rule. #### Scenario: Switching OpenCode to the session model @@ -37,6 +37,12 @@ The Settings page SHALL show one model card for OpenCode and one for Pi. Each ca - **THEN** the global config SHALL have `piProvider` `zai` and `piModel` `glm-5.3` - **AND** the next Pi capture SHALL use that model +#### Scenario: Choosing the external API for Pi + +- **WHEN** the external API is fully configured and the user chooses External API on the Pi card and saves +- **THEN** the global config SHALL have `piModel` set to `external` +- **AND** the next Pi capture SHALL call the external API + #### Scenario: The Pi model list is not available - **WHEN** the server cannot load the Pi SDK @@ -270,7 +276,7 @@ The Settings page SHALL show the most recent lines of the OMMS log file in a scr ### Requirement: Settings are saved safely to the global config -Saving on the Settings page SHALL write only the changed keys to the global config file that OMMS is reading. When that file is the legacy `~/.config/opencode/opencode-mem.jsonc`, the first save SHALL create `~/.config/omms/omms.jsonc` as a copy of it, comments included, apply the change there, and tell the user that OMMS now reads the new file. The legacy file SHALL NOT be written. Saves SHALL run one at a time, and a save SHALL be rejected without writing when the file changed after the page read it. Saving SHALL keep comments, key order, and all other keys. It SHALL reject values that fail the same validation used at startup, and SHALL leave the file unchanged when it rejects them. Running OpenCode and Pi processes SHALL use the saved values from their next capture without a restart. The page SHALL NOT write project config files. It SHALL NOT read, show, or change secret values; it SHALL show only whether a secret is set and its source type (literal, `env://`, or `file://`). +Saving on the Settings page SHALL write only the changed keys to the global config file that OMMS is reading. When that file is the legacy `~/.config/opencode/opencode-mem.jsonc`, the first save SHALL create `~/.config/omms/omms.jsonc` as a copy of it, comments included, apply the change there, and tell the user that OMMS now reads the new file. The legacy file SHALL NOT be written. Saves SHALL run one at a time, and a save SHALL be rejected without writing when the file changed after the page read it. Saving SHALL keep comments, key order, and all other keys. It SHALL reject values that fail the same validation used at startup, and SHALL leave the file unchanged when it rejects them. Running OpenCode and Pi processes SHALL use the saved values from their next capture without a restart. The page SHALL NOT write project config files. It SHALL NOT read or show secret values; it SHALL show only whether a secret is set, its source type (literal, `env://`, or `file://`), and, for `env://` and `file://`, the variable name or file path. The only secret the page SHALL change is `memoryApiKey`, and only to an `env://` or `file://` reference, including one created by saving a pasted key to a private key file. It SHALL NOT save a literal key to the config. #### Scenario: A commented config file is edited @@ -303,11 +309,16 @@ Saving on the Settings page SHALL write only the changed keys to the global conf #### Scenario: A secret is configured - **WHEN** `memoryApiKey` is set to `env://OMMS_KEY` -- **THEN** the page SHALL show that the key is set from an environment variable and SHALL NOT show its value +- **THEN** the page SHALL show that the key is set from the environment variable `OMMS_KEY` and SHALL NOT show its value + +#### Scenario: A literal key is submitted as a reference + +- **WHEN** a request tries to save `memoryApiKey` as a value that is not an `env://` or `file://` reference +- **THEN** the save SHALL be rejected and the file SHALL be unchanged ### Requirement: Changes from the page are access-controlled -Every Settings endpoint that changes config, deletes trace files, validates or browses an import source, lists import sessions, starts or cancels an import, or makes a model test call SHALL require a JSON request body and SHALL reject requests whose origin is not allowed by the web server's origin rules. When the web server is bound to a non-loopback host, these endpoints SHALL also require the existing API token or Basic Auth credentials. Turning `captureTrace` on SHALL be rejected when the server is bound to a non-loopback host without Basic Auth. +Every Settings endpoint that changes config, saves a key file, deletes trace files, validates or browses an import source, lists import sessions, starts, pauses, resumes, or cancels an import or backfill, or makes a model test call SHALL require a JSON request body and SHALL reject requests whose origin is not allowed by the web server's origin rules. When the web server is bound to a non-loopback host, these endpoints SHALL also require the existing API token or Basic Auth credentials. Turning `captureTrace` on, and saving a pasted key to a key file, SHALL be rejected when the server is bound to a non-loopback host without Basic Auth. #### Scenario: A request from another website @@ -324,9 +335,14 @@ Every Settings endpoint that changes config, deletes trace files, validates or b - **WHEN** the server is bound to `0.0.0.0` without Basic Auth and a request turns tracing on - **THEN** the server SHALL reject it +#### Scenario: Saving a key over the network + +- **WHEN** the server is bound to `0.0.0.0` without Basic Auth and a request saves a pasted key +- **THEN** the server SHALL reject it and write no key file + ### Requirement: The page controls automatic import -The Settings page SHALL have an **Automatic import** section with a switch for `autoBackfill` and, for each host, a model choice for `opencodeBackfillModel` or `piBackfillModel`: **Same as live capture** (saves `inherit`) or a manual `provider/model` chosen the same way as the host's capture model. For each host the section SHALL show the backfill state, the counts of imported, skipped, failed, and pending exchanges, the number of sessions whose project cannot be resolved, the model used, the cutoff, and the last error. The counts SHALL refresh while a run is active. It SHALL say that a change takes effect at the host's next start, except that turning the switch off also stops a running backfill after its current exchange. It SHALL say that automatic import makes model calls. +The Settings page SHALL have an **Automatic import** section with a switch for `autoBackfill` and, for each host, a model choice for `opencodeBackfillModel` or `piBackfillModel`: **Same as live capture** (saves `inherit`), **External API** (saves `external`), or a manual `provider/model` chosen the same way as the host's capture model. For each host the section SHALL show the backfill state, including paused, the counts of imported, skipped, failed, and pending exchanges, the number of sessions whose project cannot be resolved with a link to the Directory maps section, the model used, the cutoff, the last error, and the progress bar, percentage, and time left defined by the import progress capability. It SHALL offer Run now, Pause, and Resume for each host. The counts SHALL refresh while a run is active. It SHALL say that a model change takes effect at the next run, except that turning the switch off also stops a running backfill after its current exchange. It SHALL say that automatic import makes model calls. #### Scenario: Turning automatic import off @@ -340,10 +356,15 @@ The Settings page SHALL have an **Automatic import** section with a switch for ` - **THEN** the global config SHALL have `piBackfillModel` set to `zai/glm-5-turbo` - **AND** `piProvider` and `piModel` SHALL be unchanged +#### Scenario: Choosing the external API for OpenCode's backfill + +- **WHEN** the user chooses External API for OpenCode's backfill and saves +- **THEN** the global config SHALL have `opencodeBackfillModel` set to `external` + #### Scenario: Watching progress - **WHEN** a backfill runs while the section is open -- **THEN** the counts SHALL update without a page reload +- **THEN** the counts, progress bar, percentage, and time left SHALL update without a page reload ### Requirement: The page controls starting the web app at login @@ -359,3 +380,35 @@ The Settings page SHALL have a **Web app** section with a switch for `webServerA - **WHEN** the login item is installed - **THEN** the section SHALL show it as installed + +### Requirement: The page configures the external API + +The Settings page SHALL have an **External API** card that edits `memoryProvider`, `memoryApiUrl`, `memoryModel`, and `memoryApiKey` in the global config. The provider SHALL be chosen from the providers OMMS supports. The card SHALL offer three key sources: + +- **Environment variable**: the user types a variable name, and the page saves `env://NAME`. +- **Key file**: the user types the path of an existing file, and the page saves `file://` with that path. +- **Save key to a private file**: the user pastes the key once. The server SHALL write it to a file under `~/.config/omms/secrets/`, create the folder if needed, restrict the file to the current user (mode `600` on macOS and Linux, a user-only access list on Windows), and save `file://` with that path. It SHALL replace an existing key file only after the user confirms. + +The key value SHALL NOT be written to `omms.jsonc`, the log, the capture trace, or any response, and the page SHALL NOT show it after saving. The card SHALL show the saved key source type and reference, whether the key resolves in the web app's own process, and, for an environment variable, that a login web app does not see variables set only in a shell profile. A **Test** button SHALL make one small call with the saved settings and report success or an error with the key redacted. + +#### Scenario: Using an environment variable + +- **WHEN** the user chooses Environment variable, types `ZAI_API_KEY`, and saves +- **THEN** the global config SHALL have `memoryApiKey` set to `env://ZAI_API_KEY` + +#### Scenario: Saving a pasted key + +- **WHEN** the user pastes a key, chooses Save key to a private file, and saves +- **THEN** the key SHALL be written to a user-only file under `~/.config/omms/secrets/` +- **AND** `memoryApiKey` SHALL be set to `file://` with that path +- **AND** no response or log line SHALL contain the key + +#### Scenario: The variable is missing in the login web app + +- **WHEN** `memoryApiKey` is `env://ZAI_API_KEY` and the login web app's process has no such variable +- **THEN** the card SHALL say that the key does not resolve in the web app and suggest a key file + +#### Scenario: Testing the endpoint + +- **WHEN** the user clicks Test and the endpoint rejects the key +- **THEN** the card SHALL show the error with the key redacted From b77ae14b0de466fd4b4eca1ea9c10278415de105 Mon Sep 17 00:00:00 2001 From: Aizat Hawari Date: Mon, 28 Sep 2026 10:12:31 +0100 Subject: [PATCH 4/4] docs: archive a completed OpenSpec change before opening its pull request --- AGENTS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 433918bc..8c027466 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -62,11 +62,11 @@ Use a change for a new feature, a change to user-visible behaviour, a breaking c 5. Implement with the `openspec-apply-change` skill. Mark each task `- [x]` in `tasks.md` when it is done and tested. 6. If the plan changes during work, update the change artifacts with the `openspec-update-change` skill. 7. Before you report completion, run the `openspec-verify-change` skill. -8. After the pull request merges, archive the change with the `openspec-archive-change` skill. Archiving moves it to `openspec/changes/archive/` and updates `openspec/specs/`. +8. Before you create the pull request, archive the completed change with the `openspec-archive-change` skill. Archiving moves it to `openspec/changes/archive/` and syncs its spec deltas into `openspec/specs/`. Archive only when every task is done and step 7 passes. Use `openspec list` for active changes and `openspec status --change ` for artifact status. -Commit OpenSpec artifacts with the code change they describe. Commit the archive move in the pull request that archives a change. +Commit OpenSpec artifacts with the code change they describe. Commit the archive move in the same pull request as the change. A change that is only a proposal stays active until it is implemented. ## Commands