Skip to content

feat(keychain): a reinstall restores generated keys from the keychain; Export reminder; federated data keys saved too - #512

Merged
kh0pper merged 4 commits into
mainfrom
feat/keychain-restore-and-export-reminder
Oct 17, 2026
Merged

feat(keychain): a reinstall restores generated keys from the keychain; Export reminder; federated data keys saved too#512
kh0pper merged 4 commits into
mainfrom
feat/keychain-restore-and-export-reminder

Conversation

@kh0pper

@kh0pper kh0pper commented Oct 16, 2026

Copy link
Copy Markdown
Owner

This is PR 1 of 2 for the recovery-key decision on queue item 30, Option 1: generated keys live in the keychain, and the passphrase-encrypted keychain Export is the one off-machine copy. Escrow to paired instances: never. PR 2 builds the campaigns and tax keys on top of this one.

The bug this fixes

Today a rebuilt machine (crow.db restored, keychain Export imported, extension reinstalled) mints a new key. Saving that key then overwrites the imported keychain entry, so data saved with the old key becomes unreadable. The same happens if the user reinstalls first and imports later: Import skips any bundle and key that already has a readable entry.

Changes

  • Restore from the keychain. planGeneratedEnv checks, in order: installed .env, retained copy, compose fallback, keychain, new value.
    • A keychain: true generated key that a readable entry holds is reused, not minted.
    • A store_as key gets a fresh hash of the restored plaintext.
    • The job log names restored keys only, never their values.
  • Never mint over an unreadable entry. If an entry exists but is sealed with a keychain key this machine lacks (or there is no key file), the install is refused: "Import your keychain Export first…". To start fresh, the user deletes the old saved password.
  • Boot backfill. Extensions installed before their manifest saved a key to the keychain get that key saved once at boot, from .env, the retained copy or the MCP entry. The boot refresh never runs the install job, so without this the key would be in no keychain entry, and the field is hidden from Configure.
    • Never writes over an existing entry.
    • Never saves a store_as hash.
  • Export reminder.
    • A Nest info issue (strip only, never a push) appears while any extension password is in no Export. "In an Export" means its sealed value was fingerprinted at the last Export, or it came in through an Import.
    • The fingerprints are a local setting, not in the sync allowlist.
    • The export and import routes drop the health cache.
    • The install log reminds the user when nothing has been exported yet.
    • en and es strings added.
  • Mastodon, Pixelfed and PeerTube. Their generated data keys now have keychain: true:
    • Mastodon: secret_key_base, the OTP secret, and the three Active Record encryption keys.
    • Pixelfed: APP_KEY.
    • PeerTube: the server secret.
    • All three bundles go to 1.0.4.
  • mcp-addons.json is written atomically at mode 600. It carries the MCP children's generated keys.
  • crow_keychain stays LOCAL_ONLY (a test asserts it).

Tests

  • New tests/keychain-restore.test.js (11 tests):
    • restore order;
    • restore then reinstall end to end;
    • refusal over an unreadable entry, and over a missing key file;
    • backfill;
    • 0600 mode;
    • local-only.
  • New tests/keychain-export-reminder.test.js (7 tests), plus updated keychain-store and bundle-generated-secrets tests.
  • Full suite on this branch: 9729 tests, 2 failures under load, both unrelated (workspace-calendar T10-I3, artifacts-panel QW4). Both pass when rerun alone.

Live proof

Run on PR 2's branch, which contains this one. It used a scratch home only: every data path inside a session scratch directory, checked before running, no key in the env, two separate node processes, a systemd-run --user unit capped at 30 minutes, and loopback-only HTTP.

  • Home A exported the keychain through the real POST /dashboard/keychain/api/export, after a real re-auth. The Nest reminder was on before the export and off after it.
  • Home A was deleted.
  • Home B started from A's crow.db with no secrets/, .env or MCP entries:
    1. A reinstall before the Import was refused, and the sealed key was left untouched.
    2. POST …/import brought in 2 entries.
    3. The reinstall logged "Restored 1 key(s) from Crow keychain", and B's keys matched A's (compared by fingerprint).
    4. A's return, documents and Reddit credential all decrypted on B.

https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp

@kh0pper kh0pper mentioned this pull request Oct 16, 2026
Merged
@kh0pper

kh0pper commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

I ran a fresh code and security review. Fixes are in d11b6b2.

  • Restore fails closed. If the keychain can't be read, the install now refuses to mint any keychain:true key, because a new key could replace one the read missed. A damaged entry counts as unreadable instead of clearing the result.
  • Refusal message. A refused install now keeps a prior install's folder, and the message says that deleting the old entry needs a dashboard password or two-factor sign-in.
  • Backfill runs once per bundle and key (local marker). A key you delete in Passwords stays deleted at the next boot.
  • Export reminder. Fingerprints are now taken before the Export is built and cover every status. A reactivated removed extension no longer triggers the reminder, and a key saved during the export is still flagged.
  • Install-log reminder now follows the same rule as the Nest reminder.
  • Database passwords. The Mastodon, Pixelfed and PeerTube database passwords now go to the keychain too, so a database restored from a backup still opens after a reinstall.
  • New tests: fail-closed restore, a damaged entry, deleted-stays-deleted, a removed extension, and the real export and import HTTP routes.

Full suite on the stacked branch: 9756 tests, 0 failures. The scratch-home live proof was re-run after these fixes and passes every check (details in #513).

https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp

@kh0pper
feat(keychain): a reinstall restores generated keys from the keychain…
fbca92b 
…; a reminder to export; federated data keys saved too

Recovery keys, option 1 (queue item 30, decided 2026-10-16): the keychain plus its
passphrase-encrypted Export is how a generated key survives a lost machine.

- A reinstall that finds no .env and no retained copy now takes a keychain:true
  generated key from a readable keychain entry (restored crow.db + imported Export)
  instead of minting a new one, which also overwrote the imported entry and stranded
  data saved with the old key. A store_as key gets a fresh hash of the restored
  plaintext. Best-effort: no key file or an unreadable entry mints, as before, and says
  so by key name. Values are never logged; crow_keychain stays local-only.
- Export reminder: a Nest info issue while a key Crow generated is in no keychain
  Export (created after the last export, a local setting); the install log says so too.
- Mastodon (secret_key_base, OTP secret, three Active Record encryption keys), Pixelfed
  (APP_KEY) and PeerTube (server secret) save their generated keys to the keychain.
  mastodon/pixelfed/peertube 1.0.4.

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
@kh0pper
fix(keychain): review — never mint over an unreadable entry; boot bac…
220e60a 
…kfill of kept keys; reminder counts what no Export holds; mcp-addons.json at 600

Plan review findings:
- An install that would mint a keychain:true key whose entry exists but is sealed with
  a keychain key this machine lacks (crow.db restored, Export not imported) is refused:
  the new value would replace that entry and a later Import would skip it, losing the
  old key for good. Deleting the old entry is the explicit way to start fresh.
- Boot backfill: an extension installed before its manifest saved a generated key to
  the keychain gets that key (from .env, retained copy or MCP entry) saved once, so it
  is visible in Passwords and goes into the Export (the boot refresh never runs the
  install job). Never over an existing entry; never a store_as hash.
- The Export reminder fingerprints the sealed values the last Export held (and those an
  Import brought in), so a re-minted or kept key counts and an imported one does not;
  the health cache is dropped on export and import.
- mcp-addons.json (MCP children's generated keys) is written atomically at mode 600.
- The delete warning for a generated entry also covers data keys.

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
@kh0pper
fix(keychain): code review — fail closed on a keychain read error; a …
a6fba7c 
…deleted entry stays deleted; the reminder snapshot covers removed extensions

- keychainRestoreValues fails CLOSED: a read error refuses any keychain:true mint
  (it could replace an entry the read missed), and one damaged entry counts as
  unreadable instead of clearing the result.
- The refusal keeps a prior install's folder (its .env holds other secrets) and says
  that deleting needs a dashboard password or two-factor sign-in.
- Boot backfill runs once per bundle+key (local marker): a key the user deletes in
  Passwords is never brought back by the next boot.
- Export fingerprints are taken before the Export is built and cover every status, so
  a reactivated removed extension is not nagged about and a key saved mid-export is.
- The install-log reminder follows the same rule as the Nest (anything unexported).
- Mastodon/Pixelfed/PeerTube database passwords go to the keychain too: a database
  restored from a backup still opens after a reinstall.
- Tests: fail-closed, damaged entry, deleted-stays-deleted, removed extension, and the
  real export/import HTTP routes.

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
@kh0pper
feat(campaigns,tax): generated data keys in the keychain; tax documen…
4a07110 
…ts encrypted at rest (campaigns 1.0.3, tax 2.1.0) (#513)

* feat(campaigns,tax): generated data keys in the keychain; tax documents encrypted at rest (campaigns 1.0.3, tax 2.1.0)

Recovery keys, option 1 (queue item 30, decided 2026-10-16). Stacked on the keychain
restore + Export reminder change.

- CROW_CAMPAIGNS_ENCRYPTION_KEY and CROW_TAX_ENCRYPTION_KEY are generated at install and
  saved to the keychain (a reinstall takes them back from it). They reach the MCP child
  through its mcp-addons entry; both dashboard panels read them in the child's own order
  (the entry first, then the gateway env), so panel and server always use one key.
- An install still on the tax example key once published in the repo gets a new key and
  the old one moves to CROW_TAX_ENCRYPTION_KEY_PREVIOUS (generic manifest field
  replace_values_sha256, validated by the bundle contract), so its returns still open
  and move to the new key.
- Tax uploads are stored only encrypted (<CROW_HOME>/tax-documents, AES-256-GCM with a
  key id in the header); ingestion reads the upload from memory. Documents stored before
  are encrypted at dashboard start, one row at a time and verified before the plain file
  is removed: unique per-run names, compare-and-swap on the row, a lock file against two
  migrators, crash leftovers cleaned once verified, and only regular files inside the
  documents folder are ever touched. scripts/documents.mjs reverses it (in place, or as
  plain copies to a folder). The MCP ingest tool opens a stored .enc.
- A version bump now ships a non-docker bundle's engine/ and pdf/ (tax's panel imports
  the installed engine).
- The tax manifest says exactly what is encrypted (returns, documents) and what is not
  (the figures read from documents).

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp

* test(tax): the previous_key Configure test runs on a typed field; Configure never overwrites the generated tax key

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp

* fix(tax): code review — key id from the salted derived key; ingest checks the path before reading; restored rows find their .enc; safer lock

- The document key id is now an HMAC under the salted PBKDF2 output (per file), not
  under the passphrase: no cheap offline guess check, nothing linkable across files.
- The MCP ingest tool checks the extension (.pdf/.enc), that the path is a regular file,
  and a 25 MB cap BEFORE reading (the AI chooses the path).
- A crow.db restored from a backup taken before the migration (rows still naming the
  plain file) finds the single <name>.<8hex>.enc successor and repoints the row.
- The documents lock: stale takeover by atomic rename (one winner), touched per row,
  folder tightened to 0700; the panel logs a held lock. --out folders are 0700.
- Upload names get a random part (two same-millisecond uploads never share a file).
- An install on the published example key: the upload refusal and the install log say
  to reinstall, and say why when the previous-key slot is already in use.
- The CLI uses the database of the same CROW_HOME as its key and folders.

Claude-Session: https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp
@kh0pper
kh0pper force-pushed the feat/keychain-restore-and-export-reminder branch from d6966f0 to 4a07110 Compare October 17, 2026 00:41
@kh0pper
kh0pper merged commit a8fd534 into main Oct 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kh0pper