feat(campaigns,tax): generated data keys in the keychain; tax documents encrypted at rest (campaigns 1.0.3, tax 2.1.0)

[kh0pper]

Stacked on #512. Its base is feat/keychain-restore-and-export-reminder while #512 is open; retarget it to main once #512 merges. This is PR 2 of 2 for the recovery-key decision (queue item 30, Option 1 plus all four extras; no escrow to paired instances).

Changes

• Generated data keys. CROW_CAMPAIGNS_ENCRYPTION_KEY and CROW_TAX_ENCRYPTION_KEY are now generate: "secret" and keychain: true. A reinstall takes them back from the keychain (feat(keychain): a reinstall restores generated keys from the keychain; Export reminder; federated data keys saved too #512). They reach the MCP child only through its mcp-addons.json entry, never the shared gateway .env.
• The panels use the same key as the MCP server. Both dashboard panels read the key in the MCP child's own order: the instance's MCP entry first, then the gateway env, matching proxy.js. The panel and the server can never end up on different keys. New helper: bundles/campaigns/server/key.js.
• Installs on the published example tax key. An install still on the example key once published in the repo now gets a new key. The old key moves to CROW_TAX_ENCRYPTION_KEY_PREVIOUS, so returns saved with it still open and move to the new key.
  • This uses a new generic manifest field, env_vars[].replace_values_sha256, which the bundle contract validates: lowercase hex digests, and it needs both generate and previous_key.
  • A previous_key slot that is already in use is never overwritten.
  • The boot backfill never saves such a value.
• Tax documents encrypted at rest.
  • Uploads go to <CROW_HOME>/tax-documents as <name>.enc only.
  • Format: AES-256-GCM, PBKDF2 per file, and a key id in the header, so a wrong key is caught before any file is replaced.
  • Ingestion reads the upload from memory; no plaintext is ever written.
  • Uploads are refused when there is no key.
• Migration of plain documents stored before, run at dashboard start (documents-crypto.js):
  • Each new file gets a unique per-run name. The new file is verified (read back, decrypted, compared) before the row's file_path is updated with compare-and-swap. The plain file is removed only after that.
  • A lock file stops two migrators from running at once.
  • Leftovers from a crash are cleaned only after verification.
  • Only regular files inside the documents folder(s) are ever read, written or deleted. Symlinks and paths from the DB that point elsewhere are skipped. The delete route uses the same check.
  • A row from a crow.db restored under another home is matched by file name in this instance's folder.
• Reversible. bundles/tax/scripts/documents.mjs:
  • decrypt --in-place is the exact inverse of the migration.
  • decrypt --out <dir> writes plain copies and leaves the DB untouched.
  • CROW_TAX_KEEP_PLAIN_DOCUMENTS=1 stops the start-up pass from encrypting again after a downgrade.
• The MCP ingest tool can now open a stored .enc file.
• Refresh fix. A version bump now ships a non-docker bundle's engine/ and pdf/. Tax's panel imports the installed engine, and until now engine/ was never refreshed.
• Tax manifest wording. It now says exactly what is encrypted (returns, uploaded documents) and what is not (the figures read from documents). It also notes that plain copies may remain in earlier backups and on disk.
• Tests and versions.
  • bundle-form-friction: the campaigns/tax exemptions are gone. An MCP-server bundle's generated key must be in server.envKeys.
  • Tax's previous_key Configure test now runs on a typed field, plus a new test that Configure never overwrites the generated key.
  • campaigns 1.0.3, tax 2.1.0, registry rebuilt.

Tests

• tests/tax-documents-crypto.test.js (12 tests):
  • round trip and key id;
  • the example key is refused;
  • migration;
  • crash and sibling cleanup;
  • a lost compare-and-swap;
  • two concurrent migrators;
  • containment (a path outside the folder, a symlink);
  • re-encryption after a key change;
  • a row from another home;
  • reverse in place and export.
• tests/campaigns-tax-generated-keys.test.js (10 tests):
  • manifest flags and the example-key move;
  • contract validation;
  • panel key order;
  • the tax panel mounted on loopback: upload stored only encrypted, start-up migration, delete containment, no-key refusal;
  • MCP ingest of a .enc file;
  • the refresh dirs.
• Full suite on this branch: 9751 tests, 1 failure (the tax Configure test, adapted in f432fb2), 0 after the fix. Tax's own bundles/tax/tests give the same result before and after (52 pass, 1 failing integration flow, which predates this branch).

Live proof (scratch home only)

Setup:

• Every data path, HOME included, was in a session scratch directory, checked before running.
• No key was in the env.
• A stub npm ensured nothing was downloaded.
• A and B were two separate node processes under systemd-run --user -p RuntimeMaxSec=1800, with loopback-only HTTP.
• The run was registered in the box schedule and cleared after.

Home A:

1. Upgrade. Tax 2.0.3 (from origin/main) was installed on the example key, with a plain W-2 in the legacy folder.
   • The boot refresh moved it to 2.1.0 and shipped engine/.
   • The backfill did not save the example key.
2. Reinstall. A new key was generated and the example key moved to _PREVIOUS. No key value appeared in the job log.
3. Tax panel, mounted from the installed copy:
   • the start-up pass encrypted the old W-2;
   • an upload returned 201;
   • both documents are stored only as .enc, with no plaintext in either documents folder.
   • A return was saved, encrypted.
4. Campaigns. It installed, and the panel found its generated key in the MCP entry. A Reddit credential was saved, encrypted.
5. Keychain. It holds both keys.
   • The Nest reminder said "2 saved extension password(s) are not in a keychain Export yet…".
   • The keychain was exported through the real HTTP route after re-auth, and the reminder then went away.
6. The backup was a consistent VACUUM INTO copy of crow.db plus the .enc files. Home A was then deleted.

Home B:

1. A reinstall before the Import was refused.
2. An HTTP Import brought in 2 entries.
3. The reinstall logged "Restored 1 key(s) from Crow keychain". The tax and campaigns keys were equal to A's (compared by fingerprint).
4. A's return, both documents and the Reddit credential all decrypted on B.

Licensing

No new third-party component.

https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp

[kh0pper]

I ran a fresh code and security review and pushed the fixes in 66bc1fc (rebased on #512's d11b6b2).

• Document key id. It is now an HMAC under the salted PBKDF2 output, computed per file, not under the passphrase. A stored file no longer offers a cheap offline check for passphrase guesses, and files encrypted with the same key can't be linked to each other.
• MCP ingest. Before reading, it now checks the file extension (.pdf or .enc), that the path is a regular file, and a 25 MB cap. The AI chooses the path, so /dev/zero, a FIFO, or a symlink to a device is refused without being read.
• Restored crow.db. If the backup was taken before the migration, its rows still name the plain file. They now find the single <name>.<8hex>.enc that replaced it, and the row is repointed.
• Documents lock.
  • A stale lock is taken over by an atomic rename, so only one run can win.
  • The lock is touched after each row.
  • The folder is tightened to 0700.
  • The panel logs when the lock is held.
• Uploads. File names now include a random part, so two uploads in the same millisecond never share a file.
• Example key.
  • The upload refusal and the install log now tell the user to reinstall.
  • The log also explains why the key was left unchanged when the previous-key slot is already in use.
• CLI. It now uses the database of the same CROW_HOME as its key and folders.

Results:

• Tests: full suite 9756 tests, 0 failures.
• Live proof: re-run in the scratch home after these fixes, and every check passes:
  • upgrade from 2.0.3 through the boot refresh;
  • move off the example key;
  • start-up encryption and upload;
  • Export through the HTTP route; home A deleted;
  • on home B, a reinstall before the Import is refused;
  • Import over HTTP, then reinstall: the keys match A's, and A's return, documents and Reddit credential all decrypt.

https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp