Plugin architecture, with MediaAnalyzer as the first plugin

[ralyodio]

DiskPush gets a plugin architecture, and MediaAnalyzer ships as its first, built-in plugin. With it you can describe the photos and videos in a local folder, and optionally sort them into folders by what they show, from the CLI, the TUI or the desktop app.

diskpush plugins                                   # installed plugins and their commands
diskpush mediaanalyzer login                       # OAuth 2.1 + PKCE in the browser
diskpush mediaanalyzer analyze ~/Pictures/2024 --sort
diskpush mediaanalyzer undo ~/Pictures/2024

Architecture

@diskpush/plugin-api (new, 0.11.0, in release.mjs MANIFESTS). This package is the contract.

• A plugin is a plain object: { id, name, version, description, commands?, actions?, tasks?, settings?, status? }. definePlugin() validates it when it is defined, rejecting bad or reserved ids, duplicates and a mismatched apiVersion.
• Every action, command and task gets one context, whatever the surface: dir / names / entries (actions only), namespaced settings (plugin:<id>:*) and secrets (plugin-secret:<id>:*) in the settings table the CLI and the desktop share, a progress sink, openUrl (http/https only), signal, surface and env.
• PluginRegistry handles register, list, enable/disable (persisted as plugins.disabled), actionsFor(entries, dir), and require/action/task/command lookups. runAction validates the selection: an absolute dir, bare names only, symlinks skipped. It also turns a throw into a failed result.
• External plugins load from <diskpush home>/plugins/node_modules, and only if the package is listed in that directory's package.json and has a "diskpush" field. diskpush plugins add installs with --ignore-scripts and uninstalls again if the package does not load as a plugin. One broken plugin is reported and skipped.

Surfaces

CLI	diskpush <plugin-id> ... is decided on raw argv, before DiskPush's own parser, so a plugin's flags never trip DiskPush's value flags. diskpush plugins list|enable|disable|add|remove. --help lists plugin commands. Progress uses Output's status line. Ctrl+C cancels through the signal.
TUI	a opens an Actions overlay for the local row under the cursor. Keys: ↑↓/enter, or the action's letter. One click runs the row under the pointer, hover lights it, and the footer line describes the lit action. A running action gets a job panel in the transfer panel's place: esc cancels it, and a finished one is dismissed like a transfer. The a cap goes last in the key bar so q quit still fits at 100 columns. Frames stay pure and are asserted with renderToText.
Desktop	Plugin code runs in the main process only. New channels: plugins:list, actions-for, run-action, run-task, cancel, get-settings, set-settings, set-enabled, plus the plugins:progress event. The preload has named methods (plugins.*, events.onPluginProgress), still with no generic invoke. A local pane's right-click menu has a Plugins section. A plugin job reuses ActiveJob/TransferBand, which learned a job with no bytes or rate. Plugins… in the header menu can enable/disable plugins, edit their declared settings (secrets are write-only) and run tasks, e.g. Sign in to MediaAnalyzer.

@diskpush/plugin-mediaanalyzer (new, 0.11.0, in MANIFESTS)

• Auth: exactly the MediaAnalyzer CLI flow, as client diskpush. It uses a loopback 127.0.0.1:<port>/callback, S256 PKCE, state and device_name. Refresh tokens rotate: the new pair is stored the moment it arrives, only one refresh runs at a time, and the refresh token is re-read from the store before each refresh, because the other surface may have rotated it. A 401 triggers one retry after a refresh. Logout revokes. login --paste uses ${server}/oauth/code for headless machines. An ma_key_… key can come from DISKPUSH_MEDIAANALYZER_KEY or the api_key secret.
• Scan: tier is chosen from the setting, else the first online tier, else byok with the first provider. Originals are uploaded (no sharp, no native addons) in multipart batches of ≤50 files and ≤64 MB. client_ref uses the MediaAnalyzer CLI's formula, so the two tools never double-charge each other. Videos become a 3×3 ffmpeg contact sheet when ffmpeg/ffprobe are on PATH, and are skipped with a note otherwise. On 402 the upload stops, accepted files are kept, and the result says how many were skipped and where to add credit. 503 tier_offline fails with a clear message. A cancelled scan resets.
• Results: it polls finished_after every 2.5s and dedupes the inclusive cursor by client_ref. It writes <file>.description.txt with the signature line and never overwrites a sidecar that lacks it. State lives in <dir>/.mediaanalyzer/diskpush-state.json and is saved after every batch and every poll, so a re-run resumes and never re-uploads. A file that already has a signed sidecar is skipped.
• Sort and undo: each file and its sidecar move into <dir>/<Category>/ via link+unlink, which is no-clobber and atomic, with a check+rename fallback for FAT/exFAT. Clashes become (2). Each move is journaled to undo-*.json before it happens. Undo last sort walks the latest journal backwards, skips anything the user has since replaced, and removes folders the sort created if they are empty.

Security model (please read)

• Plugins run with the user's full privileges, in the CLI process or the Electron main process, never the renderer. plugins add warns before installing. docs/plugins.md says this plainly.
• The renderer can only name a plugin/action by id, plus an absolute local dir and bare EntryNameSchema names, validated by zod. Tests show it rejects ../, a/b, relative, host:/path and NUL, and strips extra fields. Only declared settings can be written, and each is type-checked. Secret values are never sent to the renderer.
• This changes a stated rule, and it is your decision: docs/security.md said persisted secrets belong in OS secure storage. Plugin sign-ins (the MediaAnalyzer refresh token) are stored in the settings table instead, because it is the only store both the CLI and the desktop can read, and a sign-in has to work in both. To offset that, diskpush.db is now chmod 0600 on open; it was 0644 under the default umask on this machine. security.md and the README now say so, and also that a plugin you run may upload the files you pick. A SecretCodec hook exists for a keychain; see the follow-ups.

Tests

Clean run in CI order: pnpm install --frozen-lockfile, pnpm build, pnpm typecheck, pnpm test, pnpm --filter @diskpush/web build, pnpm --filter @diskpush/desktop build, pnpm smoke:desktop. All exit 0.

• pnpm test: 59 files, 779 tests passed (baseline on origin/main 3d92cc2: 53 / 723).
• pnpm smoke:desktop: the three bundle/scheme/CSP checks pass. The Electron launch step was skipped locally (libatk-1.0.so.0 is missing on this box), so CI's run of it is the real check.
• pnpm test:integration was not run; it needs the SSH test container and nothing it covers changed.

New tests:

• plugin-api: registry enable/disable persistence, actionsFor, a broken appliesTo isolated, namespacing, the runAction path guard, external loading (listed + marked only, bad entry point refused, add uses --ignore-scripts and rolls back a non-plugin).
• plugin-mediaanalyzer, against a real local fake HTTP server: PKCE exchange (client id, redirect, verifier), three rounds of refresh rotation with concurrent calls sharing one refresh and no replay, 401 retry, revoke on logout, batching 120 → [50, 50, 20], 402 mid-run ([50, 50], 70 skipped), resume with no re-upload, finished_after with delayed results and an error file, sidecar format, a foreign sidecar left alone, a signed sidecar not re-charged, videos with and without ffmpeg, sort with clashes plus undo restoring the exact tree byte for byte, undo skipping a replaced file.
• CLI: dispatch to a fake plugin (raw argv, exit code passthrough, --json, help, unknown command, disabled refusal), diskpush plugins list/enable/disable against a real store, key assignment.
• TUI: renderToText frames for the overlay (with hover description) and the job panel (running and failed), key-bar fit at 100/120 columns, and a fake-host interaction: a → enter, letter on a nested row, one click runs, esc cancels through the signal, and the refusal messages.
• Desktop: plugin contract schemas reject free paths.
• Database: the store file ends up 0600.

Manual CLI smoke on the built binary with a throwaway DISKPUSH_HOME: help, plugins, plugin help, not-signed-in, a disabled plugin exits 65, --json plugins enable, and unknown command/subcommand all behave as intended.

Follow-ups / risks

• Secrets in an OS keychain that both the CLI and the desktop can reach (libsecret/Keychain via their CLIs, or safeStorage plus a CLI bridge). Wiring safeStorage in the desktop alone would lock the CLI out of the same sign-in, so it is not wired yet.
• The OAuth and API flows were tested against a fake server only, not live mediaanalyzer.pro. The first real run should be a small folder.
• plugins add needs npm on PATH (the desktop bundle has none); there is no desktop UI for adding plugins yet.
• The TUI acts on the single row under the cursor, since it has no multi-select. The desktop acts on the whole selection.
• Plugin actions are local-only; there are no remote panes yet.
• A plugin job and a transfer share the band and busy, so they do not run at the same time.

🤖 Generated with Claude Code