fix: reclaim SQLite space after removing documentation

[arabold]

Fixes #476.

SQLite does not shrink the store file on DELETE. Removed rows become free pages, and WAL mode records those deletes in documents.db-wal, so on-disk size can grow after a remove.

Changes

• After bulk deletes (remove / removeAllDocuments), run wal_checkpoint(PASSIVE) only. That never waits on readers or writers, so search stays available. It clears WAL growth from the delete; it does not rewrite documents.db.
• Add docs-mcp-server compact (and a tRPC compact mutation) for an explicit VACUUM. This takes an exclusive lock and may block searches — run it when the store is idle.
• In-memory stores skip compaction. Single-page deletes during refresh do not checkpoint or vacuum. Remove still succeeds if the post-delete checkpoint fails.

Docs

• docs/concepts/data-storage.md, docs/guides/basic-usage.md, and skills/docs-manage/SKILL.md cover the WAL vs VACUUM split.

[Copilot]

🟡 Changes recommended

There are a couple of correctness/robustness issues in the new compaction flow (filesystem size accounting can throw, and service logging can be misleading when WAL truncation reclaims space without VACUUM).

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR addresses SQLite database file growth after document removals by adding lightweight post-delete WAL maintenance and introducing an explicit compaction flow (VACUUM) that can be triggered when the store is idle.

Changes:

• Run a best-effort non-blocking WAL checkpoint after bulk deletes to curb WAL growth without blocking readers.
• Add an explicit compaction API/CLI path (compact) that can run VACUUM (exclusive lock) and report reclaimed space.
• Add a small formatBytes helper plus targeted tests, and update docs/skill guidance to explain WAL vs VACUUM behavior.

File summaries

File	Description
src/utils/string.ts	Adds formatBytes() helper for human-readable size output.
src/utils/string.test.ts	Unit tests for formatBytes().
src/store/types.ts	Introduces CompactResult type for compaction reporting.
src/store/trpc/router.ts	Adds compact mutation to the data router.
src/store/trpc/router.test.ts	Tests compact behavior for in-memory store.
src/store/trpc/interfaces.ts	Extends IDocumentManagement with compact().
src/store/DocumentStore.ts	Implements size accounting and compact() (checkpoint + optional VACUUM).
src/store/DocumentStore.test.ts	Adds integration-style tests for compaction behavior.
src/store/DocumentManagementService.ts	Runs checkpoint after bulk deletes; adds service-level compact() wrapper + logging.
src/store/DocumentManagementService.test.ts	Updates mocks/assertions and adds coverage for compaction failure tolerance and explicit compaction.
src/store/DocumentManagementClient.ts	Adds client wrapper for compact mutation.
src/cli/index.ts	Registers new compact CLI command.
src/cli/commands/compact.ts	Adds docs-mcp-server compact command implementation and output.
src/cli/commands/compact.test.ts	Unit test for the compact command output and invocation.
skills/docs-manage/SKILL.md	Documents the new compact workflow for agents.
docs/guides/basic-usage.md	Adds compact to CLI command list and output behavior notes.
docs/concepts/data-storage.md	Explains WAL checkpoint vs VACUUM and when each is used.

Review details

• Files reviewed: 17/17 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.