Add preloaded hybrid database with parallel startup

[quolpr]

Summary

• add PreloadedHybridDB with ID-only preloaded B-tree and uniqhash indexes plus on-demand row hydration
• support transactional hash indexes, external-storage merges, and safe IndexedDB bulk scans
• preload tables with configurable concurrency, defaulting to "whole"
• document the API and startup memory tradeoffs

Validation

• pnpm --filter @will-be-done/hyperdb test (866 passed, 16 skipped)
• HyperDB lint, cycle check, build, docs, and format checks
• web integration unit/type/lint checks and Playwright e2e (24 passed)

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by CodeRabbit

• New Features
  • Preloaded tables now load concurrently by default, which can reduce startup time when loading multiple tables.
  • Set a positive concurrency limit to control how many tables load at once, or use "whole" to load all supplied tables concurrently. Use a limit of 1 for sequential loading.
  • Higher concurrency may temporarily increase memory use, and storage drivers may still serialize reads.
• Documentation
  • Updated setup guides and reference documentation with loading options and behavior.

[codesandbox]

Review or Edit in CodeSandbox

Open the branch in Web Editor • VS Code • Insiders

Open Preview

[netlify]

✅ Deploy Preview for hyperdb ready!

Name	Link
🔨 Latest commit	b0d7cf8
🔍 Latest deploy log	https://app.netlify.com/projects/hyperdb/deploys/6abe425ebbf3a1000898c1cb
😎 Deploy Preview	https://deploy-preview-27--hyperdb.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: ace5b778-979a-4dcc-9a41-6c0cd75d4bc6

📥 Commits

Reviewing files that changed from the base of the PR and between 6049b27 and b0d7cf8.

📒 Files selected for processing (1)

• .changeset/parallel-table-preloads.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

PreloadedHybridDB now loads supplied tables concurrently by default. The preloadConcurrency option accepts a positive integer or "whole". Tests cover bounded and sequential loading, preload failures, and synchronous and asynchronous drivers.

Changes

Preload concurrency

Layer / File(s)	Summary
Runtime preload scheduling packages/hyperdb/src/hyperdb/runtime/preloaded-hybrid-db.ts	PreloadedHybridDB validates and stores the concurrency setting. It selects sequential or concurrent loading, preserves successful table states, and reports preload errors.
Driver coverage and option documentation packages/hyperdb/src/hyperdb/runtime/preloaded-hybrid-db.test.ts, packages/hyperdb/src/hyperdb/drivers/idb/idb-driver.scan-all.test.ts, README.md, packages/hyperdb/README.md, packages/hyperdb-doc/src/content/docs/runtime/db.md, packages/hyperdb-doc/src/content/docs/start/llm-cheat-sheet.md, .changeset/parallel-table-preloads.md	Tests cover preload limits, error handling, and synchronous and asynchronous drivers. Documentation describes concurrency settings, driver scheduling, and temporary memory effects.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant PreloadedHybridDB
  participant PreloadWorkers
  participant PrimaryDriver
  PreloadedHybridDB->>PreloadWorkers: schedule table scans using preloadConcurrency
  PreloadWorkers->>PrimaryDriver: call scanAll for each table
  PrimaryDriver-->>PreloadWorkers: return rows or scan error
  PreloadWorkers->>PreloadedHybridDB: provide completed table states

Loading

Merge Risk: ⚪ Minimal · up to b0d7c

Preloaded tables now load concurrently by default, with configurable limits and documented memory tradeoffs. The supplied implementation and test summaries establish no material issue that should block merging.

Architecture Summary

Architecture risk: 🔵 Low · up to b0d7c

The change affects 3 systems.

Changed systems: packages/hyperdb, packages/hyperdb-doc, README.md

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

• observed — packages/hyperdb (library) was modified; 4 changed files map to changed impact.
• observed — packages/hyperdb-doc (library) was modified; 2 changed files map to changed impact.
• observed — README.md (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

• observed — Modified behavior in README.md: Documented default concurrent table preloading for PreloadedHybridDB, including numeric concurrency limits, the explicit "whole" mode, driver-internal serialization, and temporary retention of multiple decoded tables.
• observed — Modified behavior in packages/hyperdb-doc/src/content/docs/runtime/db.md: The example now configures PreloadedHybridDB with preloadConcurrency: "whole" when wrapping it in SubscribableDB, replacing the default-construction example.
• observed — Modified behavior in packages/hyperdb-doc/src/content/docs/runtime/db.md: Documentation now states that tables in a loadTables call preload concurrently by default, supports positive integer concurrency limits (1 for sequential loading), uses driver-dependent scheduling, builds each table’s ID-only indexes as its scan completes, and may increase peak startup memory with higher concurrency.
• observed — Modified behavior in packages/hyperdb-doc/src/content/docs/start/llm-cheat-sheet.md: The PreloadedHybridDB example now explicitly selects "whole" preload concurrency. The surrounding documentation adds concurrency controls: default concurrent loading, positive numeric limits including sequential 1, and "whole" to start all supplied tables, along with driver serialization and temporary memory behavior.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files. (1 skipped: 1 …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: adding a preloaded hybrid database with parallel startup. It is concise and directly related to the implementation and documented behavior.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}