Skip to content

refactor: extract document and GPU engine modules - #45

Merged
codewarnab merged 2 commits into
mainfrom
refactor/document-gpu-modules
Oct 22, 2026
Merged

codewarnab merged 2 commits into
mainfrom
refactor/document-gpu-modules

Conversation

@codewarnab

Copy link
Copy Markdown
Owner

Extract DocumentIndex and WebGPUEngine implementation into focused internal modules while preserving public APIs, search behavior, snapshots, lifecycle safety, and portable runtime support.

  • DocumentIndex delegates ingestion/mutations, querying, filtering/faceting, and snapshots to document/ modules with explicit internal state.
  • WebGPUEngine coordinates buffer management, pipeline setup, and dispatch/readback through gpu/ modules.
  • Update affected tests to use public methods and injected GPU device boundaries instead of private fields. Replace the maxResults source-expression check with runtime alias/precedence checks.

Validation: independent OpenCode GLM 5.3 Flash (high) review approved the final diff. Passed check:shaders, typecheck, build, test:mock, check:public-api, test:records, test:faceting, test:parity (127 passed, 1 pending hardware), and engine-pool-regressions (12 passed). Mutation, snapshot, and worker suites also passed.

Known limitation: test:filtering exceeds its 50µs timing threshold in this environment; independent review reproduced the failure on the original HEAD. Its behavior checks pass and the threshold is unchanged.

This branch also inherits the existing browser parity test commit 89bf26b. The pre-existing local edit in api/submit-benchmark.ts is excluded.

Copilot AI balanced review requested due to automatic review settings October 21, 2026 19:26
@vercel

vercel Bot commented Oct 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
webgpu-search Ignored Ignored Oct 21, 2026 7:26pm UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codewarnab

Copy link
Copy Markdown
Owner Author

Independent verification — 5 review agents + orchestrator check run ✅

Consolidated from 5 independent read-only review agents (semantic parity, GPU subsystem, test/script audit, portability/clean-diff, adversarial concurrency probe) plus an orchestrator check pass. Review basis: commit 83b0e51 ("refactor: extract document and GPU engine modules") diffed against base 89bf26b.

Checks executed (all PASS)

  • bun run check:shaders / bun run typecheck / bun run build / bun run test:mock — green
  • bun run check:public-api / bun run test:records / bun run test:faceting / bun run test:parity — green
  • bun test packages/webgpu-search/test/engine-pool-regressions.test.ts — 12/12
  • bun test packages/webgpu-search/test/document-index-integrity.test.ts — 13/13 (agent-added re-verification)

Verdicts by dimension

  • Semantic parity (document/ modules vs base monolith): PASS — all extracted sections (option validation order, batch ops remove→update→add, enqueue/queued mutex, compaction thresholds, validateSearchOptions fail-closed checks + messages, filter/facet phase gates, snapshot merge/apply order, generation/abort/tombstone sentinel sequencing) are behavior-preserving verbatim moves. No reordered validation, changed early-exits, altered error construction, or dropped guards.
  • GPU subsystem (webgpu-engine facade + gpu/{state,buffers,pipelines,dispatch}.ts): PASS — buffer usage/portability tables verbatim; safe-swap ordering, appendRows guarded headroom writes, single/chunked dispatch (counter-reset fallback, _pad.x = base row, timestamps skipped when chunked), epoch-guarded readbacks, and the three distinct teardown paths (re-init unmap-each / destroy unmap-staging-pair / device-loss null-refs keeping allocated sizes) all match base. dispatchChunked param change (workgroupSize → WORKGROUP_SIZE const; currentDatasetSize → param) verified value-equivalent at the sole call site.
  • Test/tool-script audit: PASS — all prior finding repairs verified as moves to real boundaries, with strictly stronger coverage: queue.submit failure injection + fallbackReason === 'gpu-execution-error' assertion; faceting simulator drives the engine's real copy→mapAsync→readMappedOutput decode (byte-layout matches gpu/dispatch.ts, provably cannot touch the 16-byte ts-staging copy, disarms cleanly); records stratification/overflow assertions distinguish missed-formula/clamp-up/clamp-down/NaN leaks; parity harness withCapturedStaging + real ensureCandidateCapacity/destroy epoch races; engine-pool ts-staging injection at device.createBuffer. No assertion-free probes, mock mirrors, source-string greps, or test-only production seams remain.
  • Portability / frozen API / clean diff: PASS — all navigator refs guarded, zero DOM refs, device acquisition stays in GpuDevicePool, no new exports (InternalField/assertSnapshotFilterGettersSatisfied re-exports keep worker/snapshot-codec seams; facade method set byte-identical to base), no debug artifacts, index.ts + public-api-surface.txt unchanged.
  • Adversarial probe (concurrency/snapshot/state/error identity): PASS — mutex semantics and in-mutex destroyed re-check preserved, options write-back and hasGetter restore guards intact, PreparedDocMutation consolidation field-faithful, error types unaffected.

Non-blocking follow-up (1)

  • src/webgpu-engine.ts:523 still carries the frozen limit ?? maxResults ?? 50 alias, but the reworked check:public-api behavioral gate covers only SearchIndex + DocumentIndex (both vectors run preferGpu:false). The GPU-path alias is unequipped by any gate; suggest a follow-up gate vector with preferGpu:true + mock device. Accepted as tooling coverage debt, not a defect in this diff.

Known limitation (pre-existing, not caused by this PR)

  • test-filtering perf gate misses its 50µs target (~107µs) on base 89bf26b/main as well as this branch — threshold intentionally left unchanged.

Recommendation: approve. Library semantic parity, portability, public-surface freeze, and clean-diff audits all sign off.

@codewarnab
codewarnab merged commit 98154e4 into main Oct 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants