Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/IMPROVEMENT-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Low-risk housekeeping that clears the way for the bigger phases.
There are about 300 tests, but several core modules have none of their own. This phase covers the guarantees that 1.0 commits to.

### 2.1 Tests that GPU and CPU return the same scores
- **Browser coverage extended:** the existing `scripts/test-regression.ts` differential gate now includes joined emoji and component queries, plus 128-token ASCII and case-fold-expanded queries in substring and fuzzy modes. Matching records and last-token near misses exercise the actual WGSL path against the CPU reference. CI already runs this gate on every PR.
- Add `test/scoring-parity.test.ts`. It runs a fixed set of queries through the mock WebGPU path and through `cpu-reference.ts` / `exact-scorer.ts`.
- Check that both return the same `SearchResultItem[]`: same order, same integer scores, scores in descending order.
- Cover ASCII, CJK, emoji/ZWJ, mixed case, and queries at the maximum length.
Expand Down
29 changes: 26 additions & 3 deletions scripts/test-regression.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import puppeteer from 'puppeteer-core';
import path from 'path';
import fs from 'fs';
import { normalizeText, scoreExactMatches } from '../packages/webgpu-search/src/index';
import { normalizeText, scoreExactMatches, QUERY_TOKENS_MAX } from '../packages/webgpu-search/src/index';
import {
getChromeExecutablePath,
getChromeLaunchArgs,
Expand Down Expand Up @@ -180,6 +180,12 @@ async function main() {
// agree on NFC for this ASCII-heavy corpus (negligible risk here;
// cross-runtime NFC is probed/reported, not gated).
const FCP = String.fromCodePoint;
const joinedEmoji = FCP(0x1f469, 0x200d, 0x1f4bb);
// A distinct final token catches dispatch/scoring paths that truncate
// the query at 127 tokens. The folded query reaches the same cap with
// only 64 input characters, so the limit is measured after folding.
const maxQuery = 'a'.repeat(QUERY_TOKENS_MAX - 1) + 'z';
const foldedMaxQuery = FCP(0xdf).repeat(QUERY_TOKENS_MAX / 2);
const parityStrings: string[] = [];
for (let i = 0; i < 200; i++) {
parityStrings.push(`src/services/Auth${i % 7 === 0 ? FCP(0xdf) : 's'}Controller_${i}.ts`);
Expand All @@ -189,7 +195,16 @@ async function main() {
FCP(0x5317) + FCP(0x4eac) + 'city',
FCP(0x1f600) + 'smile',
'caf' + FCP(0xe9),
'plain'
'plain',
joinedEmoji,
'Dev ' + joinedEmoji + ' at work',
FCP(0x1f469) + ' ' + FCP(0x1f4bb), // Separate scalars are not the ZWJ sequence.
maxQuery,
'prefix ' + maxQuery + ' suffix',
'a'.repeat(QUERY_TOKENS_MAX - 1) + 'x', // Differs only at the last token.
foldedMaxQuery,
'ss'.repeat(QUERY_TOKENS_MAX / 2),
'ss'.repeat(QUERY_TOKENS_MAX / 2 - 1) + 'sx'
);
const parityCells = [
{ query: 'auth', mode: 'substring', limit: 50 },
Expand All @@ -200,6 +215,14 @@ async function main() {
{ query: 'SS', mode: 'substring', limit: 50 },
{ query: FCP(0x5317), mode: 'substring', limit: 10 },
{ query: FCP(0x1f600), mode: 'substring', limit: 10 },
{ query: joinedEmoji, mode: 'substring', limit: 10 },
{ query: joinedEmoji, mode: 'fuzzy', limit: 10 },
{ query: FCP(0x1f4bb), mode: 'substring', limit: 10 },
{ query: FCP(0x1f4bb), mode: 'fuzzy', limit: 10 },
{ query: maxQuery, mode: 'substring', limit: 10 },
{ query: maxQuery, mode: 'fuzzy', limit: 10 },
{ query: foldedMaxQuery, mode: 'substring', limit: 10 },
{ query: foldedMaxQuery, mode: 'fuzzy', limit: 10 },
{ query: 'zzz-no-match', mode: 'substring', limit: 50 },
{ query: 'zzz-no-match', mode: 'fuzzy', limit: 50 },
] as Array<{ query: string; mode: 'substring' | 'fuzzy'; limit: number }>;
Expand Down Expand Up @@ -258,7 +281,7 @@ async function main() {
}
}
(testResults as any).results.push({
name: `parity ${cell.mode} ${JSON.stringify(cell.query.slice(0, 12))} (ordered index/score/text)`,
name: `parity ${cell.mode} ${JSON.stringify(cell.query.slice(0, 12))} (${qT.length} tokens; ordered index/score/text)`,
passed: pass,
details: { totalMatches: got.totalMatches, candidateCount: got.candidateCount, expected: expected.totalMatches }
});
Expand Down
Loading