Skip to content

perf(security): one-pass multi-literal prefilter for ScanSQL [W6-08] - #942

Merged
ajitpratap0 merged 3 commits into
mainfrom
perf/w6-08-multi-literal-prefilter
Nov 3, 2026
Merged

perf(security): one-pass multi-literal prefilter for ScanSQL [W6-08]#942
ajitpratap0 merged 3 commits into
mainfrom
perf/w6-08-multi-literal-prefilter

Conversation

@ajitpratap0

@ajitpratap0 ajitpratap0 commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

W6-08 (SEC-3). Scanner.ScanSQL no longer runs ~70 bytes.Contains calls per statement. One case-folding Aho-Corasick pass (62 literals, 309 states) sets a presence bitset; each of the 36 patterns checks an alternation/optional/assertion-aware CNF filter derived from its regex. The fold buffer and pool are removed. Findings are byte-identical; no exported API change.

Also mechanically updates wp_p_bypass_test.go (CNF iteration, nil -> scanPrefilter{}), outside the brief's owned files but required to compile.

Test Plan

  • go vet, go test -count=1 ./pkg/sql/security/... pass; Codex ran race (162s) and FuzzLiteralMatcher 60s (877k execs): pass.
  • Seven mutations (M1-M7) each killed by the new tests.
  • task check, task test:cbinding, go test ./pkg/compatibility/..., staticcheck on pkg/sql/security: pass.
  • Local indicative (shared M4, Codex, 2 interleaved rounds): RW -48% to -78%, Params -52% to -57%, Large/30KB -44%, Large/1MB -18%; allocs/op unchanged, B/op not up. My own task bench:ab could not run (disk full).
  • CI perf-gate verdict (authoritative, layout-randomized gate, 20 -randlayout layouts per arm, per-layout Mann-Whitney U, A/A', Benjamini-Hochberg). ScanSQL is not in the gated suite, so run 41682394942 on probe branch perf/w6-08-gate-probe (0828262 = PR head b61cdd5 plus one gate-suite.txt line, never merged; base a72ec74; AMD EPYC 7763):
    • ScanSQLRW, all 8 dialects: -53.9% to -59.7%, p<0.001; A/A' within +/-0.24%; instructions/op -53% to -59%. This beats the PLAN threshold of -10% roughly 5x.
    • ScanSQLParams: -45.0% to -50.6%, p<0.001.
    • ScanSQLLarge/30KB: -1.31% (n.s., p=0.102). ScanSQLLarge/1MB: -0.01% (n.s.), allocs 38->35, B/op -5.55%.
    • allocs/op otherwise unchanged, B/op -0.01%. No regression in the rest of the gated suite. Low-power list holds only ParseE2E/aggregate and ParseE2E/subquery, which are not targets.
    • The PR's own shadow-gate comment (head dfbf28c) shows no gated metric moving, but ScanSQL is not in that suite, so the probe run is the authoritative verdict.

Out of scope

Caching isASCII per view (the W6-01 follow-up) is deliberately not done: with alternation-aware filters the ClickHouse guard is rarely reached, and the identifier-tautology guard already computes isASCII once per view.

Breaking Changes

None.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2

Test @claude
perf(security): one-pass Aho-Corasick literal prefilter for ScanSQL […
b61cdd5 
…W6-08]

Replace per-literal bytes.Contains checks with a single case-folding
Aho-Corasick pass and alternation-aware CNF filters derived from each
regex. Findings are byte-identical; fold buffer and pool removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
@vercel

vercel Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Nov 3, 2026 9:11pm UTC

@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 53s —— View job

Review of PR #942: one-pass multi-literal prefilter for ScanSQL

I read litmatch.go, prefilter.go and the prefilter call sites in scanner.go. I did not run the tests or benchmarks. The checkout is shallow (git diff origin/main...HEAD fails with "no merge base"), so I read the files directly instead of diffing. I did not read the test files, so I haven't checked the claims about mutation or fuzz coverage.

Verdict: The design is sound and I found no correctness bugs. There are a few things worth a second look, listed below.

What I checked and found correct

  • Aho-Corasick construction (litmatch.go).
    • The BFS builds outputs in depth order, so each failure state is finished before it is merged. Suffix matches therefore propagate correctly.
    • The uint16 state index is guarded by the 65535-state check, and the 128-literal limit matches the litSet width.
  • Case folding.
    • ASCII fold goes through the shared byte classes.
    • The two non-ASCII simple-fold characters that fold to ASCII are handled inline: U+017F (ſ → s) and U+212A (K → k). Folding them keeps the prefilter a superset of what (?i) regexes match.
    • Skipping the extra bytes with i += 1 or i += 2 keeps the DFA transitions equivalent to a single class step.
  • CNF derivation (requiredLiterals).
    • It is conservative. Anything it can't prove returns an empty literalInfo, which only weakens the filter and never skips a real match.
    • Alternation clauses union the exact sets or the best clause per branch.
    • ? and * break the run, + and {n,} keep the sub-literal, and assertions are treated as an exact empty string.
    • Empty strings in a clause drop the clause, so a| can't create a false requirement.
    • Non-ASCII literal runs are discarded.
  • Comment-normalized views. I checked the comment in scanner.go that says removing comments can't introduce a literal absent from the raw input. In normalizeCommentView, comments are replaced with a space (out.WriteByte(' ')) rather than deleted, so un/**/ion can't turn into union. The comment is accurate for that path.

Things to double-check

  1. The escapes and E'...' view modes (sqltext.go, commentViews modes 1 and 16) interpret backslash escapes. If a view can ever decode an escape into a letter that is not present in raw, a prefilter literal could appear only in that view. That would be a false skip. For example, \x75nion becomes union only if such decoding exists. The "literals contain no whitespace" argument covers comment stripping but not decoding. Please confirm that no mode synthesizes letters. If one does, scan each view, or build the presence set from the raw input and the views together. The differential test (prefilter on versus off) should include escape-bearing inputs. The fuzz target would be the place to add them.
  2. Init-time panics. buildLiteralMatcher panics if the literal count exceeds 128 or the state count exceeds 65535. That is reasonable today (62 literals, 309 states). But a future pattern added to the tables could make the package panic on first use, and initPatterns runs lazily inside sync.Once. A test that asserts headroom would turn this into a CI failure instead of a runtime one. Test files may already cover it.
  3. guard plus isASCII(sql) check per view (scanner.go, around line 1351). isASCII runs once per pattern per view. If it scans the whole input each time, large inputs lose part of the gain. Computing it once per view would avoid that. I did not measure this.
  4. wp_p_bypass_test.go was edited outside the brief's owned files. The PR description discloses this, and I agree the change is needed to compile.

Performance and tests

  • The approach is the right one: a single O(n) pass with a dense []uint16 transition table, instead of about 70 bytes.Contains calls plus a fold buffer. Removing the pool also removes a shared mutable object.
  • The matcher and filters are read-only after sync.Once, so concurrent use is safe.
  • The CI perf-gate verdict is still pending. Please post it before merge, since ScanSQL is not in the gated suite.
  • Test coverage looks thorough: the PR cites an on/off reference path, FuzzLiteralMatcher and seven mutations. Item 1 above is the one input class I'd add to the differential or fuzz coverage.

No blocking issues from my read, assuming item 1 holds.
· branch perf/w6-08-multi-literal-prefilter

@github-actions github-actions Bot added documentation Improvements or additions to documentation security labels Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Perf gate: PASS (shadow mode: report only, this check never fails)

Base 1f56d2215296, head 25acf3e7fef4. 20 interleaved rounds, 20 randomized code layouts per arm (-randlayout seeds 417193019202..417193019221; A/A' 417193019222..417193019241), benchtime 100ms, -cpu 1, GOMEMLIMIT 64MiB, pinned to CPU 3 (SMT siblings 2-3) with taskset.

Runner CPU: AMD EPYC 7763 64-Core Processor (x86_64). Base and head ran interleaved in this job, so the tables show their ratios only; absolute timings (raw files in the artifact) hold for this CPU and this job alone and are never compared across jobs.

  • no gated metric moved past its threshold

Rules: each arm runs 20 randomized code layouts (-ldflags=-randlayout); time regresses at >= +3% when significant at a false discovery rate of 0.01 over the suite (Benjamini-Hochberg on exact Mann-Whitney U p-values of the per-layout medians, so a regression must hold across layouts, not within one binary), or a suite geomean >= +1.5% (per-layout suite geomeans, same test); allocs/op must not increase; B/op must not rise more than 2%; instructions/op must not rise more than 1.5%. The A/A' control runs the base revision under a disjoint set of layout seeds; when more than 10% of benchmarks move over 3% in it, the run is noisy: re-run once, then inconclusive (neutral).

Gated suite (GOGC=off, GOMAXPROCS=1)

benchmark time p A/A' spread B/op allocs/op status
ParseE2E/simple_select +0.28% 0.725 +0.31% 1.90% +0.00% 5 → 5 ok
ParseE2E/select_where -0.36% 0.365 -0.54% 1.57% +0.00% 18 → 18 ok
ParseE2E/join +0.13% 0.675 -0.25% 0.64% +0.00% 17 → 17 ok
ParseE2E/aggregate +0.03% 0.946 -0.26% 3.02% +0.00% 18 → 18 ok
ParseE2E/subquery +0.30% 0.797 +0.27% 2.71% +0.00% 14 → 14 ok
ParseE2E/cte -0.31% 0.543 +0.10% 1.59% +0.00% 21 → 21 ok
ParseE2E/window +0.29% 0.607 -0.60% 1.68% +0.00% 11 → 11 ok
ParseE2E/complex -0.05% 0.850 -0.86% 2.09% +0.00% 49 → 49 ok
ParseE2E/insert -0.45% 0.588 -0.42% 2.18% +0.00% 17 → 17 ok
ParseE2E/update -0.44% 0.417 -0.60% 1.60% +0.00% 13 → 13 ok
ParseE2E/delete +0.00% 1.000 -0.81% 1.61% +0.00% 5 → 5 ok
ParseRW/postgresql/medium +0.38% 0.495 +0.14% 1.21% +0.00% 646 → 646 ok
ParseRW/mysql/medium +0.21% 0.565 +0.18% 1.37% +0.00% 368 → 368 ok
ParseRW/mariadb/medium +0.13% 0.841 +0.02% 1.87% +0.00% 528 → 528 ok
ParseRW/sqlserver/medium -0.51% 0.355 -0.14% 1.84% -0.00% 927 → 927 ok
ParseRW/oracle/medium -0.05% 0.989 +0.57% 1.95% -0.00% 1090 → 1090 ok
ParseRW/sqlite/medium -0.57% 0.429 -0.01% 2.08% +0.00% 347 → 347 ok
ParseRW/snowflake/medium +0.32% 0.398 +0.83% 1.05% +0.00% 776 → 776 ok
ParseRW/clickhouse/medium +0.80% 0.565 +0.54% 4.08% +0.00% 383 → 383 ok
ParseFresh/simple_select +0.03% 0.914 -0.70% 1.13% +0.00% 6 → 6 ok
ParseFresh/join -0.42% 0.211 -0.76% 1.38% +0.00% 18 → 18 ok
ParseFresh/complex -0.26% 0.725 -0.52% 1.15% +0.00% 50 → 50 ok
PoolGetPut/serial -0.15% 0.579 -0.37% 0.87% +0.00% 0 → 0 ok
Tokenize/scratch/join -0.24% 0.756 +0.60% 2.10% +0.00% 0 → 0 ok
Tokenize/scratch/complex -0.05% 0.968 +0.24% 2.65% +0.00% 0 → 0 ok
Tokenize/scratch/insert -0.38% 0.499 +1.42% 3.14% +0.00% 0 → 0 ok
Normalize/simple_select -0.34% 0.617 -0.59% 1.54% +0.00% 7 → 7 ok
Normalize/join -0.10% 0.850 -0.42% 1.58% +0.00% 23 → 23 ok
Normalize/complex -0.54% 0.298 -0.89% 1.51% +0.00% 68 → 68 ok
TransformParse/simple_select +0.00% 0.989 +0.00% 1.78% +0.00% 5 → 5 ok
TransformParse/join +0.19% 0.758 -0.24% 1.82% +0.00% 17 → 17 ok
TransformParse/complex -0.11% 0.820 +0.34% 1.80% +0.00% 49 → 49 ok
LintParallel/rules +0.36% 0.398 +0.37% 2.23% -0.00% 158311 → 158311 ok
LintParallel/lint +0.46% 0.355 +0.48% 1.47% -0.00% 236894 → 236893 ok
LintRW/postgresql -0.38% 0.314 +0.22% 1.12% +0.00% 26500 → 26500 ok
LintRW/mysql -0.71% 0.134 -0.49% 1.64% -0.00% 32413 → 32413 ok
LintRW/mariadb -0.19% 0.738 -0.36% 1.79% +0.00% 29610 → 29610 ok
LintRW/sqlserver -0.05% 0.904 -0.20% 1.51% +0.00% 25622 → 25622 ok
LintRW/oracle +0.33% 0.369 +0.05% 1.63% +0.00% 31614 → 31614 ok
LintRW/sqlite +0.36% 0.659 +0.44% 3.81% +0.00% 29427 → 29427 ok
LintRW/snowflake -0.03% 0.925 -0.52% 2.11% -0.00% 28806 → 28805 ok
LintRW/clickhouse +0.26% 0.841 +0.37% 3.15% +0.00% 32891 → 32891 ok
LintRule/L007 +0.10% 0.925 -0.39% 2.78% +0.00% 1194 → 1194 ok
LintRule/L010 -0.04% 0.758 +0.05% 0.39% +0.00% 0 → 0 ok
ParseError/syntax_mid/parse -0.23% 0.705 +0.56% 1.26% +0.00% 12 → 12 ok
ParseError/syntax_mid/validate +0.09% 0.725 +0.55% 0.89% +0.00% 12 → 12 ok
ParseError/syntax_end/parse +0.88% 0.218 +1.92% 3.82% +0.00% 16 → 16 ok
ParseError/syntax_end/validate +0.73% 0.159 +1.60% 1.30% +0.00% 16 → 16 ok
ParseError/tokenize/parse +0.48% 0.579 -1.01% 1.96% +0.00% 4 → 4 ok
ParseError/tokenize/validate -0.02% 1.000 +0.40% 1.49% +0.00% 21 → 21 ok
ParseError/long_prefix/parse +0.09% 0.925 +0.55% 2.71% -0.00% 219 → 219 ok
ParseError/long_prefix/validate +0.22% 0.883 +0.96% 3.26% +0.00% 219 → 219 ok
geomean +0.01% 0.659 +0.04%

time and A/A' are Hodges-Lehmann shifts over the per-layout medians. spread: the base arm's robust standard deviation across its layouts (1.4826 x MAD, relative), i.e. how much this benchmark moves between builds that differ only in function placement, plus run-to-run noise; a change much smaller than it cannot be resolved in one run.

A/A' control (base revision, disjoint layout seeds): 0 of 52 benchmarks moved more than 3%; 0 would have been flagged by the time rule; suite geomean +0.04%, p 0.640.

Low power

3 benchmark(s) have under 80% estimated power to be flagged on their own for a +5% slowdown (their spread across layouts is too wide for 20 layouts at the Benjamini-Hochberg level alpha/m = 0.000192). A slowdown below the last column usually goes unflagged there; instructions/op, allocs/op and the suite geomean still guard them.

benchmark spread power at +5% slowdown caught 80% of the time
ParseRW/clickhouse/medium 4.08% 49% +6.2%
ParseError/syntax_end/parse 3.82% 59% +5.8%
LintRW/sqlite 3.81% 59% +5.8%

Power is estimated with the normal approximation for the U test (Noether): power = Phi(ln(1+shift)/s * sqrt(K/2) * sqrt(3/pi) - z(1 - alpha/(2m))), with s the spread and K the layouts per arm.

Instructions per operation (cmd/perfdriver)

case instructions/op status
e2e/aggregate -0.05% ok
e2e/complex -0.03% ok
e2e/cte +0.03% ok
e2e/delete -0.15% ok
e2e/insert -0.03% ok
e2e/join -0.02% ok
e2e/select_where +0.06% ok
e2e/simple_select -0.12% ok
e2e/subquery +0.00% ok
e2e/update -0.03% ok
e2e/window +0.06% ok
rw/postgresql/medium -0.07% ok
Secondary metric: default GC, GOMAXPROCS=1 (informational)
benchmark time p
ParseE2E/simple_select +0.84% 0.272
ParseE2E/select_where -0.06% 0.880
ParseE2E/join +0.36% 0.739
ParseE2E/aggregate -0.13% 0.820
ParseE2E/subquery -0.19% 0.762
ParseE2E/cte -0.22% 0.796
ParseE2E/window -1.11% 0.063
ParseE2E/complex +0.14% 0.853
ParseE2E/insert +0.16% 0.821
ParseE2E/update +0.53% 0.384
ParseE2E/delete +0.44% 0.850
ParseRW/postgresql/medium +0.29% 0.739
ParseRW/mysql/medium -0.05% 0.912
ParseRW/mariadb/medium +0.43% 0.393
ParseRW/sqlserver/medium -0.02% 1.000
ParseRW/oracle/medium +0.39% 0.315
ParseRW/sqlite/medium +0.46% 0.579
ParseRW/snowflake/medium -0.73% 0.105
ParseRW/clickhouse/medium -0.11% 0.853
ParseFresh/simple_select +0.43% 0.739
ParseFresh/join -0.20% 0.684
ParseFresh/complex +0.11% 0.912
PoolGetPut/serial +0.47% 0.326
Tokenize/scratch/join -0.03% 1.000
Tokenize/scratch/complex -0.01% 1.000
Tokenize/scratch/insert -0.41% 0.739
Normalize/simple_select -0.95% 0.151
Normalize/join -0.25% 0.579
Normalize/complex -1.05% 0.190
TransformParse/simple_select -1.02% 0.005
TransformParse/join -0.11% 1.000
TransformParse/complex -0.35% 0.623
LintParallel/rules -1.71% 0.315
LintParallel/lint -0.48% 0.796
LintRW/postgresql -0.43% 0.631
LintRW/mysql +0.07% 0.912
LintRW/mariadb +0.05% 0.912
LintRW/sqlserver -0.61% 0.529
LintRW/oracle -0.83% 0.105
LintRW/sqlite -0.59% 0.353
LintRW/snowflake -0.29% 0.631
LintRW/clickhouse -0.55% 0.280
LintRule/L007 +0.55% 0.684
LintRule/L010 -0.05% 0.684
ParseError/syntax_mid/parse -0.29% 0.570
ParseError/syntax_mid/validate +0.00% 1.000
ParseError/syntax_end/parse -1.82% 0.186
ParseError/syntax_end/validate +0.38% 0.364
ParseError/tokenize/parse -0.64% 0.705
ParseError/tokenize/validate +0.15% 0.971
ParseError/long_prefix/parse -0.06% 0.971
ParseError/long_prefix/validate -0.49% 0.684
geomean -0.19%
Environment 
date:        2026-11-03T21:14:24Z
base:        1f56d22152962376607ac36b669093a31600face
head:        working tree of 25acf3e7fef4856d4b02ef03b84b2b4bc2dcd20b
cpu model:   AMD EPYC 7763 64-Core Processor (x86_64)
go:          go version go1.26.9 linux/amd64
kernel:      Linux 7.0.0-1017-azure x86_64
layouts:     20 per arm, -ldflags=-randlayout seeds 417193019202..417193019221 (base, head), 417193019222..417193019241 (A/A')
pinning:     pinned to CPU 3 (SMT siblings 2-3) with taskset
gcoff env:   GOGC=off GOMEMLIMIT=64MiB
clocksource: tsc
load:        7.96, 2.96, 1.07
instr count: cachegrind
perf probe:  Error: No supported events found. The instructions:u event is not supported. 

Architecture:                            x86_64
CPU(s):                                  4
Model name:                              AMD EPYC 7763 64-Core Processor
Thread(s) per core:                      2
Core(s) per socket:                      2
Socket(s):                               1
Hypervisor vendor:                       Microsoft
L1d cache:                               64 KiB (2 instances)
L1i cache:                               64 KiB (2 instances)
L2 cache:                                1 MiB (2 instances)
L3 cache:                                32 MiB (1 instance)
benchstat (primary metric, samples pooled across layouts: informational, the verdict uses the per-layout test) 
goos: linux
goarch: amd64
pkg: github.com/ajitpratap0/GoSQLX/pkg/fingerprint
cpu: AMD EPYC 7763 64-Core Processor                
                        │    base     │                head                │
                        │   sec/op    │   sec/op     vs base               │
Normalize/simple_select   2.355µ ± 1%   2.345µ ± 1%       ~ (p=0.616 n=20)
Normalize/join            7.179µ ± 1%   7.180µ ± 0%       ~ (p=0.846 n=20)
Normalize/complex         19.90µ ± 1%   19.77µ ± 1%       ~ (p=0.298 n=20)
geomean                   6.955µ        6.931µ       -0.35%

                        │     base     │                 head                  │
                        │     B/op     │     B/op      vs base                 │
Normalize/simple_select     896.0 ± 0%     896.0 ± 0%       ~ (p=1.000 n=20) ¹
Normalize/join            2.211Ki ± 0%   2.211Ki ± 0%       ~ (p=1.000 n=20) ¹
Normalize/complex         6.117Ki ± 0%   6.117Ki ± 0%       ~ (p=1.000 n=20) ¹
geomean                   2.279Ki        2.279Ki       +0.00%
¹ all samples are equal

                        │    base    │                head                 │
                        │ allocs/op  │ allocs/op   vs base                 │
Normalize/simple_select   7.000 ± 0%   7.000 ± 0%       ~ (p=1.000 n=20) ¹
Normalize/join            23.00 ± 0%   23.00 ± 0%       ~ (p=1.000 n=20) ¹
Normalize/complex         68.00 ± 0%   68.00 ± 0%       ~ (p=1.000 n=20) ¹
geomean                   22.20        22.20       +0.00%
¹ all samples are equal

pkg: github.com/ajitpratap0/GoSQLX/pkg/gosqlx
                                │    base     │                head                │
                                │   sec/op    │   sec/op     vs base               │
ParseE2E/simple_select            1.635µ ± 1%   1.643µ ± 1%       ~ (p=0.723 n=20)
ParseE2E/select_where             3.446µ ± 4%   3.443µ ± 1%       ~ (p=0.369 n=20)
ParseE2E/join                     4.613µ ± 1%   4.618µ ± 1%       ~ (p=0.673 n=20)
ParseE2E/aggregate                3.484µ ± 2%   3.481µ ± 1%       ~ (p=0.941 n=20)
ParseE2E/subquery                 3.089µ ± 2%   3.109µ ± 1%       ~ (p=0.794 n=20)
ParseE2E/cte                      4.423µ ± 1%   4.428µ ± 1%       ~ (p=0.542 n=20)
ParseE2E/window                   2.921µ ± 1%   2.916µ ± 1%       ~ (p=0.606 n=20)
ParseE2E/complex                  12.22µ ± 2%   12.15µ ± 2%       ~ (p=0.847 n=20)
ParseE2E/insert                   2.179µ ± 2%   2.167µ ± 1%       ~ (p=0.588 n=20)
ParseE2E/update                   1.945µ ± 1%   1.934µ ± 1%       ~ (p=0.417 n=20)
ParseE2E/delete                   1.241µ ± 1%   1.243µ ± 1%       ~ (p=1.000 n=20)
ParseRW/postgresql/medium         168.7µ ± 1%   168.9µ ± 2%       ~ (p=0.495 n=20)
ParseRW/mysql/medium              96.34µ ± 1%   96.77µ ± 1%       ~ (p=0.565 n=20)
ParseRW/mariadb/medium            137.9µ ± 1%   137.8µ ± 1%       ~ (p=0.841 n=20)
ParseRW/sqlserver/medium          276.4µ ± 2%   273.1µ ± 2%       ~ (p=0.355 n=20)
ParseRW/oracle/medium             326.0µ ± 1%   325.2µ ± 1%       ~ (p=0.989 n=20)
ParseRW/sqlite/medium             72.74µ ± 3%   72.66µ ± 2%       ~ (p=0.429 n=20)
ParseRW/snowflake/medium          188.9µ ± 1%   189.5µ ± 1%       ~ (p=0.398 n=20)
ParseRW/clickhouse/medium         87.51µ ± 3%   89.69µ ± 3%       ~ (p=0.565 n=20)
ParseFresh/simple_select          7.095µ ± 1%   7.096µ ± 1%       ~ (p=0.910 n=20)
ParseFresh/join                   11.50µ ± 1%   11.46µ ± 0%       ~ (p=0.211 n=20)
ParseFresh/complex                20.61µ ± 1%   20.65µ ± 1%       ~ (p=0.723 n=20)
PoolGetPut/serial                 67.12n ± 2%   66.95n ± 2%       ~ (p=0.578 n=20)
ParseError/syntax_mid/parse       2.413µ ± 1%   2.412µ ± 1%       ~ (p=0.703 n=20)
ParseError/syntax_mid/validate    2.175µ ± 1%   2.178µ ± 1%       ~ (p=0.723 n=20)
ParseError/syntax_end/parse       2.754µ ± 3%   2.775µ ± 2%       ~ (p=0.218 n=20)
ParseError/syntax_end/validate    2.629µ ± 1%   2.654µ ± 1%       ~ (p=0.159 n=20)
ParseError/tokenize/parse         907.5n ± 2%   916.2n ± 1%       ~ (p=0.579 n=20)
ParseError/tokenize/validate      2.827µ ± 1%   2.820µ ± 2%       ~ (p=0.995 n=20)
ParseError/long_prefix/parse      59.72µ ± 2%   59.83µ ± 2%       ~ (p=0.925 n=20)
ParseError/long_prefix/validate   56.61µ ± 2%   56.89µ ± 2%       ~ (p=0.883 n=20)
geomean                           9.508µ        9.520µ       +0.13%

                                │     base     │                head                 │
                                │     B/s      │     B/s       vs base               │
ParseE2E/simple_select            15.17Mi ± 1%   15.09Mi ± 1%       ~ (p=0.723 n=20)
ParseE2E/select_where             28.50Mi ± 4%   28.53Mi ± 1%       ~ (p=0.365 n=20)
ParseE2E/join                     26.05Mi ± 1%   26.02Mi ± 1%       ~ (p=0.693 n=20)
ParseE2E/aggregate                22.99Mi ± 2%   23.02Mi ± 1%       ~ (p=0.973 n=20)
ParseE2E/subquery                 25.32Mi ± 2%   25.16Mi ± 1%       ~ (p=0.805 n=20)
ParseE2E/cte                      26.74Mi ± 1%   26.71Mi ± 1%       ~ (p=0.551 n=20)
ParseE2E/window                   30.69Mi ± 1%   30.75Mi ± 1%       ~ (p=0.597 n=20)
ParseE2E/complex                  29.50Mi ± 2%   29.66Mi ± 2%       ~ (p=0.867 n=20)
ParseE2E/insert                   33.26Mi ± 2%   33.46Mi ± 1%       ~ (p=0.588 n=20)
ParseE2E/update                   26.97Mi ± 1%   27.13Mi ± 1%       ~ (p=0.387 n=20)
ParseE2E/delete                   39.98Mi ± 1%   39.90Mi ± 1%       ~ (p=0.995 n=20)
ParseRW/postgresql/medium         21.27Mi ± 1%   21.24Mi ± 2%       ~ (p=0.465 n=20)
ParseRW/mysql/medium              28.09Mi ± 1%   27.97Mi ± 1%       ~ (p=0.560 n=20)
ParseRW/mariadb/medium            25.72Mi ± 1%   25.73Mi ± 1%       ~ (p=0.825 n=20)
ParseRW/sqlserver/medium          24.08Mi ± 2%   24.36Mi ± 2%       ~ (p=0.344 n=20)
ParseRW/oracle/medium             25.71Mi ± 1%   25.78Mi ± 1%       ~ (p=0.952 n=20)
ParseRW/sqlite/medium             28.40Mi ± 3%   28.43Mi ± 2%       ~ (p=0.425 n=20)
ParseRW/snowflake/medium          26.91Mi ± 1%   26.83Mi ± 1%       ~ (p=0.351 n=20)
ParseRW/clickhouse/medium         25.75Mi ± 3%   25.13Mi ± 4%       ~ (p=0.542 n=20)
ParseError/syntax_mid/parse       18.97Mi ± 1%   18.98Mi ± 1%       ~ (p=0.683 n=20)
ParseError/syntax_mid/validate    21.05Mi ± 1%   21.02Mi ± 1%       ~ (p=0.723 n=20)
ParseError/syntax_end/parse       13.51Mi ± 3%   13.40Mi ± 2%       ~ (p=0.218 n=20)
ParseError/syntax_end/validate    14.14Mi ± 1%   14.02Mi ± 1%       ~ (p=0.159 n=20)
ParseError/tokenize/parse         48.34Mi ± 1%   47.88Mi ± 1%       ~ (p=0.579 n=20)
ParseError/tokenize/validate      15.52Mi ± 1%   15.56Mi ± 2%       ~ (p=0.995 n=20)
ParseError/long_prefix/parse      38.68Mi ± 2%   38.61Mi ± 2%       ~ (p=0.931 n=20)
ParseError/long_prefix/validate   40.81Mi ± 2%   40.60Mi ± 2%       ~ (p=0.878 n=20)
geomean                           25.52Mi        25.47Mi       -0.16%

                                │      base      │                 head                  │
                                │      B/op      │     B/op      vs base                 │
ParseE2E/simple_select              816.0 ± 0%       816.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/select_where             1.523Ki ± 0%     1.523Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/join                     1.836Ki ± 0%     1.836Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/aggregate                1.812Ki ± 0%     1.812Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/subquery                 1.828Ki ± 0%     1.828Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/cte                      2.336Ki ± 0%     2.336Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/window                   1.281Ki ± 0%     1.281Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/complex                  5.025Ki ± 0%     5.025Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/insert                     824.0 ± 0%       824.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/update                     736.0 ± 0%       736.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/delete                     400.0 ± 0%       400.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/postgresql/medium         65.91Ki ± 0%     65.91Ki ± 0%       ~ (p=0.747 n=20)
ParseRW/mysql/medium              42.37Ki ± 0%     42.37Ki ± 0%       ~ (p=1.000 n=20)
ParseRW/mariadb/medium            60.61Ki ± 0%     60.61Ki ± 0%       ~ (p=0.177 n=20)
ParseRW/sqlserver/medium          83.68Ki ± 0%     83.68Ki ± 0%       ~ (p=0.375 n=20)
ParseRW/oracle/medium             104.8Ki ± 0%     104.8Ki ± 0%       ~ (p=0.353 n=20)
ParseRW/sqlite/medium             38.12Ki ± 0%     38.12Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/snowflake/medium          77.63Ki ± 0%     77.63Ki ± 0%       ~ (p=1.000 n=20)
ParseRW/clickhouse/medium         34.82Ki ± 0%     34.82Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/simple_select            896.0 ± 0%       896.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/join                   1.914Ki ± 0%     1.914Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/complex                5.102Ki ± 0%     5.102Ki ± 0%       ~ (p=1.000 n=20) ¹
PoolGetPut/serial                   0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_mid/parse         680.0 ± 0%       680.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_mid/validate      680.0 ± 0%       680.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_end/parse       1.430Ki ± 0%     1.430Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_end/validate    1.430Ki ± 0%     1.430Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseError/tokenize/parse           272.0 ± 0%       272.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/tokenize/validate      1.758Ki ± 0%     1.758Ki ± 0%       ~ (p=1.000 n=20) ¹
ParseError/long_prefix/parse      22.75Ki ± 0%     22.75Ki ± 0%       ~ (p=0.867 n=20)
ParseError/long_prefix/validate   22.68Ki ± 0%     22.68Ki ± 0%       ~ (p=1.000 n=20) ¹
geomean                                        ²                 -0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                │     base      │                 head                 │
                                │   allocs/op   │  allocs/op   vs base                 │
ParseE2E/simple_select             5.000 ± 0%      5.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/select_where              18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/join                      17.00 ± 0%      17.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/aggregate                 18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/subquery                  14.00 ± 0%      14.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/cte                       21.00 ± 0%      21.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/window                    11.00 ± 0%      11.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/complex                   49.00 ± 0%      49.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/insert                    17.00 ± 0%      17.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/update                    13.00 ± 0%      13.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseE2E/delete                    5.000 ± 0%      5.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/postgresql/medium          646.0 ± 0%      646.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/mysql/medium               368.0 ± 0%      368.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/mariadb/medium             528.0 ± 0%      528.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/sqlserver/medium           927.0 ± 0%      927.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/oracle/medium             1.090k ± 0%     1.090k ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/sqlite/medium              347.0 ± 0%      347.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/snowflake/medium           776.0 ± 0%      776.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseRW/clickhouse/medium          383.0 ± 0%      383.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/simple_select           6.000 ± 0%      6.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/join                    18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseFresh/complex                 50.00 ± 0%      50.00 ± 0%       ~ (p=1.000 n=20) ¹
PoolGetPut/serial                  0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_mid/parse        12.00 ± 0%      12.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_mid/validate     12.00 ± 0%      12.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_end/parse        16.00 ± 0%      16.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/syntax_end/validate     16.00 ± 0%      16.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/tokenize/parse          4.000 ± 0%      4.000 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/tokenize/validate       21.00 ± 0%      21.00 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/long_prefix/parse       219.0 ± 0%      219.0 ± 0%       ~ (p=1.000 n=20) ¹
ParseError/long_prefix/validate    219.0 ± 0%      219.0 ± 0%       ~ (p=1.000 n=20) ¹
geomean                                       ²                +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                          │    base     │                head                │
                          │  sec/stmt   │  sec/stmt    vs base               │
ParseRW/postgresql/medium   14.06µ ± 1%   14.08µ ± 2%       ~ (p=0.499 n=20)
ParseRW/mysql/medium        12.04µ ± 1%   12.10µ ± 1%       ~ (p=0.565 n=20)
ParseRW/mariadb/medium      15.32µ ± 1%   15.31µ ± 1%       ~ (p=0.841 n=20)
ParseRW/sqlserver/medium    13.16µ ± 2%   13.00µ ± 2%       ~ (p=0.344 n=20)
ParseRW/oracle/medium       13.59µ ± 1%   13.55µ ± 1%       ~ (p=0.989 n=20)
ParseRW/sqlite/medium       12.12µ ± 3%   12.11µ ± 2%       ~ (p=0.417 n=20)
ParseRW/snowflake/medium    14.53µ ± 1%   14.58µ ± 1%       ~ (p=0.387 n=20)
ParseRW/clickhouse/medium   12.50µ ± 3%   12.81µ ± 3%       ~ (p=0.565 n=20)
geomean                     13.37µ        13.40µ       +0.21%

pkg: github.com/ajitpratap0/GoSQLX/pkg/linter
                   │     base     │                head                 │
                   │    sec/op    │    sec/op     vs base               │
LintParallel/rules   29.37m ±  4%   29.49m ±  1%       ~ (p=0.398 n=20)
LintParallel/lint    51.43m ±  2%   51.99m ±  1%       ~ (p=0.355 n=20)
LintRW/postgresql    6.555m ±  1%   6.550m ±  1%       ~ (p=0.314 n=20)
LintRW/mysql         6.854m ±  1%   6.801m ±  1%       ~ (p=0.134 n=20)
LintRW/mariadb       6.296m ±  1%   6.276m ±  0%       ~ (p=0.738 n=20)
LintRW/sqlserver     5.833m ±  1%   5.810m ±  1%       ~ (p=0.904 n=20)
LintRW/oracle        6.988m ±  1%   7.009m ±  1%       ~ (p=0.369 n=20)
LintRW/sqlite        5.887m ±  3%   5.955m ±  4%       ~ (p=0.659 n=20)
LintRW/snowflake     6.113m ±  1%   6.099m ±  0%       ~ (p=0.925 n=20)
LintRW/clickhouse    6.734m ±  2%   6.803m ±  4%       ~ (p=0.841 n=20)
LintRule/L007        2.728m ±  2%   2.731m ±  1%       ~ (p=0.925 n=20)
LintRule/L010        698.0µ ± 27%   698.7µ ± 27%       ~ (p=0.758 n=20)
geomean              6.690m         6.703m        +0.20%

                  │     base     │                head                 │
                  │     B/s      │     B/s       vs base               │
LintRW/postgresql   4.320Mi ± 1%   4.320Mi ± 1%       ~ (p=0.378 n=20)
LintRW/mysql        4.773Mi ± 1%   4.811Mi ± 1%       ~ (p=0.136 n=20)
LintRW/mariadb      4.845Mi ± 1%   4.859Mi ± 0%       ~ (p=0.722 n=20)
LintRW/sqlserver    5.312Mi ± 1%   5.331Mi ± 1%       ~ (p=0.773 n=20)
LintRW/oracle       5.684Mi ± 1%   5.665Mi ± 1%       ~ (p=0.364 n=20)
LintRW/sqlite       4.873Mi ± 3%   4.821Mi ± 4%       ~ (p=0.625 n=20)
LintRW/snowflake    5.102Mi ± 1%   5.112Mi ± 0%       ~ (p=0.963 n=20)
LintRW/clickhouse   4.821Mi ± 2%   4.773Mi ± 3%       ~ (p=0.773 n=20)
geomean             4.952Mi        4.947Mi       -0.10%

                   │      base      │                 head                  │
                   │      B/op      │     B/op      vs base                 │
LintParallel/rules   8.111Mi ± 0%     8.111Mi ± 0%       ~ (p=0.533 n=20)
LintParallel/lint    28.08Mi ± 0%     28.08Mi ± 0%       ~ (p=0.250 n=20)
LintRW/postgresql    3.089Mi ± 0%     3.089Mi ± 0%       ~ (p=0.952 n=20)
LintRW/mysql         3.880Mi ± 0%     3.880Mi ± 0%       ~ (p=0.344 n=20)
LintRW/mariadb       3.570Mi ± 0%     3.570Mi ± 0%       ~ (p=0.516 n=20)
LintRW/sqlserver     3.124Mi ± 0%     3.124Mi ± 0%       ~ (p=0.256 n=20)
LintRW/oracle        3.818Mi ± 0%     3.818Mi ± 0%       ~ (p=0.899 n=20)
LintRW/sqlite        3.385Mi ± 0%     3.385Mi ± 0%       ~ (p=0.815 n=20)
LintRW/snowflake     3.384Mi ± 0%     3.384Mi ± 0%       ~ (p=0.077 n=20)
LintRW/clickhouse    3.832Mi ± 0%     3.832Mi ± 0%       ~ (p=0.588 n=20)
LintRule/L007        92.98Ki ± 0%     92.98Ki ± 0%       ~ (p=1.000 n=20) ¹
LintRule/L010          0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                           ²                 -0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                   │     base      │                 head                 │
                   │   allocs/op   │  allocs/op   vs base                 │
LintParallel/rules   158.3k ± 0%     158.3k ± 0%       ~ (p=0.479 n=20)
LintParallel/lint    236.9k ± 0%     236.9k ± 0%       ~ (p=0.202 n=20)
LintRW/postgresql    26.50k ± 0%     26.50k ± 0%       ~ (p=0.256 n=20)
LintRW/mysql         32.41k ± 0%     32.41k ± 0%       ~ (p=1.000 n=20)
LintRW/mariadb       29.61k ± 0%     29.61k ± 0%       ~ (p=0.487 n=20)
LintRW/sqlserver     25.62k ± 0%     25.62k ± 0%       ~ (p=1.000 n=20) ¹
LintRW/oracle        31.61k ± 0%     31.61k ± 0%       ~ (p=0.231 n=20)
LintRW/sqlite        29.43k ± 0%     29.43k ± 0%       ~ (p=1.000 n=20) ¹
LintRW/snowflake     28.81k ± 0%     28.80k ± 0%       ~ (p=0.200 n=20)
LintRW/clickhouse    32.89k ± 0%     32.89k ± 0%       ~ (p=1.000 n=20)
LintRule/L007        1.194k ± 0%     1.194k ± 0%       ~ (p=1.000 n=20) ¹
LintRule/L010         0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                          ²                -0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                  │     base     │                head                 │
                  │   sec/stmt   │   sec/stmt    vs base               │
LintRW/postgresql   21.49µ ±  1%   21.48µ ±  1%       ~ (p=0.317 n=20)
LintRW/mysql        16.55µ ±  1%   16.43µ ±  1%       ~ (p=0.133 n=20)
LintRW/mariadb      16.57µ ±  1%   16.52µ ±  0%       ~ (p=0.738 n=20)
LintRW/sqlserver    21.06µ ±  1%   20.97µ ±  1%       ~ (p=0.889 n=20)
LintRW/oracle       18.99µ ±  1%   19.05µ ±  1%       ~ (p=0.369 n=20)
LintRW/sqlite       16.92µ ±  3%   17.11µ ±  4%       ~ (p=0.659 n=20)
LintRW/snowflake    18.69µ ±  1%   18.65µ ±  0%       ~ (p=0.941 n=20)
LintRW/clickhouse   17.09µ ±  2%   17.27µ ±  4%       ~ (p=0.841 n=20)
LintRule/L007       969.7n ±  2%   970.6n ±  1%       ~ (p=0.910 n=20)
LintRule/L010       248.2n ± 27%   248.4n ± 27%       ~ (p=0.683 n=20)
geomean             8.885µ         8.893µ        +0.09%

pkg: github.com/ajitpratap0/GoSQLX/pkg/sql/tokenizer
                         │    base     │                head                │
                         │   sec/op    │   sec/op     vs base               │
Tokenize/scratch/join      1.486µ ± 2%   1.484µ ± 2%       ~ (p=0.753 n=20)
Tokenize/scratch/complex   3.944µ ± 2%   3.930µ ± 2%       ~ (p=0.968 n=20)
Tokenize/scratch/insert    746.7n ± 4%   741.6n ± 3%       ~ (p=0.499 n=20)
geomean                    1.636µ        1.629µ       -0.41%

                         │     base     │                head                 │
                         │     B/s      │     B/s       vs base               │
Tokenize/scratch/join      80.87Mi ± 2%   81.01Mi ± 2%       ~ (p=0.743 n=20)
Tokenize/scratch/complex   91.40Mi ± 3%   91.75Mi ± 2%       ~ (p=0.968 n=20)
Tokenize/scratch/insert    97.07Mi ± 4%   97.74Mi ± 3%       ~ (p=0.516 n=20)
geomean                    89.53Mi        89.89Mi       +0.41%

                         │     base     │                head                 │
                         │     B/op     │    B/op     vs base                 │
Tokenize/scratch/join      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
Tokenize/scratch/complex   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
Tokenize/scratch/insert    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                               ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                         │     base     │                head                 │
                         │  allocs/op   │ allocs/op   vs base                 │
Tokenize/scratch/join      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
Tokenize/scratch/complex   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
Tokenize/scratch/insert    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=20) ¹
geomean                               ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/ajitpratap0/GoSQLX/pkg/transform
                             │    base     │                head                │
                             │   sec/op    │   sec/op     vs base               │
TransformParse/simple_select   1.539µ ± 3%   1.540µ ± 2%       ~ (p=0.984 n=20)
TransformParse/join            5.007µ ± 1%   4.991µ ± 1%       ~ (p=0.758 n=20)
TransformParse/complex         13.11µ ± 1%   13.09µ ± 1%       ~ (p=0.820 n=20)
geomean                        4.658µ        4.651µ       -0.15%

                             │     base     │                 head                  │
                             │     B/op     │     B/op      vs base                 │
TransformParse/simple_select     816.0 ± 0%     816.0 ± 0%       ~ (p=1.000 n=20) ¹
TransformParse/join            1.836Ki ± 0%   1.836Ki ± 0%       ~ (p=1.000 n=20) ¹
TransformParse/complex         5.023Ki ± 0%   5.023Ki ± 0%       ~ (p=1.000 n=20) ¹
geomean                        1.944Ki        1.944Ki       +0.00%
¹ all samples are equal

                             │    base    │                head                 │
                             │ allocs/op  │ allocs/op   vs base                 │
TransformParse/simple_select   5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=20) ¹
TransformParse/join            17.00 ± 0%   17.00 ± 0%       ~ (p=1.000 n=20) ¹
TransformParse/complex         49.00 ± 0%   49.00 ± 0%       ~ (p=1.000 n=20) ¹
geomean                        16.09        16.09       +0.00%
¹ all samples are equal

Run, raw data and benchstat

@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 33s —— View job

Review of PR #942: one-pass multi-literal prefilter for ScanSQL

I read litmatch.go, prefilter.go, the prefilter call sites in scanner.go, and commentViews/normalizeCommentView in sqltext.go. I did not run tests or benchmarks. I did not read the test files, so I haven't checked the mutation and fuzz claims.

Verdict: I found no blocking issues.

Earlier concern: do escape or E'...' views decode escapes into letters?

No. In normalizeCommentView (sqltext.go:223-) the escapes and eOnly flags only change the lexer state, meaning where string literals end and where comments start. The output is built from sql[start:i] slices plus a single ' ' per comment. Every view is therefore the raw input with some comment spans replaced by a space. No letters are synthesized.

So a required literal can't appear in a view unless it is already in the raw input. Literals contain no whitespace, so a space can't join two fragments into one. Building the presence set from the raw input is sound.

Adding escape-bearing inputs like \x75nion to FuzzLiteralMatcher seeds would still be cheap insurance for future view modes.

What I checked and found correct

  • Aho-Corasick construction. Outputs are merged in BFS order, so suffix matches propagate correctly. The uint16 and 128-literal limits are guarded.
  • Case folding. The ſ→s and K→k special cases keep the prefilter a superset of what (?i) regexes match.
  • CNF derivation. It is conservative: anything it can't prove yields an empty requirement, which only weakens the filter. Alternation, ?/*, +, {n,} and assertions are handled correctly. Empty alternatives drop the clause.
  • Concurrency. The matcher and filters are read-only after sync.Once. Removing the fold buffer and pool removes shared mutable state.

Minor suggestions (non-blocking)

  1. Init-time panics. buildLiteralMatcher panics above 128 literals or 65535 states. Today it uses 62 literals and 309 states. A small test asserting headroom, for example len(lits) <= 100, would turn a future first-use panic into a CI failure.
  2. isASCII per pattern per view (scanner.go:1351). The guard check calls isASCII(sql) once per surviving pattern. Per your own numbers (Large/1MB is -0.01%), the cost is only on large inputs and only for patterns that pass the filter. Caching it per view is a reasonable follow-up, and your "out of scope" note is fair.
  3. wp_p_bypass_test.go was edited outside the brief's owned files. It is disclosed and needed to compile, so I have no objection.

Performance

  • The approach is a single O(n) pass over a dense []uint16 table, replacing about 70 bytes.Contains calls.
  • The probe-branch results in the PR body (ScanSQLRW -54% to -60%, Params -45% to -51%, A/A' within ±0.24%) are the authoritative numbers. The shadow-gate comment for dfbf28c shows no regression in the gated suite, but ScanSQL is not in that suite, as you note.
  • Allocations are unchanged apart from 38→35 on Large/1MB.

Nothing I found should block merge.
· branch perf/w6-08-multi-literal-prefilter

@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 22s —— View job

Review of PR #942: one-pass multi-literal prefilter for ScanSQL

I re-read pkg/sql/security/litmatch.go in full on this push. I did not re-read the rest of the diff or run tests or benchmarks. This review also relies on the two earlier review comments for prefilter.go, scanner.go and sqltext.go.

Verdict: No blocking issues.

litmatch.go

  • Automaton construction. BFS visits states in depth order. A state's output is merged with its failure state's output before its children are expanded, so suffix matches propagate. Missing transitions copy the failure state's row, which gives a full DFA with no failure-following at scan time.
  • Case folding. ſ (U+017F) folds to s and K (U+212A) folds to k. This keeps the prefilter a superset of what a (?i) regex matches. The c >= 0xC5 guard keeps the fast path cheap. A truncated sequence such as a lone 0xC5 falls through to class 0, so it cannot cause a false match.
  • State width. uint16 states are guarded by the 65535 check, which runs after the table is built (litmatch.go:148). The make at line 65 uses len(nodes)*nclass, so a pathological pattern set would allocate first and panic second. That is fine for constant tables.

Non-blocking suggestions

  1. Headroom test. buildLiteralMatcher panics at more than 128 literals or 65535 states. Today it uses 62 literals and 309 states. initPatterns runs lazily, so a future pattern that crosses a limit would panic on first use at runtime. A test such as len(lits) <= 128 that forces init would turn that into a CI failure. Please check whether the existing tests already do this.
  2. Escape-bearing fuzz seeds. The comment-normalized views only replace comment spans with a space and never decode escapes. Adding a few inputs like \x75nion and E'\165nion' to the FuzzLiteralMatcher and differential-test seeds would guard against future view modes.
  3. isASCII per pattern per view. Your out-of-scope note is reasonable. Large/1MB is -0.01% and only patterns that pass the filter reach the guard.
  4. wp_p_bypass_test.go. The edit outside the owned files is disclosed and required to compile.

Performance

  • A single pass over a dense []uint16 table replaces about 70 bytes.Contains calls. Removing the fold buffer and pool also removes shared mutable state.
  • The latest shadow-gate run (head eb41a84) is clean: geomean +0.03% with no gated metric moved. The A/A' control is also clean, with 0 of 52 benchmarks past 3%. ScanSQL is not in that suite, so the probe-branch run in the PR body (ScanSQLRW -54% to -60%, Params -45% to -51%) is the authoritative number.
    · branch perf/w6-08-multi-literal-prefilter

@ajitpratap0
ajitpratap0 merged commit b1d8625 into main Nov 3, 2026
58 of 60 checks passed

This branch was successfully deployed

1 active deployment
Preview — 45da2414 Deployed Nov 3, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation security

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ajitpratap0