Repository navigation
perf(parser,gosqlx): reserve the parse stack once before tokenizing [W1-04] - #940
ajitpratap0 wants to merge 5 commits into
Conversation
…W1-04] A parse on a fresh goroutine grew its stack 2 -> 4 -> 8 (-> 16) KiB inside the parse. internal/stackgrow.Reserve (12 KiB frame, lands at 16 KiB) now runs first in tokscratch, four public-tokenizer pipelines and the parser core, so the stack is copied once while shallow. No output, allocs/op or B/op change; no public API. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @ajitpratap0's task in 24s —— View job Review of PR #940: reserve the parse stack once before tokenizing [W1-04]I read the diff of the single commit. I did not build or run the tests in this job. The test results below come from the PR description. Verdict: the approach is sound and the change is small and well contained. I have no blocking issues, only a few non-blocking points. What I checked
Non-blocking suggestions
SecurityNo new input handling, no unsafe, and no new allocations of attacker-controlled size. The reserved size is a compile-time constant. No concerns. Test coverageCoverage looks thorough. It includes the entry-point enumeration test ( Overall this looks good to merge once the amd64 perf gate confirms the gain. |
Perf gate: PASS (shadow mode: report only, this check never fails)Base Runner CPU:
Rules: each arm runs 20 randomized code layouts ( Gated suite (GOGC=off, GOMAXPROCS=1)
A/A' control (base revision, disjoint layout seeds): 0 of 55 benchmarks moved more than 3%; 0 would have been flagged by the time rule; suite geomean -0.12%, p 0.383. Low powerEvery benchmark has an estimated 80% or better chance of being flagged on its own for a +5% slowdown. Instructions per operation (cmd/perfdriver)
Secondary metric: default GC, GOMAXPROCS=1 (informational)
Environmentbenchstat (primary metric, samples pooled across layouts: informational, the verdict uses the per-layout test) |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
…and comment fixes [W1-04] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
|
Claude finished @ajitpratap0's task in 19s —— View job Review of PR #940: reserve the parse stack once before tokenizing [W1-04]I read the diff for Verdict: no blocking issues. The change is small, internal-only, and the amd64 gate confirms it. Evidence
Code
Non-blocking suggestions
SecurityNo new input handling, no Test coverageGood. It includes entry-point enumeration ( Overall this is good to merge once the CHANGELOG numbers and the PR-body/ |
… [W1-04] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
…-04] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
|
Claude finished @ajitpratap0's task in 21s —— View job Review of PR #940: reserve the parse stack once before tokenizing [W1-04]I read Verdict: no blocking issues. The change is small and internal-only. The latest amd64 gate run confirms the gain. Evidence
Code
Items to resolve before merge
SecurityNo new input handling, no Test coverageGood. It covers entry-point enumeration ( |
|
Claude finished @ajitpratap0's task in 28s —— View job Review of PR #940: reserve the parse stack once before tokenizing [W1-04]I read the full diff against Verdict: no blocking issues. The earlier review notes about the CHANGELOG placeholder and the Resolved since the earlier reviews
Code
Non-blocking suggestions
SecurityNo new input handling, no Test coverageGood. It includes entry-point enumeration ( Overall this looks good to merge once CI is green. |
Summary
A parse on a brand-new goroutine grew its stack 2 -> 4 -> 8 (-> 16) KiB inside the parse; for a ~1 us statement the 2-3 copies cost more than the parse.
internal/stackgrow.Reserve()(12 KiB frame, lands a fresh goroutine at 16 KiB, no-op ~1 ns afterwards) now runs first intokscratch.Bytes/String/StringContext(covers every internal tokenize-then-parse pipeline), in four public-tokenizer pipelines (ParseBytesWithTokens,Analyze,LintString, LSPparseWithRecoveryForDialect) and as a backstop in the parser core (parseTokens,parseContextTokens,parseWithRecovery). No output, allocs/op or B/op change; no public API (D18).Memory: a goroutine that parses keeps >= 16 KiB stack until exit/GC shrink (before 4-16 KiB; Windows lands at 32 KiB). Coverage over the 2,999-statement corpus: 99.90% need no further copy.
Also:
BenchmarkParseFreshRW(evidence for the 16 KiB class on real-world SQL) and the CI non-race step now runsAllocBudget|StackBudget|StackReservefor./pkg/gosqlx ./internal/stackgrow.Test Plan
go build ./...,GOOS=js GOARCH=wasm go build ./wasm/: passgo test -count=1 -run 'StackReserve|StackBudget|EntryPoints|NoUnreserved|Reserve' ./internal/stackgrow ./pkg/gosqlx(non-race and-race): pass; real-world coverage 2996/2999go test -count=1 -run 'AllocBudget|StackBudget|StackReserve' ./pkg/gosqlx ./internal/stackgrow: passtask test:cbinding,go test ./pkg/compatibility/..., staticcheck on changed packages: passtask check: format/vet/lint clean; the race run hit the 2m timeout and timing-ratio flakes (TestJoinWithComments_ScalesLinearly,TestTranspileDeepChainLinear, parser/security/tokenizer timeouts) while the M4 was at load 50-114. Re-run: parser, security, tokenizer, transpiler pass;pkg/formatterpasses alone and flaked once more under the full-tree run (B/op ratio 4.31 within bound; only the wall-clock ratio missed). No test, threshold or timeout was changed.Mutation checks (by Codex):
reserveBytes=16<<10fails T2,5<<10fails T4,neveras const fails T1, dropping the Reserve intokscratch.Stringfails T5.Local
task bench:ab BASE=origin/main BENCH='./pkg/gosqlx=^BenchmarkParseFresh$' ROUNDS=8 LAYOUTS=0 BENCHTIME=200ms(arm64 M4, load 50+, indicative only): geomean -64.88%; simple_select -79.8%, aggregate -74.1%, subquery -68.0%; other cells directionally lower but noisy (+-80%). allocs/op and B/op identical in every cell.CI perf-gate (ci(perf): randomize code layout in the A/B perf gate so layout noise cannot read as a regression #934, amd64 EPYC 7763, 20 interleaved rounds x 20 randomized layouts per arm, per-layout Mann-Whitney U, A/A', BH). Authoritative; local M4 numbers are indicative only. The gate suite now includes ParseFresh insert/update/delete (scripts/perf/gate-suite.txt), so the short-DML targets are gated.
Run 1 (base 6399dc0, head merge 58e1cfa): PASS. simple_select -58.73%, join -43.63%, complex -23.54% (all p<0.001, A/A' about -0.2%); instructions/op -60.5%, -42.0%, -24.2%; no gated time regression; allocs/op and B/op unchanged; suite geomean -2.96%; A/A' moved 0 of 52 benchmarks; no target in the Low-power list.
Run 2 (base a72ec74 = current main, head 21e8482, includes insert/update/delete):
Suite geomean -7.69% (p<0.001); A/A' 0 of 55 moved, geomean -0.02%.
Gate noise, reported not hidden: the shadow verdict reads "regression" in both PR-head runs for reasons unrelated to this change. Run (base a72ec74, head 4e90e34) flagged ParseRW/clickhouse/medium +3.22% (A/A' +2.18%, Low-power list, instructions/op -0.91%); it did not reproduce in run 2 (-0.30%, p=0.735) nor in run 1 (PASS). Run 2 flagged LintRW/postgresql allocs/op 34189 -> 34190 (+1 in 34k; run 1 measured 34189.5 -> 34189, so this is pool/GC amortization jitter in a benchmark whose Lint path this change does not touch).
PLAN W1-04 threshold MISSED on one cell: it requires BenchmarkParseFresh/* <= -40%; ParseFresh/complex reaches only -23.3% to -23.5% on amd64 (the remaining cost is parse work, not stack growth). simple_select, join (-43%), insert, update, delete all meet it. The W3-06 trigger ("complex above 1.0x Vitess") still has to be re-checked against this result.
Open decisions (recommended defaults implemented): M1 16 KiB landing; M2(a) implemented:
scripts/perf/gate-suite.txtis widened, so the short-DML cells (insert/update/delete) are in the gated suite.Breaking Changes
None.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2