Skip to content

fix(parser): end VACUUM, ANALYZE and LISTEN at their grammar - #878

Merged
ajitpratap0 merged 1 commit into
mainfrom
fix/wp-h
Oct 28, 2026
Merged

ajitpratap0 merged 1 commit into
mainfrom
fix/wp-h

Conversation

@ajitpratap0

Copy link
Copy Markdown
Owner

Summary

  • PostgreSQL / Redshift / SQLite VACUUM and ANALYZE, and PostgreSQL LISTEN / UNLISTEN / NOTIFY, now use a small per-dialect grammar (new pkg/sql/parser/utility_maintenance.go) following the existing GRANT / COPY pattern (parseGrammarStatement + utilityTailError). Trailing same-line tokens such as VACUUM t junk or LISTEN c extra are now an E2002 "expected semicolon or end of statement" error. Before this change they were folded into the verbatim text.
  • Generic mode accepts the union of the PostgreSQL, Redshift and SQLite forms and picks the longest complete match without mixing clauses. MySQL / MariaDB / Oracle ANALYZE TABLE ... and MariaDB ANALYZE <statement> keep their existing paths.
  • Adds regression tests for the boundary items that origin/main had already fixed: CREATE TRIGGER / PREPARE folding, a same-line second statement after GRANT / COPY, SplitStatements with MySQL/MariaDB backslash escapes, and PostgreSQL CREATE RULE ... DO [ALSO|INSTEAD] (a; b).

Test Plan

  • task check (fmt, vet, lint, test:race): passed
  • task test:cbinding: passed
  • New tests TestParser_MaintenanceBoundaries, TestParser_MaintenanceForms, TestParser_MaintenanceText, TestParser_MaintenanceIncomplete, TestParser_UtilitySelectBoundaryRegressions, TestSplitStatements_MySQLBackslashBoundaries, TestSplitStatements_PostgreSQLRuleCommandList
  • Parser benchmarks BenchmarkParserStatementTypes and BenchmarkParserMixedWorkload, -benchmem -count=10, before and after runs interleaved: B/op and allocs/op are identical and ns/op shows no regression (for example SELECT_Complex 6089 to 5610 ns/op, MixedWorkload_Sequential 933 to 930 ns/op)

Breaking Changes

Input that origin/main accepted is now rejected when it puts invalid trailing tokens after these maintenance statements. One example is VACUUM t junk in PostgreSQL, SQLite or generic mode. Valid forms still parse, including every such statement in the testdata/realworld corpus.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2

PostgreSQL, Redshift and SQLite VACUUM / ANALYZE and PostgreSQL LISTEN,
UNLISTEN and NOTIFY were kept verbatim, so trailing same-line tokens
(VACUUM t junk) were swallowed into the statement text. They now follow
the GRANT / COPY pattern: a small per-dialect grammar finds the end and
utilityTailError rejects anything after it. Generic mode accepts the
union of the dialect forms; MySQL / MariaDB / Oracle ANALYZE TABLE keeps
its verbatim path.

Adds regression tests for the already-fixed CREATE TRIGGER / PREPARE /
GRANT / COPY boundaries and for the splitter's MySQL backslash escapes and
PostgreSQL CREATE RULE ... DO (a; b) command lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Oct 28, 2026 5:31am UTC

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 39s —— View job


Review of PR #878: end VACUUM, ANALYZE and LISTEN at their grammar

I read the diff and the supporting code by hand. I could not run the test suite here (go test needs approval in this environment), so none of this has been executed.

  • Read the diff
  • Check the dialect gating
  • Check tests and changelog

Potential bug: VACUUM under other dialects (needs verification)

maintenanceEnd in pkg/sql/parser/utility_maintenance.go:22-46 skips every form whose dialect isn't in the mask. For MySQL, MariaDB, Oracle, SQL Server, Snowflake, ClickHouse and BigQuery, no form runs. That leaves best == -1 and firstErr == nil, so it returns (0, nil).

The new case "VACUUM": in utility_stmt.go calls parseGrammarStatement(word, p.vacuumEnd) with no dialect check. The ANALYZE case does have one. parseGrammarStatement (utility_grammar.go:55-68) then treats 0 as a valid end index:

  • utilityTailError(0) runs against the wrong token.
  • renderTokenRange(start, 0) gets an inverted range.
  • p.seek(0) rewinds the parser.

I expect that to produce garbled text, a spurious error, or a loop. On main, VACUUM in these dialects fell through to verbatim handling, so this would be a regression. The same applies to LISTEN, UNLISTEN and NOTIFY: listenEnd doesn't gate on dialect, so those become strict everywhere, which may be fine, but the PR text says PostgreSQL only.

Suggested fix, either of these:

  • In maintenanceEnd, fall back to the old verbatim path when best < 0 && firstErr == nil. Or make the VACUUM case mirror the ANALYZE gate and call parseVerbatimStatement for dialects that have no grammar.
  • Add a test for VACUUM t under mysql, snowflake, clickhouse and bigquery. The new tests only cover PostgreSQL, Redshift, SQLite and generic.

Other observations

  • ANALYZE gate (utility_stmt.go, mask == dialectAll && next != "TABLE" ...): in generic mode ANALYZE LOCAL TABLE and ANALYZE NO_WRITE_TO_BINLOG TABLE keep the old path, but ANALYZE LOCAL t goes to the grammar and fails with "table name". That seems acceptable. A short comment on why those three words are excluded would help.
  • postgresVacuum legacy options: a repeated option such as VACUUM FULL FULL t stops the option loop, and FULL is then read as a table name. That is permissive but harmless.
  • sqliteAnalyze caps names at [schema.]object by truncating nameEnd. The leftover .extra then triggers the trailing-token error, which the test covers. Good.
  • maintenanceNameAt excludes implicitStartWords. A table literally named after a statement starter, such as VACUUM analyze, would be mis-split. That is an edge case, but a quoted identifier is probably the only workaround, so it's worth a note.
  • listenEnd accepts LISTEN a, b. The boundary test expects this to be rejected, which matches PostgreSQL.
  • Performance: the grammar walk is a single pass over the tokens with no allocations beyond the stack cursor. Your benchmarks show no change, which is plausible.
  • Security: nothing concerning. The change only tightens parsing and keeps no new state.
  • Tests and changelog: coverage of the new forms and boundaries is good, and the extra regression tests (SplitStatements, CREATE RULE) are welcome. The changelog entry should list the breaking-change example, VACUUM t junk, because it is a behaviour change.

Summary

The design follows the existing GRANT/COPY pattern and reads cleanly. The main risk is the missing fallback for dialects with no grammar, so please check that before merging. The rest are minor nits.
· branch fix/wp-h

@github-actions github-actions Bot added documentation Improvements or additions to documentation parser labels Oct 28, 2026
@ajitpratap0
ajitpratap0 merged commit 7ae7adf into main Oct 28, 2026
43 checks passed
@ajitpratap0
ajitpratap0 deleted the fix/wp-h branch October 29, 2026 05:32

This branch was successfully deployed

1 active deployment
Preview — 880b191f Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation parser

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant