Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed
- **BigQuery PIVOT / UNPIVOT** (#854): the parser gated PIVOT/UNPIVOT on a
hard-coded SQL Server / Oracle / Snowflake list, so BigQuery rejected them
with E2002 although its capabilities advertise `SupportsPivotUnpivot`. The
gate is now `Capabilities().SupportsPivotUnpivot` (true for exactly SQL
Server, Oracle, Snowflake and BigQuery; the default, Generic, PostgreSQL,
MySQL, MariaDB, SQLite, ClickHouse and Redshift dialects still treat `pivot`
/ `unpivot` as identifiers). Supported BigQuery forms: aggregate aliases and
multiple aggregates, `IN` value aliases with or without `AS`,
`UNPIVOT INCLUDE NULLS` / `EXCLUDE NULLS`, row-value aliases with or without
`AS` (string or INT64) and multi-column UNPIVOT. The extensions follow each
dialect's grammar: Oracle also gets aliased/multiple aggregates, the NULLS
modifier and multi-column UNPIVOT; Snowflake gets the NULLS modifier and
bare `IN` value aliases; SQL Server is unchanged.
- PIVOT/UNPIVOT quoted aggregate aliases, UNPIVOT value columns, and
backtick-quoted `IN` names and aliases keep their source delimiters when
rendered. This preserves BigQuery backticks and Oracle alias case.
- Numeric UNPIVOT row aliases after `AS` are limited to BigQuery and Oracle;
Snowflake and SQL Server reject them.

### Added
- `ast.PivotClause.AggregateAlias`, `ast.PivotClause.ExtraAggregates`
(`[]ast.PivotAggregate`), `ast.UnpivotClause.NullHandling`,
`ast.UnpivotClause.ValueColumns` and `ast.UnpivotNullHandling`; rendered by
`SQL()` and `pkg/formatter`.

### Behavior changes
- PIVOT / UNPIVOT `IN` lists now require commas between entries and an alias
after `AS` in every PIVOT dialect. Where bare aliases are supported,
`IN (a b)` is one entry (`a AS b`); otherwise a missing comma is rejected.
`IN ('a' AS)` no longer silently drops the `AS`.

## [1.16.0] - 2026-10-22 — Performance Audit, SQL Highlighting, Dialect-Aware LSP, Claude Code Plugin & ChatGPT App, Correctness Sweep

Covers everything merged since v1.15.0: the October 2026 performance audit
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ Formatting turns one-line SQL into a readable query:
- **[Analysis](https://gosqlx.dev/docs/reference/linting-rules/)** - SQL injection scanner, 30 lint rules (L001-L030), 20 optimizer rules and metadata extraction.
- **[Tooling](https://gosqlx.dev/docs/cli-guide/)** - AST-based formatter, query transforms API, VS Code extension and GitHub Action.
- **[Syntax highlighting](docs/HIGHLIGHTING.md)** - dialect-aware highlighting for terminals (ANSI), HTML/CSS, JSON spans, LSP semantic tokens and WASM; `gosqlx highlight query.sql` or `gosqlx.HighlightANSI(sql)`.
- **[Multi-dialect](https://gosqlx.dev/docs/reference/sql-compatibility/)** - 8 dialects with dedicated grammar support: PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, SQLite, Snowflake, ClickHouse. The `--dialect` flag and `WithDialect` accept 11 names: those 8 (`postgresql`, `mysql`, `mariadb`, `sqlserver`, `oracle`, `sqlite`, `snowflake`, `clickhouse`) plus `generic`, `bigquery` and `redshift`. BigQuery and Redshift have no dialect-specific keyword set (they use the generic keywords) and only capability-gated grammar, such as BigQuery's `QUALIFY`, `UNNEST` in `FROM` and `(struct).field` access.
- **[Multi-dialect](https://gosqlx.dev/docs/reference/sql-compatibility/)** - 8 dialects with dedicated grammar support: PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, SQLite, Snowflake, ClickHouse. The `--dialect` flag and `WithDialect` accept 11 names: those 8 (`postgresql`, `mysql`, `mariadb`, `sqlserver`, `oracle`, `sqlite`, `snowflake`, `clickhouse`) plus `generic`, `bigquery` and `redshift`. BigQuery and Redshift have no dialect-specific keyword set (they use the generic keywords) and only capability-gated grammar, such as BigQuery's `QUALIFY`, `PIVOT`/`UNPIVOT`, `UNNEST` in `FROM` and `(struct).field` access.
- **[AI-ready](https://gosqlx.dev/docs/integrations/mcp/)** - MCP server with 7 tools, public remote endpoint, Streamable HTTP.
- **[Tested](https://gosqlx.dev/docs/reference/sql-compatibility/)** - race-detector clean in CI, ~80-85% SQL-99 (estimated).

Expand Down
12 changes: 10 additions & 2 deletions docs/SQL_COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ MariaDB inherits all MySQL features (SHOW, DESCRIBE, REPLACE INTO, ON DUPLICATE
| Feature | Support Level | GoSQLX Parser | Test Coverage | Notes |
|---------|---------------|---------------|---------------|-------|
| **MERGE** | ✅ Full | ✅ Full | 95% | MERGE statements with WHEN clauses |
| **PIVOT/UNPIVOT** | ✅ Full | ✅ Full | — | Parsed and rendered since v1.14.0 (`FROM s PIVOT (...) AS p`) |
| **PIVOT/UNPIVOT** | ✅ Full | ✅ Full | — | Parsed and rendered since v1.14.0 (`FROM s PIVOT (...) AS p`). Also parsed for Oracle, Snowflake and BigQuery: PIVOT/UNPIVOT is gated on the dialect capability `SupportsPivotUnpivot` |
| **CROSS/OUTER APPLY** | ✅ Full | 🔧 Syntax | 10% | Keywords reserved, no parsing logic |
| **HierarchyID** | ✅ Full | 🔧 Syntax | 20% | Data type recognition |
| **T-SQL functions** | ✅ Full | ⚠️ Partial | 40% | Subset of T-SQL functions |
Expand Down Expand Up @@ -636,7 +636,7 @@ GoSQLX provides a first-class dialect mode engine that threads the SQL dialect t
| **Snowflake** | `"snowflake"` | Snowflake keywords | MATCH_RECOGNIZE, @stage, SAMPLE, QUALIFY, VARIANT paths, time-travel, MINUS, LATERAL FLATTEN, TRY_CAST, IGNORE/RESPECT NULLS, LIKE ANY/ALL, CREATE STAGE/STREAM/TASK/PIPE stubs | ✅ 87/87 QA pass |
| **ClickHouse** | `"clickhouse"` | ClickHouse keywords | PREWHERE, FINAL, GLOBAL IN/NOT IN, nested types, parametric aggregates, bare-bracket arrays, ORDER BY WITH FILL, CODEC, WITH TOTALS, LIMIT BY, ANY/ALL JOIN, SETTINGS, TTL, INSERT FORMAT | ✅ 83/83 QA pass (was 69/83 in v1.14.0) |
| **MariaDB** | `"mariadb"` | MariaDB keywords (superset of MySQL) | All MySQL features + SEQUENCE DDL, FOR SYSTEM_TIME, WITH SYSTEM VERSIONING, PERIOD FOR, CONNECT BY | ✅ |
| **BigQuery** | `"bigquery"` | BigQuery keywords | Composite field access `(expr).field`, table functions | ⚠️ Keywords + basic parsing |
| **BigQuery** | `"bigquery"` | BigQuery keywords | Composite field access `(expr).field`, table functions, QUALIFY, PIVOT/UNPIVOT | ⚠️ Keywords + basic parsing |
| **Redshift** | `"redshift"` | Redshift keywords | Keyword recognition | ⚠️ Keywords + basic parsing |
| **Generic** | `"generic"` | Standard SQL keywords | No dialect-specific syntax; composite field access allowed | ✅ |

Expand Down Expand Up @@ -715,6 +715,14 @@ gosqlx format --dialect mysql query.sql
- Table functions in FROM: `numbers(N)`, `remote(...)`, `file(...)`, `url(...)`, `merge(...)`, `s3(...)`, `values(...)`, etc.
- 30+ ClickHouse-specific keywords: TTL, CODEC, FORMAT, SETTINGS, DISTRIBUTED, etc.

#### BigQuery
- PIVOT / UNPIVOT (unreleased, #854): aggregate aliases and multiple aggregates (`PIVOT (SUM(s) AS total, COUNT(*) AS n FOR q IN ('Q1' AS q1, 'Q2' q2))`), `UNPIVOT INCLUDE NULLS` / `EXCLUDE NULLS`, row-value aliases with or without `AS` (string or INT64), and multi-column UNPIVOT (`UNPIVOT ((a, b) FOR k IN ((c1, c2) AS 'x', (c3, c4) AS 'y'))`)
- PIVOT `IN` values must be literals or names; constant expressions such as `IN (1 + 1)` or `IN (DATE '2024-01-01')` are not parsed
- QUALIFY, `UNNEST` in FROM and `(struct).field` access are capability-gated; BigQuery has no dedicated keyword set
- Quoted PIVOT aggregate aliases and UNPIVOT value-column names retain their source delimiters, including BigQuery backticks and Oracle double quotes

Oracle also accepts aggregate aliases, multiple aggregates, `INCLUDE`/`EXCLUDE NULLS` and multi-column UNPIVOT; Snowflake accepts `INCLUDE`/`EXCLUDE NULLS`. SQL Server keeps the single-aggregate form without aliases or a NULLS modifier. Oracle's tuple row-value aliases (`(c1, c2) AS ('a', 'b')`) are not parsed.

## SQL Standards Compliance Summary

### Overall Compliance
Expand Down
184 changes: 184 additions & 0 deletions pkg/formatter/bigquery_pivot_render_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
// Copyright 2026 GoSQLX Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package formatter_test

import (
"testing"

"github.com/ajitpratap0/GoSQLX/pkg/formatter"
"github.com/ajitpratap0/GoSQLX/pkg/gosqlx"
"github.com/ajitpratap0/GoSQLX/pkg/sql/ast"
"github.com/ajitpratap0/GoSQLX/pkg/sql/keywords"
)

// TestBigQueryPivot_FormatRoundTrip_Issue854 verifies BigQuery PIVOT / UNPIVOT
// (including aggregate aliases, multiple aggregates, INCLUDE/EXCLUDE NULLS and
// the multi-column UNPIVOT form) render through both pkg/formatter and
// ast SQL(), and that the output re-parses to the same rendering.
func TestBigQueryPivot_FormatRoundTrip_Issue854(t *testing.T) {
bq := keywords.DialectBigQuery
cases := []struct {
name string
input string
want string
}{
{
"issue_repro",
"SELECT * FROM sales PIVOT (SUM(amount) FOR quarter IN ('Q1', 'Q2'))",
"SELECT * FROM sales PIVOT (SUM(amount) FOR quarter IN ('Q1', 'Q2'))",
},
{
"aliases_and_multiple_aggregates",
"SELECT * FROM produce PIVOT (SUM(sales) total_sales, COUNT(*) AS num_records FOR quarter IN ('Q1' AS q1, 'Q2' q2)) p",
"SELECT * FROM produce PIVOT (SUM(sales) AS total_sales, COUNT(*) AS num_records FOR quarter IN ('Q1' AS q1, 'Q2' AS q2)) AS p",
},
{
"backtick_alias",
"SELECT * FROM sales PIVOT (SUM(amount) AS `total amount` FOR quarter IN ('Q1' AS `q 1`))",
"SELECT * FROM sales PIVOT (SUM(amount) AS `total amount` FOR quarter IN ('Q1' AS `q 1`))",
},
{"backtick_first_and_extra_alias", "SELECT * FROM sales PIVOT (SUM(amount) AS `日本`, COUNT(*) `a``b` FOR quarter IN ('Q1'))", "SELECT * FROM sales PIVOT (SUM(amount) AS `日本`, COUNT(*) AS `a``b` FOR quarter IN ('Q1'))"},
{"unpivot_backtick_value", "SELECT * FROM sales UNPIVOT (`日本` FOR quarter IN (Q1, Q2))", "SELECT * FROM sales UNPIVOT (`日本` FOR quarter IN (Q1, Q2))"},
{"unpivot_backtick_value_set", "SELECT * FROM sales UNPIVOT ((`日本`, `a``b`) FOR quarter IN ((Q1, Q2)))", "SELECT * FROM sales UNPIVOT ((`日本`, `a``b`) FOR quarter IN ((Q1, Q2)))"},
{"unpivot_backtick_spaced_value", "SELECT * FROM sales UNPIVOT (`total amount` FOR quarter IN (`Q 1`, `Q 2`))", "SELECT * FROM sales UNPIVOT (`total amount` FOR quarter IN (`Q 1`, `Q 2`))"},
{"unpivot_backtick_spaced_value_set", "SELECT * FROM sales UNPIVOT ((`a b`, c) FOR quarter IN ((`Q 1`, `Q 2`)))", "SELECT * FROM sales UNPIVOT ((`a b`, c) FOR quarter IN ((`Q 1`, `Q 2`)))"},
{
"unpivot_include_nulls",
"SELECT * FROM produce UNPIVOT INCLUDE NULLS (sales FOR quarter IN (Q1 'first', Q2 AS 2)) AS u",
"SELECT * FROM produce UNPIVOT INCLUDE NULLS (sales FOR quarter IN (Q1 AS 'first', Q2 AS 2)) AS u",
},
{
"unpivot_multi_column",
"SELECT * FROM produce UNPIVOT EXCLUDE NULLS ((h1, h2) FOR semesters IN ((Q1, Q2) AS 'semester_1', (Q3, Q4) AS 'semester_2'))",
"SELECT * FROM produce UNPIVOT EXCLUDE NULLS ((h1, h2) FOR semesters IN ((Q1, Q2) AS 'semester_1', (Q3, Q4) AS 'semester_2'))",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
tree, err := gosqlx.ParseWithDialect(tc.input, bq)
if err != nil {
t.Fatalf("parse %q: %v", tc.input, err)
}
defer ast.ReleaseAST(tree)

got := formatter.FormatAST(tree, ast.CompactStyle())
if got != tc.want {
t.Errorf("FormatAST(compact) mismatch\n got: %s\nwant: %s", got, tc.want)
}
if len(tree.Statements) != 1 {
t.Fatalf("expected 1 statement, got %d", len(tree.Statements))
}
sel, ok := tree.Statements[0].(*ast.SelectStatement)
if !ok {
t.Fatalf("expected *ast.SelectStatement, got %T", tree.Statements[0])
}
if sqlOut := sel.SQL(); sqlOut != tc.want {
t.Errorf("SQL() mismatch\n got: %s\nwant: %s", sqlOut, tc.want)
}

// Re-parse the formatted output (compact and readable styles)
// and check it renders identically.
for _, style := range []ast.FormatOptions{ast.CompactStyle(), ast.ReadableStyle()} {
out := formatter.FormatAST(tree, style)
tree2, err := gosqlx.ParseWithDialect(out, bq)
if err != nil {
t.Fatalf("re-parse %q: %v", out, err)
}
again := formatter.FormatAST(tree2, ast.CompactStyle())
ast.ReleaseAST(tree2)
if again != tc.want {
t.Errorf("round trip mismatch\n got: %s\nwant: %s", again, tc.want)
}
}
})
}
}

// TestPivotItem_ColumnSetQuoting_Issue854 verifies a hand-built UNPIVOT IN
// column set is written verbatim only when well formed.
func TestPivotItem_ColumnSetQuoting_Issue854(t *testing.T) {
cases := []struct {
in, want string
}{
{"(a, b)", "(a, b)"},
{"(a, `b c`) AS 'k'", "(a, `b c`) AS 'k'"},
{"(a, b) AS 1", "(a, b) AS 1"},
{"(a, (b))", `"(a, (b))"`},
{"(a, b", `"(a, b"`},
{"(a,b)", `"(a,b)"`},
{"(a, b)) --", `"(a, b)) --"`},
{"()", `"()"`},
}
for _, tc := range cases {
if got := ast.QuotePivotItem(tc.in); got != tc.want {
t.Errorf("QuotePivotItem(%q) = %q, want %q", tc.in, got, tc.want)
}
}
}

func TestOraclePivotQuotedAggregateAliasRoundTrip(t *testing.T) {
const want = `SELECT * FROM sales PIVOT (SUM(amount) AS "Total", COUNT(*) AS "Count" FOR quarter IN ('Q1'))`
tree, err := gosqlx.ParseWithDialect(want, keywords.DialectOracle)
if err != nil {
t.Fatal(err)
}
defer ast.ReleaseAST(tree)
out := formatter.FormatAST(tree, ast.CompactStyle())
if out != want {
t.Errorf("formatted = %s, want %s", out, want)
}
reparsed, err := gosqlx.ParseWithDialect(out, keywords.DialectOracle)
if err != nil {
t.Fatalf("reparse: %v", err)
}
defer ast.ReleaseAST(reparsed)
if got := formatter.FormatAST(reparsed, ast.CompactStyle()); got != want {
t.Errorf("reparsed = %s, want %s", got, want)
}
if got := tree.SQL(); got != want {
t.Errorf("SQL() = %s, want %s", got, want)
}
}

func TestPivotQuotedNameFallback(t *testing.T) {
makeSelect := func(ref ast.TableReference) *ast.SelectStatement {
return &ast.SelectStatement{Columns: []ast.Expression{&ast.Identifier{Name: "*"}}, From: []ast.TableReference{ref}}
}
cases := []struct {
name string
ref ast.TableReference
want string
}{
{"first_alias", ast.TableReference{Name: "t", Pivot: &ast.PivotClause{AggregateFunction: &ast.Identifier{Name: "x"}, AggregateAlias: "a); DROP", PivotColumn: "q", InValues: []string{"a"}}}, `SELECT * FROM t PIVOT (x AS "a); DROP" FOR q IN (a))`},
{"string_alias", ast.TableReference{Name: "t", Pivot: &ast.PivotClause{AggregateFunction: &ast.Identifier{Name: "x"}, AggregateAlias: "'total'", PivotColumn: "q", InValues: []string{"a"}}}, `SELECT * FROM t PIVOT (x AS "'total'" FOR q IN (a))`},
{"extra_alias", ast.TableReference{Name: "t", Pivot: &ast.PivotClause{AggregateFunction: &ast.Identifier{Name: "x"}, ExtraAggregates: []ast.PivotAggregate{{Function: &ast.Identifier{Name: "y"}, Alias: "select"}}, PivotColumn: "q", InValues: []string{"a"}}}, `SELECT * FROM t PIVOT (x, y AS "select" FOR q IN (a))`},
{"numeric_extra_alias", ast.TableReference{Name: "t", Pivot: &ast.PivotClause{AggregateFunction: &ast.Identifier{Name: "x"}, ExtraAggregates: []ast.PivotAggregate{{Function: &ast.Identifier{Name: "y"}, Alias: "123"}}, PivotColumn: "q", InValues: []string{"a"}}}, `SELECT * FROM t PIVOT (x, y AS "123" FOR q IN (a))`},
{"single_value", ast.TableReference{Name: "t", Unpivot: &ast.UnpivotClause{ValueColumn: "a); DROP", NameColumn: "q", InColumns: []string{"a"}}}, `SELECT * FROM t UNPIVOT ("a); DROP" FOR q IN (a))`},
{"string_value", ast.TableReference{Name: "t", Unpivot: &ast.UnpivotClause{ValueColumn: "'total'", NameColumn: "q", InColumns: []string{"a"}}}, `SELECT * FROM t UNPIVOT ("'total'" FOR q IN (a))`},
{"multiple_values", ast.TableReference{Name: "t", Unpivot: &ast.UnpivotClause{ValueColumns: []string{"select", "a); DROP"}, NameColumn: "q", InColumns: []string{"(a, b)"}}}, `SELECT * FROM t UNPIVOT (("select", "a); DROP") FOR q IN ((a, b)))`},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
sel := makeSelect(tc.ref)
if got := sel.SQL(); got != tc.want {
t.Errorf("SQL() = %s, want %s", got, tc.want)
}
tree := &ast.AST{Statements: []ast.Statement{sel}}
if got := formatter.FormatAST(tree, ast.CompactStyle()); got != tc.want {
t.Errorf("formatter = %s, want %s", got, tc.want)
}
})
}
}
36 changes: 35 additions & 1 deletion pkg/formatter/render.go
Original file line number Diff line number Diff line change
Expand Up @@ -1432,6 +1432,12 @@ func tableRefSQL(t *ast.TableReference, f *nodeFormatter) string {
sb.WriteString(f.kw("PIVOT"))
sb.WriteString(" (")
sb.WriteString(exprSQL(t.Pivot.AggregateFunction))
writePivotAggregateAlias(&sb, f, t.Pivot.AggregateAlias)
for _, a := range t.Pivot.ExtraAggregates {
sb.WriteString(", ")
sb.WriteString(exprSQL(a.Function))
writePivotAggregateAlias(&sb, f, a.Alias)
}
sb.WriteString(" ")
sb.WriteString(f.kw("FOR"))
sb.WriteString(" ")
Expand All @@ -1445,8 +1451,23 @@ func tableRefSQL(t *ast.TableReference, f *nodeFormatter) string {
if t.Unpivot != nil {
sb.WriteString(" ")
sb.WriteString(f.kw("UNPIVOT"))
if nh := ast.UnpivotNullHandling(t.Unpivot.NullHandling); nh != "" {
sb.WriteString(" ")
sb.WriteString(f.kw(nh))
}
sb.WriteString(" (")
sb.WriteString(ast.QuotePivotColumn(t.Unpivot.ValueColumn))
if len(t.Unpivot.ValueColumns) > 0 {
sb.WriteString("(")
for i, c := range t.Unpivot.ValueColumns {
if i > 0 {
sb.WriteString(", ")
}
sb.WriteString(ast.QuoteUnpivotValueColumn(c))
}
sb.WriteString(")")
} else {
sb.WriteString(ast.QuoteUnpivotValueColumn(t.Unpivot.ValueColumn))
}
sb.WriteString(" ")
sb.WriteString(f.kw("FOR"))
sb.WriteString(" ")
Expand All @@ -1468,6 +1489,19 @@ func tableRefSQL(t *ast.TableReference, f *nodeFormatter) string {
return sb.String()
}

// writePivotAggregateAlias writes " AS alias" for a PIVOT aggregate alias,
// preserving source delimiters or quoting it like a table alias; an empty
// alias writes nothing.
func writePivotAggregateAlias(sb *strings.Builder, f *nodeFormatter, alias string) {
if alias == "" {
return
}
sb.WriteString(" ")
sb.WriteString(f.kw("AS"))
sb.WriteString(" ")
sb.WriteString(ast.QuotePivotAggregateAlias(alias))
}

// writePivotItems writes PIVOT IN values or UNPIVOT IN columns separated by
// ", ", each in its stored source form when well formed and double-quoted
// otherwise (see ast.QuotePivotItem).
Expand Down
Loading
Loading