feat(sql): quoted and qualified table names in FROM, JOIN and DELETE (story 21.2) - #294
Merged
Conversation
…(story 21.2) The FROM/JOIN table-name surface becomes a call-site swap onto story 21.1's exported lexing surface plus a leading-qualifier `rep`, and the qualifier a statement writes is PRESERVED in the AST instead of being discarded. Lexing (#252 part 2) - new `Parser.qualifierPart` / `Parser.tableParts`; `FromParser.table`, `source` and `join` rewritten onto them, `quotedSchemaPrefix` deleted - `identifierRegexStr` / `identifierRegex` deleted: zero callers left in this repo and zero in jdbc / arrow / extensions - `source` now returns a partially-built `StandardJoin`, which types `(unnest | source)` and removes an unchecked erasure pattern in `join` Semantics (#85) - the parser preserves, it does not interpret - `Table.parts` / `StandardJoin.parts` record the ordered `NamePart`s the statement wrote; `Table.name` stays byte-for-byte its previous value and `SingleSearch.sources` is unchanged for every statement that parsed before - `Table.render` emits each part as ONE lexeme (never split on its dots) with qualifier parts always quoted; `StandardJoin` overrides `sql` to use it, so a dotted index name on a JOIN leg cannot be re-quoted into a different index - the render stops deleting the qualifier - in SELECT, DELETE, CTAS and MATERIALIZED VIEW bodies alike Two approved fixes in the same surface - CROSS JOIN without ON now parses and validates: `Join.validate`'s CrossJoin exemption was unreachable dead code behind `StandardJoin.validate`'s unconditional ON requirement. A bare JOIN without ON stays rejected - two tables differing only by qualifier keep both aliases: `From.tableAliases` keys an entry by its qualified reference when - and only when - this FROM uses that bare name for more than one distinct qualified reference. The unconditional form the spec prescribed was measured to break `Identifier.table` and two softclient4es-extensions call sites that look tables up by the bare index name Also fixed in-branch, found while implementing - `Delete.sql` rendered the bare index, so a DELETE's qualifier vanished from the rendering and the AST fixed point broke once `Table` carried `parts` - `FromlessSelect.toSingleSearch` built its `Table` without `parts`, so the rewrite it documents as parser-equivalent was no longer AST-equal to one Behaviour changes worth a release note - renderings now carry the qualifier, canonicalised to the ANSI double quote - `Table` and `StandardJoin` gained a field => binary-incompatible - a JOIN source can no longer be an expression (`JOIN customers::BIGINT c` and `JOIN DISTINCT customers c` parsed before, and silently discarded the cast) Tests: `QuotedTableNameSpec` (39) and `QuotedTableRoundTripSpec` (40) new, two appended to ParserSpec's Issue-#57 block (its four originals untouched and green), one retargeted in QuotedIdentifierSpec. sql 768, core 887, softclient4es-sql-bridge 165, macrosTests 19; `+ sql/test` and `+ core/test` green on 2.12.20 and 2.13.16; `++ 2.12.20 sql/Test/compile` green. Docs: a Qualified and quoted table names section in dql_statements.md and a rewritten quoting section in known_limitations.md; every SQL example in both was parse-probed against the built parser. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…w follow-ups (story 21.2)
Independent-review findings on feature/21.2. The lead ruled: KEEP AD-6' (the alias-map key is
qualified only where the bare name is ambiguous) and fix the desync it left behind.
F1 - the desync AD-6's scaladoc claimed did not exist
`Identifier.table` IS a `From.tableAliases` key, so in the ambiguous branch it is the QUALIFIED
reference - while `TemporalLiterals` built its cross-index join-leg guard out of the BARE
`joinAliases` source names. Measured on
`SELECT o.a, p.b FROM orders o JOIN "prod_eu".orders p ON o.cid = p.id`: main compares
`prod_eu.orders` against `{prod_eu.orders}` and the guard fires; the branch compared it against
`{orders}` and it did not, so a temporal literal was resolved against the wrong index's schema.
New `From.joinSourceKeys` applies the same `aliasKey` to every JOIN leg and `TemporalLiterals`
consumes it. The AD-6' scaladoc no longer claims to be self-sufficient and states the obligation
any new `Identifier.table` consumer inherits.
F4 - `Join.validate`'s `CrossJoin` exemption was STILL dead after AD-7
`Unnest` fixes `joinType = None` and `StandardJoin` returns on its own `on` match before
delegating, so the predicate is unsatisfiable for every input, before and after. AD-7 duplicated
the exemption rather than making the original reachable. Deleted, under the same rule that deleted
`identifierRegex` in this story.
F3d - an UNDECLARED behaviour change, now pinned
`CREATE OR REPLACE WATCHER ... FROM "a".orders o, "b".orders p WHERE o.x = 1 ...` is ACCEPTED
before this story and REJECTED after: #191's multi-index guard tests whether any predicate
identifier resolves to a table, and AD-6' keeps both aliases where the map used to drop one. A
wholly unqualified self-join still defeats it. Source comment corrected, both halves pinned.
F5 - the `joinReferences` comment stated the OPPOSITE of measured behaviour (two same-name JOIN
legs under different qualifiers do NOT collapse; keying them bare is what would collapse them).
F8 - pins for behaviour that already measured correct: the mixed qualified/bare ambiguous FROM,
the three-table mix where one name is ambiguous and another is not, and the F1 interaction.
Documentation
- F2: the spec's "arrow is unaffected by anything this story does" is REFUTED. Federation's
`CatalogTablePattern` matches a backtick qualifier + a BARE table name, so a fully quoted
``FROM `prod_us`.`orders` `` is never stripped, now parses, and both legs silently run against
the default cluster where they used to fail loudly in the parser. Cross-repo, so not fixable
here: warned at the site that publishes the syntax (joins.md) and in known_limitations.md, and
`docs/issues/local-arrow-positional-catalog-loss.md` records the loud-to-silent transition.
- F3a: a JOIN leg with a quoted qualifier reads a DIFFERENT index than on current main
(`prod_us.customers` -> `customers`) - documented as a behaviour change, not a widening.
- F3b: AC-2's "the index never changes" wording was too strong; both shapes that move go from a
wrong index to the intended one, and both are pinned. Test title softened to match.
- F6: "Four things" listed five. F7: the qualified key shares a namespace with real dotted index
names - recorded, deliberately not engineered around.
sql 772, core 887, softclient4es-sql-bridge 165, macrosTests 19; `+ sql/compile`,
`+ core/compile`, `++ 2.12.20 sql/Test/compile` and `scalafmtCheckAll` green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fupelaqu
marked this pull request as ready for review
September 6, 2026 21:03
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #252
Closes #85
Closes #292
Closes #293
Story 21.2 — the FROM/JOIN table-name surface.
FROM(andJOIN, andDELETE FROM) becomes acall-site swap onto story 21.1's exported lexing surface plus a leading-qualifier
rep, and thequalifier a statement writes is preserved in the AST instead of being discarded.
The rule
Quoting is the discriminator, and it is #57's shipped decision generalised — not replaced. A
qualifier is the maximal leading run of parts that are each quoted AND each followed by a dot;
the index name is everything after it. A bare dot never separates, which is what has always
protected
logs-2025.03.FROM elastic.bi_eventselastic.bi_eventsFROM "elastic".bi_eventsbi_eventselasticFROM `elastic`.`bi_events`bi_eventselasticFROM "logs-2025.03"logs-2025.03FROM "elasticsearch"."prod-cluster"."bi_events"bi_eventsThe parser preserves, it does not interpret (#85).
Table.namestays byte-for-byte its previousvalue;
Table.parts/StandardJoin.partsrecord the orderedNameParts. Nothing insql,core,bridgeor any client readsparts— role assignment (schema? catalog? server alias?) belongs tothe resolver, which knows the venue.
Task 0 baseline — re-measured on the CURRENT
origin/main(99d1f438), not the spec'sac54a079The spec's baseline was four merges stale, and two of its "rejected today" rows had become live
defects, both created by 21.1:
99d1f438JOIN "prod_us".customers cprod_us.customerswhile the FROM leg readsordersFROM elastic."bi_events"elastic.(the quoted lexeme was stolen as the ALIAS)elastic.bi_eventsFROM bi_events `bi_events`/FROM bi_events "bi_events"JOIN `customers` c/JOIN "customers" csourceflip)The six review rows reproduced exactly: the
CROSS JOINmessage verbatim,FROM "prod_us".orders o, "prod_eu".orders p⇒ListMap(orders -> p),CREATE TABLE dest (COL INTEGER)parsing,CREATE LOCAL TEMPORARY TABLErejecting withregex '(?i)OR\b' expected but 'L' found,FROM elastic."bi_events"and the ON-less/ON-ful CROSSJOIN pair. Corpus counts re-derived and confirmed (25 backtick + 23 double-quote qualified
statements, zero overlap; 24 DDL probes).
The four Issue-#57 tests are green, byte-unmodified
ParserSpec.scala's// ── Schema-qualified table names (Issue #57) ──block is untouched; this PRappends two tests to it. If one of the four had gone red, the implementation would have joined
the parts and AD-1 would have been violated.
Design decisions taken at dev time
tablePartsuses 21.1's joinedqualifiedName;qualifiedNameitself was nottouched, so 21.1's "one lexing surface" holds with a zero-line diff. Splitting the tail would
render
elastic."bi_events"as"elastic"."bi_events", which re-parses to a different index.qualifierPartkeeps a separate dot terminal rather than one adjacency-strictregex. Measured: the strict form sends
FROM "elastic" .bi_events(accepted today, readsbi_events) downqualifiedName, whose tail matches.bi_eventsadjacently, and it wouldsilently start reading
elastic.bi_events.in its own
FROM. The unconditional form was implemented first and measured to breakIdentifier.tableand twosoftclient4es-extensionscall sites that look tables up by the bareindex name, for statements that work today.
Identifier.tableis an alias-map key, so anything insqlcomparing it against an index name must speak the same key language. New
From.joinSourceKeys;TemporalLiterals' cross-index join-leg guard consumes it.Join.validate'sCrossJoinexemption was unsatisfiable for everyinput before and after AD-7, so it is deleted rather than left as a second, unreachable
statement of the same rule.
Consumer sweep for
tableAliases/aliasesToTable/joinAliases(AC-12)sql/…/package.scala:1266Identifier.updateIdentifier.tablejoinSourceKeyssql/…/query/TemporalLiterals.scala:304,349Identifier.tableagainst join sourcessql/…/query/OrderBy.scala:58sql/…/query/package.scala:119aliasesToTablegraph/JoinDependencyGraph.scala:517-528,graph/FieldAnalyzer.scala:94,97,152,250,296schemasmap keyed by the bare indexplanner/PredicatePushdown.scala:52-65Setof aliases#85 — the consumer note
softclient4es-arrow'sJoinPlanner.extractCatalogList/stripAllCatalogPrefixescan retire forthe FROM/JOIN half on the next core bump and read
Table.parts/StandardJoin.partsinstead,which also removes
nextCatalogFor's positional mis-association. A retiring resolver must treatan UNREGISTERED qualifier as "no qualifier", not as an error — after story 20.3 a BI tool's
qualifier is routinely the cluster name.
CatalogTablePatternneeds a backtick qualifier + a bare table name, so a fully quotedFROM `prod_us`.`orders`is never stripped, now parses, and both legs silently run against thedefault cluster where they used to fail loudly. Cross-repo, so not fixable here — warned in
joins.mdandknown_limitations.md, and recorded in the local arrow follow-up record.Release notes
Table.sql/From.sqlrenderings now carry the qualifier the statement had, canonicalised tothe ANSI double quote. Anything pinning generated SQL text must look.
TableandStandardJoingained apartsfield ⇒ binary-incompatible; downstream rebuildrequired. Shared with 21.1's
GenericIdentifier/Aliasnote — one line for the train.CREATE OR REPLACE MATERIALIZED VIEWon an unchanged definition REBUILDS once after the upgrade(the extension string-compares the stored render against a fresh one). Not data loss; an
expensive surprise on first re-deploy. Shared with 21.1 — one train-level line.
CROSS JOINwithoutONnow validates; the alias maps of a table whose bare name is ambiguousin its own FROM are keyed by its qualified name (visible only to callers introspecting
From.tableAliases/aliasesToTable/Identifier.table).JOINleg with a quoted qualifier reads a different index.JOIN "prod_us".customers cread
prod_us.customersbefore and readscustomersnow, matching what theFROMleg of thesame statement has always done. Write the dotted name unquoted to keep the old reading.
JOIN customers::BIGINT candJOIN DISTINCT customers cparsed before — the cast and the DISTINCT reached nothing, while therender kept them — and are loud rejections now, matching the FROM side.
CREATE OR REPLACE WATCHER … FROM "a".orders o, "b".orders p WHERE o.x = 1 …was accepted and isnow rejected. A wholly unqualified self-join still defeats it, as before.
the default cluster where it used to fail loudly. Quote the prefix, leave the table name bare
(
FROM `prod_us`.orders), until the arrow-side retirement lands.Story 21.7 owns the rest
Parser.ident— quoted names forINSERT/UPDATE/CREATE/DROP/ALTER, table andcolumn — is not in this PR and stays rejected, pinned as such.
known_limitations.mdnames theresidual as DML/DDL-only. The
#-prefixed temp-table probe outcome and the temp-table productdecision ride 21.7.
Verification
sql772 ·core887 ·softclient4es-sql-bridge165 ·macrosTests19 ·+ sql/testand+ core/testgreen on 2.12.20 and 2.13.16 ·+ sql/compile/+ core/compile·
++ 2.12.20 sql/Test/compile·scalafmtCheckAll. No emitted-JSON change — the index list isbyte-identical by construction.
build.sbt's0.23.0-SNAPSHOTverified and untouched. Nocore,bridgeores{N}source was modified. Every SQL example added to the docs was parse-probedagainst the built parser.
🤖 Generated with Claude Code