Route params and literals through a Dialect

design/dialects.md

@@ -0,0 +1,69 @@
+# Dialects: one builder, several databases
+
+Status: steps 1 and 2 landed (params and literals go through a `Dialect`). Steps 3 to 6 are open.
+
+## Goal
+
+Keep one query builder, one tenant-scope analysis, and one row-decoding model, and let the
+database-specific parts (SQL syntax, literals, param binding, column wire formats, the
+function catalog) vary per dialect. ClickHouse output stays byte-identical at every step; the
+exact-SQL tests in `src/ch/compile.test.ts` are the guard.
+
+## What we took from Kysely and Drizzle
+
+Both were read from source (Kysely 0.28.17, Drizzle 1.0.0-rc.5).
+
+| Idea | Where it comes from | How it lands here |
+| --- | --- | --- |
+| Builders produce a config object; a dialect turns it into SQL | Drizzle `PgDialect.buildSelectQuery(config)` | `CHQueryState` already is that config. `compile.ts` becomes the ClickHouse dialect's `buildSelect` |
+| SQL as a chunk tree rendered at the end with `escapeName` / `escapeParam` / `escapeString` | Drizzle `SQL` chunks + `BuildQueryConfig` | `SqlFragment` gets a real `Param` chunk; `Str`/`Lazy` stop rendering ClickHouse text early |
+| Inline vs bound params as a switch | Drizzle `inlineParams` | `ParamStyle`: `inline` (ClickHouse today) or `bind` |
+| A small override surface per dialect | Kysely `DefaultQueryCompiler` hooks (MySQL overrides ~10 methods) | The `Dialect` interface grows hook by hook, never a copy of the compiler |
+| Capability flags instead of dialect checks | Kysely `DialectAdapter` (`supportsReturning`, ...) | Flags such as `supportsFilterClause`, `aliasInWhere`, `limitBy` |
+| A compiled query that keeps its structure | Kysely `CompiledQuery.query` | `CompiledQuery` already carries tenant scope and row schema; `parameters` added in step 1 |
+| Rewrites as passes over the tree | Kysely plugins (`transformQuery` / `transformResult`) | Tenant-scope proof and empty-`IN` handling as passes over the state |
+| Logical type separate from how a driver sends it | Drizzle v1 codecs, `refineGenericPgCodecs` per driver | `CHType` splits into a type and a transport codec (step 3) |
+| Shared operators, per-dialect function catalogs | Drizzle `sql/expressions` vs `pg-core` / `mysql-core` | `eq`, `and`, `in_` in core; `countIf`, `percentileCont` per dialect |
+
+What we deliberately do not copy:
+
+- Kysely decodes nothing at runtime; its row types are a promise. Our schema-backed
+  `decodeRows` stays.
+- Drizzle copies the whole query builder per dialect package. We share the builder and vary
+  only types and functions.
+- Tenant scoping stays a proof, not a plugin that injects `OrgId = ...`. Injection would hide a
+  missing tenant condition instead of reporting it.
+
+## Steps
+
+1. **Params through a dialect (done).** `renderParams` resolves placeholders once, at the top
+   of the statement, so a binding dialect can number them across unions and subqueries.
+   `CompiledQuery.parameters` holds the bound values. Tenant bounds still render as ClickHouse
+   literals, since they are compared as text and never sent.
+2. **Literals behind the dialect (done).** `Dialect.quoteString` and `Dialect.literal` write
+   every `Str` fragment, column literal, and inline param. `compile` installs the dialect for
+   the length of the compile (`withDialect`, the same save/restore pattern as
+   `withSubqueryCompiler`), because literals are written inside the query callbacks, which
+   have no dialect argument. The `__PARAM_` safety is now enforced rather than assumed: a
+   literal that contains the marker fails with `InvalidLiteral`, so a dialect with weaker
+   escaping cannot turn a value into a placeholder.
+3. **Identifier quoting.** Postgres folds unquoted names to lower case, so `OrgId` must be
+   written `"OrgId"`. Column refs, aliases, qualified names, `groupBy` keys and `orderBy`
+   specs are raw text today (`raw(name)` in `expr.ts`, `orderByClause` in `compile.ts`), so
+   this is its own step: column refs become `Ident` fragments that carry their qualifier.
+4. **Split `CHType`.** A logical type (`sql` name, TS type) plus a codec for the transport. The
+   UInt64-as-string rule belongs to ClickHouse's `FORMAT JSON` over HTTP, not to ClickHouse;
+   the native client sends something else, and Postgres drivers send int8 as a string and
+   timestamptz as a `Date`.
+5. **Move `compile.ts` into `ClickHouseDialect.buildSelect(state)`.** `compileQuery` and the
+   terminal clauses (`FORMAT`, `SETTINGS`, `LIMIT BY`) become ClickHouse-only.
+6. **Postgres dialect.** `pg.T` column types, `$n` binding, `"ident"` quoting, a function
+   catalog covering the common cases (`FILTER (WHERE ...)`, `date_bin`, `percentile_cont`,
+   jsonb access), and capability flags for what ClickHouse allows and Postgres does not
+   (select aliases in `WHERE`/`HAVING`, default values instead of `NULL` in outer joins).
+
+## Open questions
+
+- ClickHouse also supports server-side binding (`{name:Type}` with `query_params`). Adding it
+  needs the param kind to name a ClickHouse type, which `param.of` already has.
+- Whether the package splits (`core`, `clickhouse`, `postgres` entry points) at step 4 or 5.

docs/params-and-compilation.md

@@ -147,6 +147,7 @@ CH.compileUnsafe(query, params, options?)  // CompiledQuery, throws
 | `options.rowSchema`   | Effect `Schema` used by `decodeRows` / `decodeFirstRow`              |
 | `options.skipFormat`  | Omit a trailing `FORMAT` clause (used internally for subqueries)     |
 | `options.deferParams` | Leave placeholders unresolved, for SQL spliced into an outer compile |
+| `options.dialect`     | How params reach the server; `clickhouseDialect` when omitted        |
 
 `compileCH` is the internal name; the package exports it as `compile`. Unions use
 `compileUnion(union, params)`.
@@ -156,6 +157,7 @@ CH.compileUnsafe(query, params, options?)  // CompiledQuery, throws
 ```ts
 interface CompiledQuery<Output> {
 	readonly sql: string
+	readonly parameters: ReadonlyArray<unknown>
 	readonly tenantScope: "single-tenant" | "cross-tenant" | "untenanted"
 	readonly rowSchemaSource: "declared" | "derived" | "none"
 	readonly rowSchema: CompiledQueryRowSchema<Output> | undefined
@@ -172,6 +174,7 @@ interface CompiledQuery<Output> {
 | Field                           | Purpose                                                                                  |
 | ------------------------------- | ---------------------------------------------------------------------------------------- |
 | `sql`                           | The statement to execute. The builder never runs it.                                     |
+| `parameters`                    | Values a binding dialect sends beside `sql`, in placeholder order; empty by default      |
 | `tenantScope`                   | Whether the query pins a single tenant — see [Tenant scoping](./tenant-scoping.md)       |
 | `rowSchemaSource`               | Where the row schema came from, so a caller can tell real validation from a pass-through |
 | `rowSchema`                     | The codec itself, for a caller that needs a `Schema` rather than a call                  |
@@ -184,6 +187,40 @@ interface CompiledQuery<Output> {
 
 Use `decodeRows` to validate wire values against the row schema.
 
+## Dialects
+
+A `Dialect` decides how literals are written and how resolved params reach the server. Its
+`quoteString` and `literal` write every string fragment, every value compared against a column,
+and every inline param, for the length of the compile. The default, `clickhouseDialect`,
+writes each value into the SQL as a ClickHouse literal and leaves
+`parameters` empty. A dialect whose `params` style is `bind` leaves a placeholder instead and
+returns the encoded values in `parameters`, numbered once across the whole statement, unions
+and subqueries included:
+
+```ts
+const numbered: CH.Dialect = {
+	...CH.clickhouseDialect,
+	name: "numbered",
+	params: { _tag: "bind", placeholder: (index) => `$${index}`, reuse: true },
+}
+
+const compiled = CH.compileUnsafe(query, { orgId: "org_1" }, { dialect: numbered })
+// compiled.sql:        ... WHERE OrgId = $1
+// compiled.parameters: ["org_1"]
+```
+
+`reuse: true` lets one numbered placeholder stand for every use of a param; set it to `false`
+for positional `?` placeholders, which bind a value each time they appear. Either way a bound
+value is the column codec's wire form, the same value an inline literal is written from, and a
+missing or ill-typed param still fails the compile.
+
+Params are resolved by rewriting placeholders in the finished SQL, so a dialect's literals must
+never spell the param marker `__PARAM_`. ClickHouse writes it as `\x5F_PARAM_`; a literal that
+does contain the marker fails the compile with `InvalidLiteral` rather than being rewritten.
+
+Identifiers, clauses, and functions are still ClickHouse SQL; a dialect changes literals and
+param binding today.
+
 ## Handwritten SQL
 
 When you need SQL the builder cannot express, `rawCompiledQuery` wraps a string in the same

docs/reference.md

@@ -88,6 +88,10 @@ Note `/sql` exports a `compile` (fragment → string) distinct from the root `co
 | `compileUnion`       | `(union, params, options?) => Effect<CompiledQuery<Output>, QueryBuilderError>`      |
 | `compileUnionUnsafe` | The same, throwing instead                                                           |
 | `rawCompiledQuery`   | `({ sql, tenantScope, reason, justification, rowSchema?, route? }) => CompiledQuery` |
+| `clickhouseDialect`  | The default `Dialect`: params written into the SQL as ClickHouse literals            |
+
+`Dialect` and `ParamStyle` describe how params reach the server; pass one as
+`options.dialect`. See [Params and compilation](./params-and-compilation.md#dialects).
 
 ### Params
 
@@ -269,7 +273,7 @@ Types: `WindowSpec`, `CompiledWindowSpec`, `WindowFrameBound`, `WindowRowsFrame`
 **Everything else** — `Table`, `TableOptions`, `Expr`, `ColumnRef`, `Condition`, `Comparable`
 (what a value of a type may be compared against), `MapValueOf`, `Subquery`, `ParamMarker`,
 `ParamKind`, `CHQuery`, `CHUnionQuery`, `ColumnAccessor`, `JoinedColumnAccessor`,
-`JoinOnCallback`, `CompiledQuery`, `CompiledQueryInput`, `CompiledQueryRowSchema`, `RowSchemaMismatch`, `TenantScope`, `FnResult`,
+`JoinOnCallback`, `CompiledQuery`, `CompiledQueryInput`, `CompiledQueryRowSchema`, `RowSchemaMismatch`, `TenantScope`, `Dialect`, `ParamStyle`, `FnResult`,
 `WindowFunnelMode`, `WindowSpec`, `WindowRowsFrame`, `WindowFrameBound`,
 `WindowOrderDirection`, `CompiledWindowSpec`.