Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

## Unreleased

- Postgres: wrap each `UNION ALL` branch in parentheses (`DialectClauses.parenthesizeUnionBranches`).
A branch with its own `WITH`, `ORDER BY` or `LIMIT` was a syntax error.
- Postgres: bind `param.float` as `$n::float8`, `param.bool` as `$n::boolean`, and the
`dateTime` kinds as `$n::timestamptz`. A float compared with an int8 column was bound as
int8 and rejected, and a param in a select list was bound as text.
- `ParamStyle.placeholder` receives the param kind as a second argument.
- Docs: exact int8 needs a codec that reads a `bigint` as well as a string; PGlite and
postgres.js send `bigint`, so the documented `custom("int8", Schema.String)` failed there.
- Rename the package to `@maple-dev/effect-orm` and the repository to `MapleTechLabs/effect-orm`.
Imports, error `_tag` prefixes (`@maple-dev/effect-orm/QueryBuilderError`, ...) and the live-test
variables (`EFFECT_ORM_CLICKHOUSE_URL`, `_USER`, `_PASSWORD`) change with it.
Expand Down
1 change: 1 addition & 0 deletions docs/params-and-compilation.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ than `1`/`0`.
| `clauses.format` | Whether `FORMAT` exists; `.format()` fails to compile where it does not |
| `clauses.derivedTableAlias` | Whether a subquery in FROM needs an alias |
| `clauses.groupByAlias` | Whether GROUP BY resolves select aliases; if not, keys are written by position |
| `clauses.parenthesizeUnionBranches` | Whether each `UNION ALL` branch is wrapped in parentheses |
| `paramCodecs` | Per-kind codec overrides for `param.*` |

Params are resolved by rewriting placeholders in the finished SQL, so a dialect's literals must
Expand Down
5 changes: 4 additions & 1 deletion docs/postgres.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,10 @@ literal that still contained it would fail the compile with `InvalidLiteral`.
| `custom(sql, schema, literalSchema?)` | anything | the schema's type | whatever the schema reads |

`int8` and `numeric` decode to `number`, so values beyond 2^53 or a double's precision lose
digits. Declare `PG.custom("int8", Schema.String)` where exact digits matter. A `timestamptz`
digits. Where exact digits matter, declare
`PG.custom("int8", Schema.Union([Schema.BigInt, Schema.BigIntFromString]))`: drivers send int8 as a
`bigint` (PGlite, postgres.js with `types.bigint`) or as a string (node-postgres), and this reads
both as a `bigint`. A `timestamptz`
compared against a `Date`, a `DateTime.Utc` or a string is written as an ISO-8601 instant,
which no session time zone can reinterpret; a zoneless string is read as UTC.

Expand Down
23 changes: 23 additions & 0 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,29 @@ suite additionally checks unmatched joins under both settings and DateTime64 bou
Fixtures pin the session timezone to UTC, matching the timestamp codecs' wire contract.
Both JSON and JSONEachRow response formats are exercised.

## One builder suite, every dialect

`tests/core-cases.ts` holds the builder cases every dialect runs: selection, the shared
operators, params, grouping, joins on tables and subqueries, CTEs, unions, routing, and
`format`. A case builds its query from a `CoreContext`, which supplies the dialect's column
types, aggregate catalog and `compile`, and reads the same fixture rows on every database
(a `WITH` over `values(...)` on ClickHouse and `VALUES` on Postgres, so nothing is written).

- `tests/core.clickhouse.test.ts` runs them live, under both `join_use_nulls` settings.
- `tests/core.postgres.test.ts` runs them on PGlite (Postgres 17) on every `vitest run`.
- `tests/core-sql.test.ts` snapshots the exact SQL and `parameters` per dialect.

Where the databases genuinely disagree, the case says so: `expectedBy` gives a target its
own rows (`/` is integer division on Postgres; ClickHouse fills a missing join row with
defaults unless `join_use_nulls=1`), and `rejects` names a dialect that must refuse to
compile it (`format` on Postgres). A case a dialect cannot run yet goes in `coreSkips`
with a reason. The core manifest in `tests/dialect-coverage.test.ts` requires every query
and union method, expression and condition operator, and param kind to have a core case.

Postgres functions and types have their own manifest: every export of the `./postgres`
entry is run by a case in `tests/dialect-cases.postgres.ts` (`tests/dialect.postgres.test.ts`)
or exempted with a reason.

Tests preserve documented behavior: arithmetic chains follow SQL precedence, not call
order, and `windowFunnel` with `strict_order` rejects intervening events.

Expand Down
5 changes: 3 additions & 2 deletions src/ch/compile.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1086,7 +1086,8 @@ function compileUnionInner<Output extends Record<string, any>, Params extends Re
? "untenanted"
: "cross-tenant"

let sql = subQueries.map((q) => q.sql).join("\nUNION ALL\n")
const parenthesize = currentDialect().clauses.parenthesizeUnionBranches
let sql = subQueries.map((q) => (parenthesize ? `(\n${q.sql}\n)` : q.sql)).join("\nUNION ALL\n")

// Wrap in outer SELECT if ordering/pagination is needed
const hasOuter =
Expand Down Expand Up @@ -1184,7 +1185,7 @@ function renderParams(
const existing = style.reuse ? bound.get(key) : undefined
if (existing !== undefined) return existing
parameters.push(value)
const marker = style.placeholder(parameters.length)
const marker = style.placeholder(parameters.length, kind)
bound.set(key, marker)
return marker
})
Expand Down
2 changes: 1 addition & 1 deletion src/ch/dialect.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ describe("dialect identifiers and clauses", () => {
...CH.clickhouseDialect,
name: "quoted",
quoteIdent: (name) => `"${name.replace(/"/g, '""')}"`,
clauses: { format: false, derivedTableAlias: true, groupByAlias: true },
clauses: { format: false, derivedTableAlias: true, groupByAlias: true, parenthesizeUnionBranches: false },
}
const services = CH.table("db.services", { OrgId: CH.string, Service: CH.string }, { tenantColumn: "OrgId" })

Expand Down
10 changes: 7 additions & 3 deletions src/ch/dialect.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ export type ParamStyle =
| { readonly _tag: "inline" }
| {
readonly _tag: "bind"
/** The placeholder for the value at 1-based `index`. */
readonly placeholder: (index: number) => string
/** The placeholder for the value at 1-based `index`, bound for a param
* of `kind` (`string`, `int`, `float`, `bool`, `dateTime`, ...). */
readonly placeholder: (index: number, kind: string) => string
/**
* Whether one placeholder may stand for every use of the same param.
*
Expand All @@ -49,6 +50,9 @@ export interface DialectClauses {
/** Whether GROUP BY resolves a select alias before an input column of the
* same name. Where it does not, keys are written by select-list position. */
readonly groupByAlias: boolean
/** Whether each `UNION ALL` branch is wrapped in parentheses. Postgres needs
* it for a branch with its own WITH, ORDER BY or LIMIT. */
readonly parenthesizeUnionBranches: boolean
}

/**
Expand Down Expand Up @@ -81,7 +85,7 @@ export const clickhouseDialect: Dialect = {
literal: sqlLiteral,
dateTimeLiteral: (value) => quoteClickHouseString(chDateTimeLiteral(value)),
params: { _tag: "inline" },
clauses: { format: true, derivedTableAlias: false, groupByAlias: true },
clauses: { format: true, derivedTableAlias: false, groupByAlias: true, parenthesizeUnionBranches: false },
}

// The dialect of the enclosing compile, beside the syntax installed for the
Expand Down
22 changes: 20 additions & 2 deletions src/pg/dialect.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,20 @@ const literal = (value: unknown, context: string): string => {
/** `param.dateTimeSeconds`: the same instant, floored to whole seconds. */
const timestampSeconds = timestampLiteral((epochMillis) => new Date(Math.floor(epochMillis / 1000) * 1000).toISOString())

/**
* Casts for the param kinds whose Postgres type is unambiguous. Postgres types
* an untyped `$n` from its context, so a float compared with an int8 column
* binds as int8 and rejects `19.5`, and a param in a select list binds as text.
* `string` and `int` stay uncast: a cast would stop them comparing with an enum
* or int4 column.
*/
const placeholderCasts: Readonly<Record<string, string>> = {
float: "float8",
bool: "boolean",
dateTime: "timestamptz",
dateTimeSeconds: "timestamptz",
}

/**
* Postgres: double-quoted identifiers, standard string literals, and params
* bound to `$1`, `$2`, … and returned in `CompiledQuery.parameters`.
Expand All @@ -76,8 +90,12 @@ export const postgresDialect: Dialect = {
quoteString,
literal,
dateTimeLiteral: (value) => `TIMESTAMPTZ ${quoteString(DateTime.formatIso(value))}`,
params: { _tag: "bind", placeholder: (index) => `$${index}`, reuse: true },
clauses: { format: false, derivedTableAlias: true, groupByAlias: false },
params: {
_tag: "bind",
placeholder: (index, kind) => (Object.hasOwn(placeholderCasts, kind) ? `$${index}::${placeholderCasts[kind]}` : `$${index}`),
reuse: true,
},
clauses: { format: false, derivedTableAlias: true, groupByAlias: false, parenthesizeUnionBranches: true },
paramCodecs: {
bool: Schema.Boolean,
dateTime: PgTimestampLiteral,
Expand Down
3 changes: 2 additions & 1 deletion src/pg/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ export type PgType<Tag extends string, A, I = A> = CHType<Tag, A, I>

/** A number as any Postgres driver sends one: a number, a numeric string, or a
* `bigint`. Decodes to `number`, so an int8 beyond 2^53 loses precision; declare
* `custom("int8", Schema.String)` where that matters. */
* `custom("int8", Schema.Union([Schema.BigInt, Schema.BigIntFromString]))`
* where that matters. */
export const PgNumber: Schema.Codec<number, number | string | bigint> = Schema.Union([
Schema.Finite,
Schema.FiniteFromString,
Expand Down
Loading
Loading