Fix insert review findings and record the gap review

CHANGELOG.md

@@ -2,6 +2,17 @@
 
 ## Unreleased
 
+- Add `update(table).set(...).where(...)` and `deleteFrom(table).where(...)` (see
+  `docs/updates-and-deletes.md`), with `returning` on Postgres and `settings` on ClickHouse,
+  where they compile to an `ALTER TABLE ... UPDATE` mutation and a lightweight `DELETE`. A write
+  with no `where()` is refused unless `allRows()` says so, and one whose conditions all came out
+  `undefined` fails. Tenant scope is derived from the WHERE. `CompiledQuery.kind` gains `update`
+  and `delete`; `DialectClauses.insertSettings` is renamed `writeSettings` (unreleased), and
+  `DialectClauses.alterTableUpdate` is added.
+- `returning()` with no arguments returns every column, as in Drizzle. `insertInto(table)` now
+  returns `CHInsertStart`, which offers only `values` and `select`, so an insert without rows
+  no longer type-checks. Add `TableOptions.computed` for generated columns. `INSERT ... SELECT`
+  accepts a plain primitive into a branded column, as comparisons do.
 - Add `insertInto(table).values(rows)` (see `docs/inserts.md`): INSERT ... VALUES from the same
   table definitions, for ClickHouse and Postgres. The row type requires every column that is not
   nullable and has no default; values are encoded through the column codecs, written as literals
@@ -20,7 +31,7 @@
   against the table at the type level. Tenant scope follows the read and where the written
   tenant comes from.
 - Add `settings(record)` to an insert (ClickHouse): `INSERT ... SETTINGS name = value`. Add
-  `DialectClauses.insertSettings`, optional.
+  `DialectClauses.writeSettings`, optional.
 - Add `CompiledQuery.kind` (`"select"` or `"insert"`); `rawCompiledQuery` takes it as an option.
 - Add `ParamStyle.maxParameters`; Postgres sets 65535, and a statement over it fails to compile.
 - Add `@maple-dev/effect-orm/database`, opt-in (see `docs/database.md`): a `Database` over the

README.md

@@ -150,6 +150,7 @@ Full guides live in [`docs/`](./docs/README.md):
 | [Joins and subqueries](./docs/joins-and-subqueries.md)     | The join family, `fromQuery`, correlated subqueries             |
 | [Unions and CTEs](./docs/unions-and-ctes.md)               | `unionAll`, `fromUnion`, `withCTE`                              |
 | [Inserting rows](./docs/inserts.md)                        | `insertInto`, the insert row type, `DEFAULT`, binding            |
+| [Updating and deleting](./docs/updates-and-deletes.md)     | `update`, `deleteFrom`, `allRows`, ClickHouse mutations         |
 | [Params and compilation](./docs/params-and-compilation.md) | `param.*`, how values reach the SQL, `CompiledQuery`            |
 | [Decoding results](./docs/decoding-results.md)             | `rowSchema`, `decodeRows`, decode errors                        |
 | [Running a query](./docs/running-queries.md)               | Executing the SQL with a real client, wire settings, `SETTINGS` |

design/gap-review.md

@@ -0,0 +1,86 @@
+# Gap review: effect-orm against Drizzle and Kysely
+
+Status: review of `main` after the INSERT builder landed (PRs #9 to #12). Four reviews, one per
+area, each read the installed sources (Drizzle ORM 1.0.0-rc.5, Kysely 0.28.17) and checked every
+claim against this repository. Maple call-site counts come from `design/transactions.md` §5.
+
+Priority: **P0** blocks Maple replacing Drizzle on Postgres, or is table-stakes for a query
+builder; **P1** commonly used; **P2** niche.
+
+## Found in the INSERT builder (fixed alongside this note)
+
+1. `returning()` with no arguments type-checked and then failed to compile. It now returns every
+   column, as Drizzle's bare `.returning()` does.
+2. `insertInto(table)` could be compiled and run before it had rows. It now returns
+   `CHInsertStart`, with only `values` and `select`.
+3. `INSERT ... SELECT` rejected a plain primitive into a branded column, which `values` and every
+   comparison accept. It now uses the comparison rule.
+4. A Postgres `GENERATED ALWAYS` column could not be marked on `table()`. `TableOptions.computed`
+   does that.
+5. jsonb and array values in `onConflictDoUpdate`'s SET were untested. They now have a PGlite
+   round trip.
+
+## P0
+
+| Gap | Maple | Effort |
+| --- | --- | --- |
+| ~~UPDATE builder: SET values and expressions, WHERE, RETURNING~~ (built) | ~120 | M |
+| ~~DELETE builder: WHERE, RETURNING~~ (built) | ~79 | S |
+| A typed, value-binding `sql` template usable inside expressions; `sql.join` / `raw` / `empty` on `Db.sql` | ~163 | M |
+| Postgres column types: `timestamptz` as `Date`, `timestamp`, `date`, `interval`, `varchar(n)`, serial / identity | 226 timestamp columns | S |
+| DISTINCT (and DISTINCT ON) | ~10 | S |
+| `FOR UPDATE` / `FOR SHARE` / `SKIP LOCKED` / `NOWAIT` | 7 | S |
+| jsonb and array operators (`@>`, `->`, `?`, `&&`, `ANY`) | ~12 | M |
+| `isNull` / `isNotNull` / `between`; variadic `and` / `or` that skip `undefined` | everywhere | S |
+| Constraint error helpers (unique, foreign key, not null); keep ClickHouse's numeric error codes, which `sqlStateOf` drops today | all upserts | S |
+| Tenant-scope enforcement in `Database`, opt in, with an explicit cross-tenant entry point | safety | S |
+| Postgres `defineTable` (indexes, unique, FKs), Postgres migrations, a drizzle-kit importer | 68 tables, 90 indexes, 47 unique, 75 folders | L; can wait, drizzle-kit can keep migrating |
+
+## P1
+
+- Queries: ORDER BY an expression with NULLS FIRST/LAST; GROUP BY an expression; right and full
+  joins; ON callbacks that see earlier joins; UNION, INTERSECT, EXCEPT; `row_number`, `rank`,
+  `lag`, `lead` and RANGE frames; portable `case` and `cast` (`if_` / `multiIf` emit ClickHouse
+  function names, which Postgres does not have); `select *`.
+- ClickHouse: FINAL, PREWHERE, LIMIT BY, SETTINGS on SELECT, the ARRAY JOIN clause.
+- Writes: UPDATE ... FROM with a typed VALUES source; ClickHouse `ALTER TABLE ... UPDATE/DELETE`
+  and lightweight DELETE; JSONEachRow bulk insert (`encodeInsertRows`); a set-all-from-`excluded`
+  helper; client-side default hooks like `$defaultFn` / `$onUpdate`.
+- Runtime: `runFirst` / `runSingle` / `stream`; `observe` with duration, rows and error; replica
+  routing for `route()`; a `SET LOCAL` helper for RLS; a mock `Database` for users' tests.
+- Types: select, insert and update row types and Schemas per table.
+- Tooling: `pull` (introspect to `defineTable`) and `push`, ClickHouse first.
+
+## P2
+
+Lateral joins, recursive and materialized CTEs, ROLLUP / CUBE, FETCH, MERGE, `db.batch`,
+TRUNCATE, DELETE USING, `DEFAULT VALUES`, expression conflict targets, schemas / namespaces, enums,
+views, relations and a relational query API, casing (has to live in the builder: decoding reads
+aliases exactly), a query cache, EXPLAIN on `Database`, controlled transactions, seeding, a studio,
+more dialects.
+
+## Where effect-orm is ahead
+
+- Tenant scope derived at compile time, through joins, CTEs, subqueries, unions and inserts.
+- Row decoding derived from each query, reversible (`encodeRows`), with `rowSchemaSource`,
+  `untypedColumns` and `rowSchemaMismatch` saying where typing was lost.
+- Transactions: typed COMMIT and ROLLBACK failures, `TransactionClosed`, contention retry,
+  `requireTransaction` in the type.
+- Dialects refuse unsupported clauses before sending SQL, and a query compiled for another one.
+- The ClickHouse migration runner: per-statement journal and resume, cluster and replication as
+  render options, drift checked against the last applied snapshot.
+- Columns are codecs: `jsonb(schema)` validates at runtime; wire quirks are absorbed once.
+
+Drizzle 1.0 now ships Effect drivers (`effect-postgres`, `effect-pglite`) and Effect Schema
+validators, which narrows the lead on Effect integration.
+
+## Order of work
+
+1. The INSERT fixes above.
+2. UPDATE and DELETE with RETURNING, reusing the insert's SET record, value encoding and
+   RETURNING path.
+3. The expression-level `sql` template, null and range predicates, variadic `and` / `or`,
+   DISTINCT and locking.
+4. Postgres column types and modes, jsonb operators, constraint error helpers.
+5. Tenant enforcement and observability in `Database`.
+6. Postgres schema-as-code and migrations.

design/writes.md

@@ -1,7 +1,8 @@
 # Writes: INSERT
 
 Status: phases 1 to 4 built (`values`, `returning`, `onConflictDoNothing` /
-`onConflictDoUpdate`, `select`, `settings`; see `docs/inserts.md`). `encodeInsertRows` (§7) is
+`onConflictDoUpdate`, `select`, `settings`; see `docs/inserts.md`), and UPDATE and DELETE (§12,
+`docs/updates-and-deletes.md`). `encodeInsertRows` (§7) is
 not built: no consumer has asked for it. Section 11 lists where the build differs from the plan. UPDATE and DELETE come later and
 will reuse what this note sets up (the write-statement state, the `RETURNING` path, value
 encoding).
@@ -257,4 +258,23 @@ note, reusing `kind: "write"`, the returning path and the `set` record type from
   Its tenant scope is the SELECT's for an untenanted target (the read), and for a tenant target
   single-tenant only when the read is and each row takes its tenant from a source tenant column
   or the same param.
-- **Settings are a dialect clause**, `DialectClauses.insertSettings`, like the others.
+- **Settings are a dialect clause**, `DialectClauses.writeSettings`, like the others.
+
+## 12. UPDATE and DELETE
+
+`update(table).set(...).where(...)` and `deleteFrom(table).where(...)`, in `src/ch/update.ts`,
+compiled beside INSERT and sharing its value encoding (`valueCells`), SET record
+(`setAssignments`, also used by `onConflictDoUpdate`), RETURNING (`returningOf`) and settings.
+
+- **No WHERE is refused.** No `where()` is a defect; a `where()` whose conditions are all
+  `undefined` is a failure, since optional filters come from data and that case would widen the
+  write to every row. `allRows()` opts in.
+- **ClickHouse**: UPDATE is an `ALTER TABLE ... UPDATE` mutation (`DialectClauses.alterTableUpdate`),
+  the one form every supported server takes; DELETE is the lightweight `DELETE FROM`. Both need
+  a WHERE, so `allRows()` writes `WHERE 1`. Settings go last (`mutations_sync`,
+  `lightweight_deletes_sync`), and `DialectClauses.insertSettings` became `writeSettings`.
+  Checked on 26.2.19.43 and 26.8.2.7.
+- **Tenant scope** is derived from the WHERE as for a query over the table; an UPDATE that sets
+  the tenant column to anything but the pinned value is cross-tenant.
+- Not built: UPDATE ... FROM / joins, DELETE USING, ORDER BY / LIMIT, a VALUES source for bulk
+  updates. See `design/gap-review.md`.

docs/README.md

@@ -16,7 +16,7 @@ You do not need a Maple account, Maple's schema, or tenant columns. Tenant analy
 optional feature for applications that share tables between tenants.
 
 The root builder does not manage connections, create tables, or run migrations. It builds
-SELECTs and [INSERTs](./inserts.md); UPDATE and DELETE are not built yet. Opt-in [schema and migration entry points](./migrations.md) add DDL and migrations for
+SELECTs, [INSERTs](./inserts.md), and [UPDATEs and DELETEs](./updates-and-deletes.md). Opt-in [schema and migration entry points](./migrations.md) add DDL and migrations for
 ClickHouse. It does not validate SQL against a live server, choose query plans, enforce authorization,
 or supply retries. Existing ClickHouse tables and your executor own those responsibilities.
 [Getting started](./getting-started.md) covers npm installation and building from source.
@@ -47,6 +47,7 @@ Roughly in reading order.
 | [Joins and subqueries](./joins-and-subqueries.md)     | The join family, `fromQuery`, correlated subqueries                                 |
 | [Unions and CTEs](./unions-and-ctes.md)               | `unionAll`, `fromUnion`, `withCTE`                                                  |
 | [Inserting rows](./inserts.md)                        | `insertInto`, the insert row type, `DEFAULT`, binding                               |
+| [Updating and deleting](./updates-and-deletes.md)     | `update`, `deleteFrom`, `allRows`, ClickHouse mutations                             |
 | [Params and compilation](./params-and-compilation.md) | `param.*`, how values reach the SQL, `CompiledQuery`                                |
 | [Decoding results](./decoding-results.md)             | `rowSchema`, `decodeRows`, `decodeFirstRow`, decode errors                          |
 | [Running a query](./running-queries.md)               | Executing the SQL with a real client, wire settings, `SETTINGS`                     |

docs/database.md

@@ -99,12 +99,13 @@ Calling `withdraw(1, 30)` outside `transfer` does not compile: `requireTransacti
 
 ### Queries and statements
 
-`run` takes the query you built, a `unionAll`, an `insertInto`, or a query compiled elsewhere. It compiles with
+`run` takes the query you built, a `unionAll`, an `insertInto`, `update` or `deleteFrom`, or a
+query compiled elsewhere. It compiles with
 the database's dialect, so you never pick a `compile`; `params` fills the query's `param.*`
 markers, and a missing one fails with `QueryBuilderError`. A query compiled elsewhere must
 have been compiled for the same dialect, or `run` dies: the root `compile` is ClickHouse's.
 
-`sql` writes the statements the builder does not have yet (UPDATE, DELETE, DDL, advisory
+`sql` writes the statements the builder does not have yet (DDL, bulk `UPDATE ... FROM`, advisory
 locks). Each `${value}` is bound, as `$1, $2, ...` on Postgres and as an escaped literal on
 ClickHouse, so nothing in a value becomes SQL. A `sql` inside another is spliced, so
 statements compose. Names go through `sql.identifier`, which accepts only plain identifiers
@@ -247,7 +248,8 @@ fails at BEGIN today: through the query path with a syntax error, and through `a
 
 ## Writes
 
-The builder compiles SELECTs and [INSERTs](./inserts.md). `run` runs an insert and returns its
-`returning` rows, decoded, or none without `returning`; an insert without it goes through
-`command`, as `execute` does. Write UPDATE and DELETE with `sql`, as above, and read `RETURNING`
-with `query` and a schema.
+The builder compiles SELECTs, [INSERTs](./inserts.md) and
+[UPDATEs and DELETEs](./updates-and-deletes.md). `run` runs a write and returns its `returning`
+rows, decoded, or none without `returning`; a write without it goes through `command`, as
+`execute` does. Write other statements with `sql`, as above, and read `RETURNING` with `query`
+and a schema.

docs/inserts.md

@@ -53,7 +53,11 @@ Each row is typed from the table:
 `defaults`: a Postgres `serial` or `DEFAULT now()`, a ClickHouse `DEFAULT`. A table declared
 with [`defineTable`](./migrations.md) works this out from its column options: a column with
 `default` or `defaultExpr` is optional, and a `materialized` or `alias` column cannot be inserted
-at all (it is not in the row type, and a row that names it anyway fails to compile).
+at all (it is not in the row type, and a row that names it anyway fails to compile). On `table()`,
+list such columns (a Postgres `GENERATED ALWAYS` column) with `computed`; they stay readable.
+
+`insertInto(table)` offers only `values` and `select` until it has rows (its type is
+`CHInsertStart`), so an insert cannot be compiled or run before it says what to insert.
 
 ClickHouse fills every column it is not given with a default, even without a `DEFAULT` clause:
 `0` for a number, `''` for a string. The row type still requires those columns unless you list
@@ -124,8 +128,8 @@ timeout; run those in slices.
 ## Returning
 
 On Postgres, `returning` adds a RETURNING list and `Database.run` returns the inserted rows,
-decoded. It takes column names, or a callback building one expression per alias, as `select`
-does:
+decoded. With no arguments it returns every column, as Drizzle's bare `.returning()` does; it
+also takes column names, or a callback building one expression per alias, as `select` does:
 
 ```ts
 const created = CH.insertInto(ApiKeys)

docs/reference.md

@@ -62,7 +62,9 @@ Note `/sql` exports a `compile` (fragment → string) distinct from the root `co
 | `fromQuery` | `(query, alias) => CHQuery`               |
 | `fromUnion` | `(union, alias) => CHQuery`               |
 | `unionAll`  | `(...queries) => CHUnionQuery`            |
-| `insertInto` | `(table) => CHInsert`; `.values(row \| rows)` or `.select(query)` sets its rows, `.settings(record)` ClickHouse `SETTINGS`, `.returning(...)` the RETURNING list, `.onConflictDoNothing(options?)` / `.onConflictDoUpdate(options)` the ON CONFLICT clause (Postgres). See [Inserting rows](./inserts.md) |
+| `update`     | `(table) => CHUpdateStart`, then `CHUpdate`: `.set(record \| fn)`, `.where(fn)` or `.allRows()`, `.returning(...)`, `.settings(record)`. See [Updating and deleting](./updates-and-deletes.md) |
+| `deleteFrom` | `(table) => CHDelete`: `.where(fn)` or `.allRows()`, `.returning(...)`, `.settings(record)` |
+| `insertInto` | `(table) => CHInsertStart`, then `CHInsert`; `.values(row \| rows)` or `.select(query)` sets its rows, `.settings(record)` ClickHouse `SETTINGS`, `.returning(...)` the RETURNING list, `.onConflictDoNothing(options?)` / `.onConflictDoUpdate(options)` the ON CONFLICT clause (Postgres). See [Inserting rows](./inserts.md) |
 
 ### `CHQuery` methods
 
@@ -279,7 +281,7 @@ Types: `WindowSpec`, `CompiledWindowSpec`, `WindowFrameBound`, `WindowRowsFrame`
 
 **Everything else** — `Table`, `TableOptions`, `Expr`, `ColumnRef`, `Condition`, `Comparable`
 (what a value of a type may be compared against), `MapValueOf`, `Subquery`, `ParamMarker`,
-`ParamKind`, `CHQuery`, `CHUnionQuery`, `CHInsert`, `InsertRow`, `InsertRowOf`, `InsertValue`, `InsertSelectMisfits`, `InsertSelectMissing`, `InsertSettingValue`, `ConflictTarget`, `ConflictSet`, `OnConflictDoNothing`, `OnConflictDoUpdate`, `ColumnAccessor`, `JoinedColumnAccessor`,
+`ParamKind`, `CHQuery`, `CHUnionQuery`, `CHInsert`, `CHInsertStart`, `CHUpdate`, `CHUpdateStart`, `CHDelete`, `CHWrite`, `UpdateSet`, `UpdateSetOf`, `InsertRow`, `InsertRowOf`, `InsertValue`, `InsertSelectMisfits`, `InsertSelectMissing`, `InsertSettingValue`, `ConflictTarget`, `ConflictSet`, `OnConflictDoNothing`, `OnConflictDoUpdate`, `ColumnAccessor`, `JoinedColumnAccessor`,
 `JoinOnCallback`, `CompiledQuery`, `CompiledQueryInput`, `CompiledQueryRowSchema`, `RowSchemaMismatch`, `TenantScope`, `Dialect`, `DialectClauses`, `DialectTransactions`, `IsolationLevel`, `TransactionSettings`, `ParamStyle`, `FnResult`,
 `WindowFunnelMode`, `WindowSpec`, `WindowRowsFrame`, `WindowFrameBound`,
 `WindowOrderDirection`, `CompiledWindowSpec`.

docs/tables-and-types.md

@@ -24,7 +24,8 @@ query time. Treat the declaration as a contract you keep in sync with your migra
 
 The third argument takes options. `tenantColumn` names the column that carries tenancy (see
 [Tenant scoping](./tenant-scoping.md)); `defaults` lists the columns the database fills in when
-an insert leaves them out (see [Inserting rows](./inserts.md#which-columns-have-defaults)).
+an insert leaves them out, and `computed` the ones an insert may not write at all (see
+[Inserting rows](./inserts.md#which-columns-have-defaults)).
 
 ## Column types