Repository navigation
Add PostgreSQL partial-column inserts with composable CTE support - #827
Conversation
- Introduced `pgWithSyntax` to prefix PostgreSQL statements with CTEs. - Added new types for data-modifying CTEs: `PgDataModifyingCommonTableExpressionSyntax` and `PgCommonTableExpressionSyntax`. - Implemented type classes for handling CTEs in SQL queries, ensuring correct placement of data-modifying CTEs. - Created unit and integration tests for rendering and type safety of CTEs, including negative tests for invalid placements. - Updated documentation to reflect new CTE capabilities and usage examples. - Modified test setup to use PostgreSQL version 18.4 for better compatibility.
- Updated CTE handling in tests to reflect PostgreSQL's behavior with zero-column CTEs and side-effect-only CTEs. - Added new tests for materialization execution and rendering, ensuring compliance with PostgreSQL 12+ features. - Refined error handling for invalid CTE placements, particularly for side-effect-only operations. - Improved documentation on PostgreSQL-specific CTE usage, including materialization options and nested CTEs. - Adjusted dependencies in the SQLite library for compatibility. - Ensured reproducibility of CI runs by pinning PostgreSQL server version.
…mn inserts in PostgreSQL
|
@LaurentRDC could you check this please. |
|
I haven't forgotten about this; I'm just swamped at work right now |
|
Looks good to me overall. |
|
Thanks for the review — pushed a finishing pass, and I'm not planning further changes. Conflict-support docs (your comment on Renamed the new CTE adapters: Documented a limitation: Version: corrected Existing exports and behaviour are unchanged. If it looks good to you we can merge it and make a new release please @LaurentRDC |
LaurentRDC
left a comment
There was a problem hiding this comment.
Thanks for your patience! I will cut a new release today
Summary
Adds PostgreSQL-specific partial-column inserts that preserve
ON CONFLICTsupport and compose cleanly withRETURNINGand data-modifying CTEs.Motivation
The existing PostgreSQL insert helpers support
insertValues,insertExpressions, andinsertFrom, but require values for every table column. This prevents CTE inserts from omitting generated, identity, or database-defaulted columns.Beam’s backend-independent
insertOnlysupports partial-column inserts, but cannot express PostgreSQL’sON CONFLICTbehavior. Beam also provides backend-independent conflict handling throughBeamHasInsertOnConflict, whoseinsertOnConflictaccepts full-table values.pgInsertOnlyadds partial-target selection with conflict handling.For query-backed inserts,
default_cannot substitute for omitted columns: PostgreSQL rejectsSELECT DEFAULT, source.name FROM source, whileINSERT INTO users (name) SELECT source.name FROM sourcelets the omitted column use its default.Changes
pgInsertOnlyfor partial-column PostgreSQL inserts withPgInsertOnConflict.cteInsertCommandto lift anySqlInsert Postgres tableinto a side-effect-only CTE.cteInsertCommandReturningto attach aRETURNINGprojection and expose the inserted rows as a reusable CTE.insert,insertReturning,cteInsert, andcteInsertReturningAPIs.beam-postgres-0.6.4.0.BEAM_POSTGRES_TEST_CONNSTR.Design
Partial-column selection is treated as a property of insert construction, while conflict handling,
RETURNING, and CTE placement remain independently composable:pgInsertOnlydelegates target-column projection and base SQL generation to Beam’s existinginsertOnlyimplementation, then adds PostgreSQL’s conflict clause. This avoids maintaining a second insert renderer and keeps column ordering, quoting, empty-input behavior, and type checking aligned with the backend-independent implementation.The command-level CTE adapters also avoid introducing narrowly specialized partial-insert CTE APIs.
Compatibility
This is additive and does not require changes to existing user code. Existing PostgreSQL insert and CTE helpers retain their signatures and behavior.
cteInsertCommandReturningreturnsNothingonly forSqlInsertNoRows. A validON CONFLICT DO NOTHINGcommand still returns a reusable relation, which may contain no rows after execution.For both command adapters,
SqlInsertdoes not carry the target table’s database type, so that type is not tied to the enclosingPgWithblock.Validation
beam-postgrestest suite: 149 tests passed against PostgreSQL 18.4 with GHC 9.6.7.insertDataandinsertFrom;ON CONFLICT DO NOTHING;ON CONFLICT DO UPDATEandexcludedvalues;git diff --checkpasses.