Repository navigation
Conversation
There was a problem hiding this comment.
📋 Review Summary
This PR resolves three important bugs in Spanner Change Streams (SpannerIO.readChangeStream()): fixing PartitionMetadataDao.tableExists() for PostgreSQL-dialect databases by checking table_schema = 'public', parameterizing getPartitionsMatchingState() and short-circuiting empty token updates in updateToScheduled(), and adding support for schema-qualified change stream and TVF names across SpannerIO and ChangeStreamDao. The changes are well-structured, adhere to existing SpannerIO patterns, and include comprehensive unit tests for both GoogleSQL and PostgreSQL dialects.
🔍 General Feedback
- Switching
PartitionMetadataDao.getPartitionsMatchingState()in the PostgreSQL branch from string concatenation to bound query parameters ($1andANY ($2)) is a great improvement for both query plan caching and safety. - The schema-qualification handling across
fetchPartitionMode,checkTvfExistence, andChangeStreamDao.changeStreamQuerycleanly accounts for the differences between GoogleSQL (<schema>.READ_<name>) and PostgreSQL ("spanner"."read_json_<name>"/"spanner"."read_proto_bytes_<name>").
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
72bf8ca to
77dbaa6
Compare
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
77dbaa6 to
4ece3c1
Compare
|
Assigning reviewers: R: @nielm for label spanner. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Summary
This PR fixes two PostgreSQL metadata query issues in the Spanner Change Streams connector (
PartitionMetadataDao):PartitionMetadataDao.tableExists()on PostgreSQL-dialect databases:tableExists()hardcodedWHERE t.table_catalog = '' AND t.table_schema = ''. In PostgreSQL-dialect Spanner databases,table_catalogis the database name and user tables reside intable_schema = 'public', causingtableExists()to always returnfalse.InitializeDoFnruns more than once (e.g., on retry or pipeline restart with an existing metadata table),createPartitionMetadataTable()(CREATE TABLE IF NOT EXISTS) succeeds as a no-op, andcreateFakeParentPartition()subsequently fails withALREADY_EXISTS: Row [Parent0] in table Metadata_... already exists.tableExists()to checkt.table_schema = 'public'whenthis.isPostgres()is true (consistent withfindAllTableIndexes()).PartitionMetadataDao.getPartitionsMatchingState()on PostgreSQL-dialect databases:getPartitionsMatchingState()to use bound query parameters (WHERE "State" = $1 AND "PartitionToken" = ANY ($2)) instead of string concatenation, and to omit the"PartitionToken"filter whencomposedPartitionTokensis empty (matching the GoogleSQL branch behavior).Testing
PartitionMetadataDaoTestcovering both GoogleSQL and PostgreSQL dialects.