Repository navigation
SNOW-4483962: Support change-tracking reads (CHANGES clause) and END - #4505
Open
sfc-gh-yixie wants to merge 1 commit into
Open
sfc-gh-yixie wants to merge 1 commit into
sfc-gh-yixie wants to merge 1 commit into
Conversation
Add `changes` ("default" / "append_only") and `end_statement`, `end_offset`,
`end_timestamp`, `end_timestamp_type` to Session.table, DataFrameReader.table
(also as reader options) and Table. They emit
`CHANGES (INFORMATION => ...) AT|BEFORE (...) [END (...)]` on top of the
existing time travel clause. Validation: `changes` requires AT/BEFORE, END
requires `changes`, at most one END argument.
Add METADATA_ACTION, METADATA_ISUPDATE and METADATA_ROW_ID column constants.
Local testing raises NotImplementedError for CHANGES reads instead of silently
returning the plain table. The new parameters are not encoded in the AST yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sfc-gh-yixie
requested review from
sfc-gh-aling,
sfc-gh-mayliu and
sfc-gh-yuwang
November 13, 2026 00:54
3 of 9 tasks
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4505 +/- ##
==========================================
- Coverage 95.47% 94.98% -0.49%
==========================================
Files 179 179
Lines 46927 46979 +52
Branches 8110 8119 +9
==========================================
- Hits 44803 44625 -178
- Misses 1332 1513 +181
- Partials 792 841 +49 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-4483962
Fill out the following pre-review checklist:
snowflakedb/local-testingPlease describe how your code solves the related issue.
This PR adds Snowflake change-tracking reads, the
CHANGESclause, plus theENDbound of the change interval, by extending the existing time-travel arguments.New keyword arguments on
Session.table,DataFrameReader.tableandTable:changes("default"or"append_only", case-insensitive)end_statement,end_offset,end_timestamp,end_timestamp_typeThe same names work as
DataFrameReaderoptions, e.g.session.read.option("changes", "append_only").Validation in
TimeTravelConfigis kept minimal:changesrequires a time-travel mode and exactly oneAT/BEFOREargument. Without them Snowflake silently returns no rows and no columns.ENDarguments requirechanges, and only one may be given.end_offsetis converted to an int, so a string from an option can't add arbitrary SQL.Snowflake checks everything else; for example, it accepts
BEFORE (TIMESTAMP => …)withCHANGES, although its docs only listSTATEMENT.ENDtimestamps are normalized the same way as the starttimestamp, using a helper shared by both.Metadata columns: new constants
METADATA_ACTION,METADATA_ISUPDATEandMETADATA_ROW_IDinsnowflake.snowpark.columnfor the extra columns of a change-tracking read.Local testing: a
CHANGESread raisesNotImplementedError. The mock does not model time travel, and without this check it would silently return the plain table.AST: not included in this PR (checkbox intentionally unticked). The
TableandReadTableAST messages are generated from the monorepo IR (Snowpark/ast/dataframe.iranddataframe-io.ir). Recording the new arguments therefore needs an IR change there first. The new arguments are not recorded in the AST yet, the same as the existing Iceberg time-travel options. Code comments mark where they would go, and the IR and AST work will follow separately.Testing:
tests/unit/test_utils.py): SQL generation for bothchangesvalues, eachENDkind (including tz-aware datetimes and quote escaping), and every validation error.tests/integ/test_dataframe.py): on a change-tracking table with an insert, an update and a delete, they check the exact rows, includingMETADATA$ACTION/METADATA$ISUPDATE. They cover DEFAULT and APPEND_ONLY,ATandBEFORE, with and withoutEND, the reader-option path,copy.copy, and the SQL simplifier on and off.NotImplementedError. The local-testing runs oftest_dataframe.py/test_function.py,tests/mockand the existing time-travel and AST tests still pass.🤖 Generated with Claude Code