Repository navigation
SNOW-4432455: Make the DataFrame lineage trace a per-session debug feature - #4487
Merged
Merged
Conversation
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 9, 2026 18:16
c62932f to
020e198
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 9, 2026 18:16
6524d44 to
88d69c0
Compare
sfc-gh-yuwang
marked this pull request as ready for review
November 9, 2026 18:16
sfc-gh-yuwang
requested review from
sfc-gh-aling,
sfc-gh-jzeng and
sfc-gh-yixie
and removed request for
a team
November 9, 2026 18:16
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 10, 2026 18:05
020e198 to
a2090c7
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 10, 2026 18:05
88d69c0 to
27e9bf3
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4487 +/- ##
=======================================
Coverage 95.47% 95.47%
=======================================
Files 179 179
Lines 46927 46922 -5
Branches 8110 8106 -4
=======================================
- Hits 44803 44800 -3
+ Misses 1332 1331 -1
+ Partials 792 791 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 10, 2026 19:42
a2090c7 to
5a347f9
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 10, 2026 19:42
27e9bf3 to
388ab40
Compare
sfc-gh-jzeng
approved these changes
Nov 10, 2026
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 10, 2026 22:15
5a347f9 to
882bfdf
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 10, 2026 22:15
388ab40 to
10db98f
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 10, 2026 22:57
882bfdf to
cde2afd
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 10, 2026 22:57
10db98f to
16924c9
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432458-line-mapping-session
branch
from
November 11, 2026 03:15
cde2afd to
09e040f
Compare
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 11, 2026 03:15
16924c9 to
abb3a9e
Compare
sfc-gh-yuwang
changed the base branch from
yuwang-SNOW-4432458-line-mapping-session
to
main
November 11, 2026 03:22
…ature session.debug.enable(dataframe_trace_on_error=True) adds the lineage of a failed dataframe to the errors of one session; configure_development_features sets every feature through the sessions, and the context module keeps no debug flag. tracing_flags_on() counts the sessions that record the trace. Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
….debug Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
…e failed query has a plan So pyright knows the plan is not None where it is used; the behavior is unchanged. Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code) Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
sfc-gh-yuwang
force-pushed
the
yuwang-SNOW-4432455-lineage-session
branch
from
November 11, 2026 05:44
abb3a9e to
40c1fb5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #4485 (SNOW-4432458, SQL error line mapping per session) and targets main: until #4484 and #4485 merge, the diff also contains their commits. This PR's own commits are the
SNOW-4432455ones.Summary
dataframe_trace_on_erroras a trace debug feature:session.debug.enable_dataframe_trace_on_error()(orset(dataframe_trace_on_error=True)) adds the lineage of a failed dataframe to the errors of that session.wrap_exceptionreads it from the session of the failed plan;is_tracing_enabled(session)reads only the session;tracing_flags_on()(thepublicapifast path, which has no session) checks the count of tracing sessions.configure_development_featuresnow sets all three features in every active session and as the defaults of new ones. Thecontextmodule keeps no debug flag: removescontext._enable_dataframe_trace_on_error.dataframe_trace_on_errorto the features of theSession.debugentry.With this PR,
session.debug.set(eager_schema_validation=True, dataframe_trace_on_error=True, trace_sql_errors_to_dataframe=True)or the per-featuresession.debug.enable_<feature>()/disable_<feature>()configure one session, andconfigure_development_features(...)stays the shortcut for every session.Test plan
test_lineage_trace_is_per_session: two sessions in one process; only the session with the feature on adds the lineage to the error.test_tracing_flags_on_counts_the_tracing_sessions. Thepublicapiunit tests turn the trace on through a session instead of the global.main.test_df_trace_*.py, debug trace, context, session, eager validation and CTE suites: 921 passed, 1 skipped, 1 xpassed.test_df_trace_*.py, debug trace, eager validation and context suites: 622 passed, 1 skipped.Generated with Snowflake CoCo