Schema filtering - #30
Merged
Merged
Conversation
Composable with the existing -- area: tag filtering, but derived automatically by parsing each file's SQL instead of requiring a tag. Wired into pgdb compare, migrate check/apply, and testdb up/reset (the latter two also gain --area/--exclude-area, which they lacked entirely until now). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01StWgZw5TDd7KNGY3VzNGMg
…refs The EXEC(...)-literal-misparse guard in sql_schemas() rejected any Table node without an identifier-shaped name, including the legitimate empty name sqlglot gives a bare CREATE SCHEMA (whose name lives in `db`, not `this`). That silently dropped the CREATE SCHEMA's schema whenever the same file had another real table reference too (the empty-schemas regex-fallback safety net never triggered). Found by code review. Also de-duplicates the system-schema exclusion list (pg_catalog/ information_schema/sys) into dialect.SYSTEM_SCHEMAS, shared by schemas.py and testdb/schema.py instead of each keeping its own copy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01StWgZw5TDd7KNGY3VzNGMg
New public surface (pgdevkit.schemas module, --schema/--exclude-schema on compare/migrate check/apply, --area/--exclude-area and --schema/--exclude-schema on testdb up/reset) warrants a minor bump, not a patch. auto-release.yml tags/releases/publishes automatically once this lands on main. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AggszcqpDQAqPPEJCVVCLM
Verified empirically that parse_directory() and _iter_sql_files() already skip parse_areas()/sql_schemas() entirely when that axis isn't being filtered on (each is gated by "(areas or exclude_areas) and ..." / "(schemas or exclude_schemas) and ..."), and that filter_by_area()/filter_by_schema() never touch the filesystem when neither only nor exclude is set -- no behavior change needed, this was already the case. Adds explicit call-counting tests so a future edit can't silently make either check unconditional. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AggszcqpDQAqPPEJCVVCLM
- sql_schemas()'s regex fallback scanned raw file text, so a schema merely *mentioned* in a "--" comment (e.g. "-- see billing.invoice for context") was wrongly detected as a real reference, defeating both --schema (falsely excluded) and --exclude-schema (falsely dropped) for such a file. Fixed by stripping comments first, via a comment-stripper extracted out of migrate.py into a new shared pgdevkit/sql_text.py module (schemas.py couldn't import migrate.py's copy directly -- migrate.py already imports schemas.py). - The guard against sqlglot's EXEC(...)-literal misparse required a table's name to fully match \w+, which also rejected legitimate quoted identifiers containing punctuation (e.g. billing."my-table"), silently dropping that table's schema. Loosened to reject only names containing whitespace, which still catches the original misparse (a full SQL statement embedded as literal text) without rejecting real quoted identifiers. Both reproduced and confirmed fixed against sql_schemas() directly; regression tests added for each, plus for the existing EXEC(...) and bare-CREATE-SCHEMA cases they abut. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AggszcqpDQAqPPEJCVVCLM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.