Repository navigation
Parse MODULE.bazel with Python's ast in the msan guard - #433
Conversation
Follow-up to #423, mirroring #424. Starlark's syntax is a subset of Python's, so ast.parse handles strings, comments, continuations and statement boundaries exactly. This fixes the scanner joining "dep_function = bazel_dep" and a following "(...)" tuple into a fictitious call, and replaces ~1000 lines of lexer and its tokenizer-level tests with ~70 lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mirrors #424. ast records keyword unpacking as a keyword with arg None, so local_path_override(**{"module_name": "toolchains_llvm", ...}) read as having no module_name and slipped past the guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Mirrored #424's keyword-unpacking fix (6085e98): |
There was a problem hiding this comment.
🟢 Approval recommended
The focused rewrite addresses the parsing defect, with regression coverage and no identified blocking issues.
0 open findings
What changed in this PR
Simplifies the MemorySanitizer CI guard by replacing its hand-written Starlark lexer with Python’s ast parser.
Changes:
- Fixes incorrect joining of separate statements into calls.
- Preserves dependency-version and override checks.
- Consolidates parser tests and adds statement-boundary and argument-unpacking coverage.
| File | Description |
|---|---|
python/tests/ci_msan_test.py |
Replaces the lexer with AST helpers and updates regression tests. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Follow-up to #423, which was merged with a hand-written Starlark lexer in
python/tests/ci_msan_test.py. This applies the same rewrite as #424.Why
The lexer kept drawing review findings. The latest: it joined separate statements into a fictitious call. With
_toolchains_llvm_bazel_dep_version()returned9.9.9. A one-line version bump had also grown the file by ~1400 lines.What
Starlark's syntax is a subset of Python's, so the guard now parses
MODULE.bazelwithast.parse. It walks theast.Callnodes whose callee is a barebazel_dep/*_overridename. Python's tokenizer handles strings, escapes, raw strings, comments, continuations and statement boundaries exactly:ext.bazel_dep(...)are excluded;The regression test for the example above was added first and confirmed failing. Tests that only exercised tokenizer details (escapes, raw-string backslash parity, comment placement, continuations) were removed. The fail-closed cases are now subtests of one test. The file goes from 1683 to 614 lines; the non-parser tests are unchanged. Full
//python/...suite green (24/24).🤖 Generated with Claude Code