Repository navigation
fix!: parse qualified generics and unpacked annotations as types - #28
Merged
Merged
Conversation
Reuse recursive type operands while retaining expression-valued annotation syntax and ordinary value-unpack trees. Verify public highlighting and incremental parsing against nested qualified annotations. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Capture the terminal qualified type member and retain ordinary namespace identifier coloring in annotations, bounds, and defaults. Align crate runtime documentation with the supported 2.x ABI. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Keep the scoped class-base and variadic captures and the shared CLI test runner while retaining terminal-member coloring for qualified types. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Check simple and nested member bases without unpacking, their argument container and type captures. Clarify the documented base-node contract. Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Qualified annotations such as
typing.Tuple[*Ts]and composite unpacked annotations such as*args: *tuple[int, str]parse as value expressions. Their type identifiers therefore miss the shipped highlighting query's type captures. This fixes that separate parser defect; PR #26 covers bare variadic parameter names and class-base names in the existing trees.Requirements
required: Improve the fork by reviewing our issues and eligible upstream issues, PRs, and recent commits; create a dedicated nonduplicate PR for each item worth resolving, ignoring backward compatibility.
required: Run review-booster and complete-pr concurrently, then release the changes.
required: Existing valid Python annotation and value-expression syntax remains parseable, including expression-valued unpacked annotations and calls inside nested generics.
chosen: Resolve the confirmed qualified-generic-base and composite-unpack annotation highlighting gaps through type-role syntax trees, because the shipped query already captures identifiers under type, generic_type, and member_type. Preserve expression fallback rather than treating all subscripts as types.
chosen: Wrap unpacked generic arguments and structured parameter-annotation operands in type nodes inside splat_type, and allow qualified member bases in generic_type, so nested type operands receive the same type captures as ordinary annotations. Keep direct expression children for expression-valued starred parameter annotations: the shared primary-expression fallback preserves the existing call tree; wrapping that fallback too is unnecessary for the type-capture defect. Public AST changes warrant a major release; backward compatibility is waived.
chosen: Keep ordinary value and default-value unpacks as expressions. Class-base composite unpacks remain a separate issue because class bases use argument-list expression grammar and need their own contextual decision.
chosen: Retain primary-expression boundaries for starred parameter annotations and the existing broader expression boundary for generic arguments. CPython 3.14.7 rejects bare lambda, not, and conditional operands in starred annotations, while parenthesized expression-valued operands remain valid.
chosen: Highlight only the final member of a qualified type path as a type, leaving module qualifiers with their normal identifier captures. The existing broad member_type pattern colored typing and nested namespace components as types when qualified generic trees became available; retaining that behavior would misclassify modules. Existing default-type highlighting assertions explicitly included qualifiers as types; those expectations change consistently with this chosen semantic correction and the user's compatibility waiver.
chosen: Keep plain qualified generic-argument operands and bare contextual-keyword generic bases as separate follow-ups. Paired published 2.3.1/current Wasm parses show the same missing operand/keyword captures and unchanged bare trees; this PR improves qualified bases without worsening those existing expression fallbacks. A broad request to process eligible items still applies to the follow-ups.
chosen: Verify ordinary simple and nested qualified generic bases without unpacking through public tree and query APIs, because highlighting alone does not pin the new member_type nesting that tree consumers observe.
Customer Summary
Qualified generics and nested unpacked annotations expose type nodes and receive type highlighting. Calls and other expression-valued annotations stay parseable, and ordinary value unpacks retain their expression trees.
Technical Summary
grammar.jsreuses the recursive default-type helpers for unpacked operands and qualified generic bases. Starred parameter annotations retain their primary-expression boundary, while generic arguments retain their existing expression fallback. The highlight query captures the terminal type member while retaining ordinary identifier coloring for module qualifiers. Six corpus expectations now include the intended type wrappers. Review the ambiguity between structured types and expression fallback most closely.Testing
Canonical generation, all 207 native and Wasm corpus cases,
bun run verify, and twelve tests in six annotation, variadic, lazy-import, type-parameter, and performance files pass after integrating main. Four added public tests check actual CLI highlighting, expression-valued operands and recovery, incremental parsing/captures, and plain qualified-base ASTs without unpacking. A one-time CPython 3.14.7 oracle compared 200 inputs against published 2.3.1: all 181 valid inputs remain accepted, no new invalid inputs are accepted, and ordinary value/default-value trees are unchanged. The full suite runs in PR CI.Notes
The public AST changes require a major release. Composite class-base unpacks, plain qualified generic-argument operands, and bare contextual-keyword generic bases retain their pre-existing expression paths and remain separate follow-ups. The grammar uses the fork runtime 2.x.
Seven pre-existing code-gauge errors remain outside the changed code: the two Python2 fixtures'
testFuncdeffunctions (159 NCSS each), the two Python3 fixtures'testFuncdeffunctions (187 NCSS each), the Python3.8 fixture file (1200 NCSS) and itstest_funcdef(232 NCSS), and the scanner dispatch (complexity 175, 202 NCSS). The fixtures intentionally exercise broad source syntax, and splitting the unrelated scanner would expand this repair. CRLF fixtures intentionally mirror LF fixtures; a code-gauge exclusion for those intentional mirrors would be appropriate. No exclusion is added.