Skip to content

fix!: parse qualified generics and unpacked annotations as types - #28

Merged
exKAZUu merged 5 commits into
mainfrom
fix/qualified-variadic-highlights
Oct 19, 2026
Merged

exKAZUu merged 5 commits into
mainfrom
fix/qualified-variadic-highlights

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Why

Qualified annotations such as typing.Tuple[*Ts] and composite unpacked annotations such as *args: *tuple[int, str] parse as value expressions. Their type identifiers therefore miss the shipped highlighting query's type captures. This fixes that separate parser defect; PR #26 covers bare variadic parameter names and class-base names in the existing trees.

Requirements

  • required: Improve the fork by reviewing our issues and eligible upstream issues, PRs, and recent commits; create a dedicated nonduplicate PR for each item worth resolving, ignoring backward compatibility.

  • required: Run review-booster and complete-pr concurrently, then release the changes.

  • required: Existing valid Python annotation and value-expression syntax remains parseable, including expression-valued unpacked annotations and calls inside nested generics.

  • chosen: Resolve the confirmed qualified-generic-base and composite-unpack annotation highlighting gaps through type-role syntax trees, because the shipped query already captures identifiers under type, generic_type, and member_type. Preserve expression fallback rather than treating all subscripts as types.

  • chosen: Wrap unpacked generic arguments and structured parameter-annotation operands in type nodes inside splat_type, and allow qualified member bases in generic_type, so nested type operands receive the same type captures as ordinary annotations. Keep direct expression children for expression-valued starred parameter annotations: the shared primary-expression fallback preserves the existing call tree; wrapping that fallback too is unnecessary for the type-capture defect. Public AST changes warrant a major release; backward compatibility is waived.

  • chosen: Keep ordinary value and default-value unpacks as expressions. Class-base composite unpacks remain a separate issue because class bases use argument-list expression grammar and need their own contextual decision.

  • chosen: Retain primary-expression boundaries for starred parameter annotations and the existing broader expression boundary for generic arguments. CPython 3.14.7 rejects bare lambda, not, and conditional operands in starred annotations, while parenthesized expression-valued operands remain valid.

  • chosen: Highlight only the final member of a qualified type path as a type, leaving module qualifiers with their normal identifier captures. The existing broad member_type pattern colored typing and nested namespace components as types when qualified generic trees became available; retaining that behavior would misclassify modules. Existing default-type highlighting assertions explicitly included qualifiers as types; those expectations change consistently with this chosen semantic correction and the user's compatibility waiver.

  • chosen: Keep plain qualified generic-argument operands and bare contextual-keyword generic bases as separate follow-ups. Paired published 2.3.1/current Wasm parses show the same missing operand/keyword captures and unchanged bare trees; this PR improves qualified bases without worsening those existing expression fallbacks. A broad request to process eligible items still applies to the follow-ups.

  • chosen: Verify ordinary simple and nested qualified generic bases without unpacking through public tree and query APIs, because highlighting alone does not pin the new member_type nesting that tree consumers observe.

Customer Summary

Qualified generics and nested unpacked annotations expose type nodes and receive type highlighting. Calls and other expression-valued annotations stay parseable, and ordinary value unpacks retain their expression trees.

Technical Summary

grammar.js reuses the recursive default-type helpers for unpacked operands and qualified generic bases. Starred parameter annotations retain their primary-expression boundary, while generic arguments retain their existing expression fallback. The highlight query captures the terminal type member while retaining ordinary identifier coloring for module qualifiers. Six corpus expectations now include the intended type wrappers. Review the ambiguity between structured types and expression fallback most closely.

Testing

Canonical generation, all 207 native and Wasm corpus cases, bun run verify, and twelve tests in six annotation, variadic, lazy-import, type-parameter, and performance files pass after integrating main. Four added public tests check actual CLI highlighting, expression-valued operands and recovery, incremental parsing/captures, and plain qualified-base ASTs without unpacking. A one-time CPython 3.14.7 oracle compared 200 inputs against published 2.3.1: all 181 valid inputs remain accepted, no new invalid inputs are accepted, and ordinary value/default-value trees are unchanged. The full suite runs in PR CI.

Notes

The public AST changes require a major release. Composite class-base unpacks, plain qualified generic-argument operands, and bare contextual-keyword generic bases retain their pre-existing expression paths and remain separate follow-ups. The grammar uses the fork runtime 2.x.

Seven pre-existing code-gauge errors remain outside the changed code: the two Python2 fixtures' testFuncdef functions (159 NCSS each), the two Python3 fixtures' testFuncdef functions (187 NCSS each), the Python3.8 fixture file (1200 NCSS) and its test_funcdef (232 NCSS), and the scanner dispatch (complexity 175, 202 NCSS). The fixtures intentionally exercise broad source syntax, and splitting the unrelated scanner would expand this repair. CRLF fixtures intentionally mirror LF fixtures; a code-gauge exclusion for those intentional mirrors would be appropriate. No exclusion is added.

exKAZUu and others added 2 commits October 19, 2026 13:34
Reuse recursive type operands while retaining expression-valued annotation
syntax and ordinary value-unpack trees. Verify public highlighting and
incremental parsing against nested qualified annotations.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu self-assigned this Oct 19, 2026
exKAZUu and others added 3 commits October 19, 2026 13:45
Capture the terminal qualified type member and retain ordinary namespace
identifier coloring in annotations, bounds, and defaults. Align crate
runtime documentation with the supported 2.x ABI.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Keep the scoped class-base and variadic captures and the shared CLI test
runner while retaining terminal-member coloring for qualified types.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Check simple and nested member bases without unpacking, their argument
container and type captures. Clarify the documented base-node contract.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu
exKAZUu merged commit fe09d9a into main Oct 19, 2026
10 checks passed
@exKAZUu
exKAZUu deleted the fix/qualified-variadic-highlights branch October 19, 2026 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant