Skip to content

fix!: parse annotation unions consistently as types - #37

Merged
exKAZUu merged 3 commits into
mainfrom
fix/annotation-union-consistency
Oct 20, 2026
Merged

exKAZUu merged 3 commits into
mainfrom
fix/annotation-union-consistency

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Why

The same union annotation produces different syntax trees depending on operand order. str | int is a binary expression, while IO[bytes] | int | List[int] mixes a union with a binary expression and a subscript. This is the ambiguity reported in tree-sitter/tree-sitter-python#293, tree-sitter/tree-sitter-python#333, and tree-sitter/tree-sitter-python#334.

Requirements

  • required: Improve our forked grammar repositories as much as possible, ignoring backward compatibility, and create a dedicated nonduplicate PR from latest main for each actionable upstream item.
  • required: Run review-booster and complete-pr concurrently, then release the changes.
  • required: Continue accepting valid annotation expressions, generic type arguments, type parameter bounds and defaults, and variadic annotations.
  • chosen: Reuse the contextual type-expression rules for ordinary annotations so recognizable type unions have the same recursive type structure regardless of operand order, while runtime value expressions retain their expression trees.
  • chosen: Keep constrained type syntax in parameter lists and generic arguments, preserving bound and slice handling without allowing a function-body colon to start another annotation.

Customer Summary

Ordinary parameter, return, variable and type-alias annotations consistently expose named, qualified and generic union operands as types. Generic receivers retain their type nodes through member access and repeated type arguments. Qualified annotation names and constrained tuple elements also receive contextual type nodes. Value expressions and default argument values keep their existing expression representation.

Technical Summary

Route type through the contextual expression grammar already used for type-parameter defaults. Its member and generic receivers accept nested generic types, supporting chains such as Outer[int].Inner[str].Leaf and a[b][c] | d. Remove the duplicate generic, union and member rules and their stale conflicts. Constrained expressions retain their outer type wrapper in declaration and generic-argument contexts. Regenerate the parser and update the three affected corpus expectations.

The union nodes remain recursive, preserving operand grouping. Arbitrary annotation expressions retain the general expression fallback.

Testing

  • The union regression fails on main for str | int; its value-expression control passes. The generic-receiver regression reproduces the missing type nodes before the repair.
  • CPython accepts 18 ordinary annotation regression sources and all six generic-receiver boundary annotations checked with ast.parse.
  • Canonical bun run build-wasm passes after integration with current main.
  • Eleven focused test files pass (23 tests), including all 210 corpus cases, annotation/query boundaries, qualified/contextual generics, variadic annotations, type parameters, comparison/match fields, deep indentation, f-string formats and incremental continuation edits.
  • bun run verify passes. The existing code-gauge violations concern imported fixtures and scanner logic unchanged by this PR.
  • All PR CI checks pass, including integration, browser/Workers, Rust, platform, and fuzz checks.

Notes

This intentionally changes annotation AST shapes, so the title requests a major release. The runtime dependency and ABI remain unchanged.

Existing code-gauge errors remain in the unchanged scanner function tree_sitter_python_external_scanner_scan; GrammarTests.testFuncdef in examples/python2-grammar-crlf.py, examples/python2-grammar.py, examples/python3-grammar-crlf.py, and examples/python3-grammar.py; and file NCSS plus GrammarTests.test_funcdef in examples/python3.8_grammar.py. The imported parser fixtures stay intact, and changing the unrelated scanner would broaden this annotation change.

exKAZUu and others added 3 commits October 21, 2026 03:17
Co-authored-by: WillBooster (Codex CLI) <codex@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <codex@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <codex@willbooster.com>
@exKAZUu exKAZUu self-assigned this Oct 20, 2026
@exKAZUu
exKAZUu merged commit dd07c04 into main Oct 20, 2026
10 checks passed
@exKAZUu
exKAZUu deleted the fix/annotation-union-consistency branch October 20, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant