Skip to content

feat: parse type parameter defaults - #22

Merged
exKAZUu merged 16 commits into
mainfrom
feat/type-parameter-defaults
Oct 18, 2026
Merged

exKAZUu merged 16 commits into
mainfrom
feat/type-parameter-defaults

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 18, 2026 •

Copy link
Copy Markdown
Member

Why

Python 3.13 type parameter defaults currently produce parse errors in class, function, and type-alias declarations, as reported in tree-sitter/tree-sitter-python#346. The published parser reproduces all three failures.

Requirements

  • required: Improve the fork by addressing eligible upstream items in dedicated PRs, avoiding duplicate PRs.

  • required: Parse PEP 696 defaults in class, function, and type-alias declarations, including TypeVar, TypeVarTuple, and ParamSpec forms.

  • chosen: Keep defaults restricted to declaration lists, since generic type arguments do not permit assignment syntax.

  • chosen: Represent each defaulted declaration with a default_type_parameter node exposing name, optional bound, and value fields; keep existing non-default declarations intact.

  • chosen: Reuse the parser's existing expression language for defaults, allowing an unpacked default only for TypeVarTuple; leave compiler-level ordering and type-validity checks to Python tooling. Parenthesized named expressions are accepted by Python 3.14 ast.parse but rejected by compile in symbol-table validation; recursive filtering would diverge from existing bound behavior.

  • chosen: Preserve the existing multi-version expression handling of legacy async/await identifiers in bounds and values. Published 2.1.1 accepts both in nondefault bounds and annotations, although current CPython ast.parse rejects them there too. Reject a special bare-identifier filter for defaults: it would inconsistently leave nested occurrences accepted and duplicate the expression grammar; tightening shared expression syntax is a separate language-version policy change. Declaration names introduced here remain restricted to current contextual identifiers.

  • chosen: Maintain one structural type subset for names, qualified names, generics, unions, grouping, lists and tuples, recursively exposing type nodes for actual rendered highlighting; retain the complete existing expression grammar as a fallback for calls, conditions and other expressions. The subset must accept only expression syntax and preserve operator precedence; do not import declaration constraints or promise type coloring for arbitrary value computations. Dynamic selection preserves the fallback, whereas enclosing query captures are overridden by nested variable captures. These are the shared invariants for the highlight repair chain.

  • chosen: Scoped generic arguments use expression alternatives and the ordinary slice production, never declaration constraints; this preserves slice limits and excludes declarations nested in defaults while retaining recursive type highlighting.

  • chosen: Share inline grammar rules for defaulted declaration names and member/generic bases so each accepted identifier set has one definition; inlining avoids additional parser reductions or public tree nodes.

  • chosen: Parse unpacked generic operands through the same structural expression subset and share current contextual declaration names in generic/member bases; this keeps nested type coloring without adding legacy await/async ambiguity to the structural subset. Legacy names continue through the existing expression fallback.

  • chosen: Render highlights with a worktree-local compiled-parser cache and bounded native CLI startup (170-second process timeout within a 180-second test), because cold compilation on CI can exceed the default five-second assertion timeout.

Customer Summary

Declarations such as class Container[T = str]: pass, def factory[T = str](): pass, and type Alias[T = str] = list[T] parse successfully. Bounds, constraints, parameter-specification defaults, and variadic type defaults are supported.

Technical Summary

Inline name and member-base rules share contextual identifier definitions across declaration forms. Unpacked generic operands retain recursive type nodes for highlighting, and every segment of qualified type names receives the type capture, including nested default member chains.

Add a declaration-only default_type_parameter production and regenerate the profiled parser. The explicit conflict with primary_expression preserves bounded parameters while the parser determines whether a default follows. Ordinary generic arguments retain their separate production. Remove the known-failure entry for CPython’s type-parameter tests, which now parse successfully. The supported declaration forms follow PEP 696; expressions retain the parser's existing language-version policy. A recursive structural type subset preserves rendered highlighting for qualified names, generics, unions, grouping and containers, while the expression fallback preserves other computations.

Testing

  • Six focused tests pass, including rendered CLI HTML for nested, unpacked, qualified and contextual-name generic types and an expression acceptance/rejection matrix.
  • Canonical generation, 206 native corpus cases, 206 Wasm corpus cases and bun run verify pass. Existing code-gauge violations are outside changed code.
  • Published 2.1.1 reproduces the original declaration failures. CPython accepts the positive corpus inputs.
  • CI previously parsed 5,901 real-world files without new failures; the only baseline difference was the now-successful pinned CPython type-parameter file.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu self-assigned this Oct 18, 2026
exKAZUu and others added 12 commits October 18, 2026 23:19
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Regenerate the parser with type parameter defaults and the f-string colon precedence fix.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu changed the title feat: support Python type parameter defaults feat: support type parameter defaults Oct 18, 2026
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu changed the title feat: support type parameter defaults feat: parse type parameter defaults Oct 18, 2026
exKAZUu and others added 2 commits October 19, 2026 05:50
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu
exKAZUu merged commit 90b66f9 into main Oct 18, 2026
10 checks passed
@exKAZUu
exKAZUu deleted the feat/type-parameter-defaults branch October 18, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant