Skip to content

feat!: generate the Python grammar for runtime 7 ABI 6 - #35

Merged
exKAZUu merged 2 commits into
mainfrom
feat/runtime-7-abi-6
Oct 20, 2026
Merged

exKAZUu merged 2 commits into
mainfrom
feat/runtime-7-abi-6

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Why

The published Python grammar cannot be set on runtime 7: its language version is 0x57420001, while the current runtime requires 0x57420006. This addresses the Python portion of WillBooster/tree-sitter#213 by regenerating both native and Wasm artifacts with the current published toolchain.

Requirements

  • required: Improve our forked tree-sitter and grammar repositories as much as possible, ignoring backward compatibility; review our issues and major upstream issues, PRs and recent commits and create dedicated nonduplicate PRs from latest main for eligible items.
  • required: Publish a Python grammar whose Wasm and Rust crate are generated for the latest published runtime ABI and whose peer range covers that runtime.
  • required: Run review-booster and complete-pr concurrently, then release the changes.
  • chosen: Pin the web test runtime and locked Rust/CLI runtime to published 7.0.5, generating ABI 6 while keeping the existing grammar, query behavior and exact runtime-version check.
  • chosen: Update three invalid-source corpus trees to the current runtime's recovery output. Public parsing confirms complete source coverage, retained errors, preserved following definitions and fresh/incremental agreement; a literal NUL remains distinct from EOF.
  • chosen: Publish a major grammar release because consumers must migrate to runtime 7; update the runtime and crate examples to that release contract.

Customer Summary

The Python Wasm grammar and Rust crate load with the current WillBooster runtime 7. Consumers must use runtime 7.x with the new major grammar version.

Technical Summary

The dependency manifests and lockfiles select runtime 7.0.5. Canonical generation updates src/parser.c and its generated headers to ABI 6. Runtime and crate examples document the new compatible versions. Grammar rules and shipped queries are unchanged.

Three existing invalid-source expectations follow the new runtime: incomplete trailing backslashes stay inside their error ranges instead of becoming EOF-matching continuation tokens, and reserved-keyword recovery gives the malformed dot its own error node. Literal NUL controls confirm that the existing NUL token alternative no longer matches EOF.

Testing

  • bun run build/ci using the official released CLI 7.0.5; generated language version is 0x57420006.
  • Both native and Wasm corpus suites: all 207 cases pass.
  • Fourteen focused test files: 21 tests pass, covering f-string formats, field/query traversal, incremental edits, lazy imports, generic/type-parameter trees and highlights, and runtime version alignment.
  • TREE_SITTER_ITERATIONS=100 TREE_SITTER_EDITS=5 bun wb test test/unit/incremental.test.ts: passes across the corpus.
  • cargo test --locked: the Rust binding test and parsing doctest pass. bun run verify passes.
  • Manual public API checks cover each changed invalid-source recovery case, exact error/source ranges, full fresh/incremental tree and range equality, literal NUL versus EOF, and raw-string prefix changed ranges. The old Wasm artifact fails Parser.setLanguage on runtime 7 with the expected ABI error; the regenerated artifact loads and parses through the tests above.
  • Full PR CI passes: examples, longer incremental fuzzing, browser and Workers checks, Rust, Linux ARM, both macOS runners, and sanitizer fuzzing.

Notes

Seven existing code-gauge errors remain outside this change: Python2 example GrammarTests.testFuncdef in both newline variants (159 NCSS), Python3 example GrammarTests.testFuncdef in both variants (187 NCSS), the Python3.8 example file (1,200 NCSS) and its test_funcdef (232 NCSS), and the external scanner dispatch (complexity 175/202 NCSS). Refactoring imported fixtures or the unrelated scanner would expand this ABI migration. Forty-six existing warnings remain. The intentional LF/CRLF example duplicates would be suitable for a code-gauge exclusion; no exclusion configuration is added. Canonical generated C retains the generator's trailing whitespace, so a plain git diff --check reports it.

@exKAZUu exKAZUu self-assigned this Oct 20, 2026
@exKAZUu
exKAZUu merged commit 202f902 into main Oct 20, 2026
10 checks passed
@exKAZUu
exKAZUu deleted the feat/runtime-7-abi-6 branch October 20, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant