Repository navigation
fix: reserve Java keywords during recovery - #21
Merged
Merged
Conversation
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
After an incomplete member access, Java keywords can become identifier nodes:
object.followed byif (condition)is parsed as a method call, andint class = 1is accepted without an error. This addressestree-sitter/tree-sitter-java#194.Requirements
required: Review eligible upstream items and resolve them in dedicated fork PRs, without requiring backward compatibility.
required: Reserve Java keywords globally so incomplete code cannot consume a following keyword as an identifier.
required: Reject reserved keywords and boolean/null literals as identifiers while preserving valid contextual identifiers and unnamed variables.
chosen: Preserve the existing contextual-name productions and yield precedence because global keyword reservation does not require changing their parsing behavior.
chosen: Represent unused keywords const and goto as hidden lexical tokens so they can be reserved without adding accepted syntax.
chosen: Add explicit unnamed lambda and qualified-super method-reference productions because global reservation exposes valid constructs previously accepted through identifier fallback.
chosen: Keep a lone underscore reserved in expression positions: JLS 25 sections 3.8 and 3.9 forbid it as an identifier, and javac 25 rejects argument/default-value uses; unnamed declaration and lambda bindings remain supported.
chosen: Follow JLS 25 section 15.13 for qualified-super references: the qualifier is a TypeName without type arguments and the target is an Identifier, not new. Reject the suggested permissive alternatives because they accept invalid syntax.
chosen: Retain the current recovery assertions: published 3.0.4 produces neither the requested if_statement nor return_statement for the exact test inputs, whereas the current parser produces both with their full text.
Customer Summary
Keyword recovery preserves following control-flow statements, and declarations using reserved names produce parse errors. Contextual names such as
moduleandyieldremain available in identifier positions.Technical Summary
The global reserved set contains all 51 reserved Java keywords and the three boolean/null literals from JLS §3.8–3.9. Existing named tokens retain their nodes; hidden regex tokens supply const and goto, which have no Java production. Contextual keyword handling remains separate. Explicit productions preserve unnamed lambda parameters and qualified superclass method references; conflicts retain name/type interpretations until the following token distinguishes them.
Testing
Published 3.0.3 baseline accepts all eight invalid-name regression inputs. Added native/Wasm corpus coverage for recovery and contextual names, and public Wasm checks for control-flow recovery, invalid names, literals, keyword prefixes, contextual identifiers, and unnamed variables.
bun run verifypasses. Canonical generation and all 148 corpus cases pass in native and Wasm parsers; all 7,940 pinned real-world examples and 17 focused tests pass. Seven added lambda/method-reference cases fail before the repair and pass afterward; typed lambda and other unnamed-binding controls remain valid.