Skip to content

Generation of Java AST classes - #3809

Draft
wadoon wants to merge 5 commits into
mainfrom
weigl/javaastgen
Draft

wadoon wants to merge 5 commits into
mainfrom
weigl/javaastgen

Conversation

@wadoon

@wadoon wadoon commented Apr 19, 2026 •

Copy link
Copy Markdown
Member

Related Issue

This PR supersedes/continues the discussion from the KaKeY meeting on writing
KeY-Java-AST. No dedicated issue; tracking happens in this PR.

Intended Change

Writing the KeY Java AST by hand is tedious and error-prone (ExtList/ImmutableArray
plumbing, equals+hashCode+computeHashCode for every node). This PR generates the
AST node classes and the necessary visitor infrastructure from a single, small metamodel file
using JavaParser.

  • New Gradle module key.ncore.java (settings.gradle: include 'key.ncore.java'):
    • src/adt/java-ast.java — the metamodel (declared style, ~180 types; imports reference
      existing KeY types like KeyType, SchemaVariable, ...)
    • src/generator/java/org/key_project/ncore/java/ — the generator (5 files:
      Generator, PreSteps, NodeSteps, PostSteps, ReadAllHierarchy) and a
      runGenerator Gradle task that regenerates the AST
    • src/generated/java/org/key_project/java/ast/ — the generated output (AST nodes,
      visitor package with Visitor/ArgVisitor/VoidVisitor/CopyVisitor/
      CopyOnWriteVisitor)
    • src/main/java/org/key_project/java/ast/ — small hand-written support types
      (Matchable, Visitable, MatchHelper, Root, EqEx, Internal, ...)
  • The generated AST compiles cleanly — ./gradlew :key.ncore.java:classes is
    BUILD SUCCESSFUL with 0 errors. (This was the current iteration's goal; it replaces
    the previous state with ~1250 linker/type errors.)

Java AST features/decisions

  • All fields are immutable.
    • Getter in record style, e.g. String name();
    • Lists are ImmutableList from org.key_project.util.collection.
  • Inner AST nodes are interfaces.
    • @Root node is the single abstract base class JavaSourceElement
      (implements Visitable, Matchable); everything else is a (sealed) interface or a
      concrete final class.
  • Inner AST nodes are sealed for pattern matching support. Marker types without any
    metamodel subtype (e.g. IGuard, LoopInitializer, IExecutionContext) are generated as
    plain (non-sealed) interfaces.
  • Type-safe builder for each node (node.builder()...build(), plus withXyz setters).
  • equals/hashCode/toString and structural match(...) are generated.
  • Visitors: Visitor<R> (with return value), VoidVisitor, visitor with default methods,
    ArgVisitor<R, A>, CopyVisitor and CopyOnWriteVisitor (structural copy,
    identity-preserving copy-on-write).
  • The metamodel's implements clauses on abstract types are preserved (merged into the
    extends clause of the generated sealed interfaces, since interfaces cannot have
    implements); concrete classes derive their implemented interfaces from the metamodel
    hierarchy.

Plan

  • AST generation pipeline (metamodel → generated classes + visitors) working
  • Generated AST compiles (:key.ncore.java:classes, 0 errors)
  • Metamodel fixes: missing type declarations, sealed-hierarchy/orphan handling,
    faithfulness to the hand-written AST, dead code removal
  • Generator bug fixes reviewed and covered by tests
  • Add key.ncore.java to the CI test matrix in .github/workflows/tests.yml
  • Migrate de.uka.ilkd.key.java.ast usages onto the generated AST (follow-up, e.g. a
    key.core.java module feeding the same metamodel)
  • Code cleanup
  • Document the changes (incl. the metamodel format for contributors)
  • Final KaKeY discussion on remaining design aspects

Type of pull request

  • New feature (non-breaking change which adds functionality)
  • There are changes to the (Java) code
  • There are changes to the deployment/CI infrastructure (gradle, github, ...)

Ensuring quality

  • I made sure that introduced/changed code is well documented (javadoc and inline comments).
  • I made sure that new/changed end-user features are well documented (https://github.com/KeYProject/key-docs).
  • I have tested the feature as follows: ./gradlew :key.ncore.java:runGenerator and
    ./gradlew :key.ncore.java:classes succeed; generated code compiles with 0 errors.
  • I have checked that runtime performance has not deteriorated (no hot paths affected yet;
    the generated AST is not yet wired into the prover).

Additional information and contact(s)

This is a DRAFT; the branch is weigl/javaastgen. The metamodel is the single source of
truth and is also intended to feed the planned key.core.java module (generator + support
classes get copied there once the ncore variant is stable).

The contributions within this pull request are licensed under GPLv2 (only) for inclusion in KeY.

@wadoon wadoon added this to the v3.1.0 milestone Apr 19, 2026
@wadoon wadoon self-assigned this Apr 19, 2026
@wadoon wadoon added the Java Pull requests that update Java code label Apr 19, 2026
@wadoon wadoon changed the title Weigl/javaastgen Genration of Java AST classes Apr 19, 2026
@Drodt Drodt changed the title Genration of Java AST classes Generation of Java AST classes Apr 21, 2026
@wadoon
wadoon force-pushed the weigl/javaastgen branch from f0bfb60 to bf1f381 Compare May 2, 2026 00:31
@wadoon
wadoon force-pushed the weigl/javaastgen branch from bf1f381 to b6c5eb1 Compare May 10, 2026 01:02
@wadoon
wadoon force-pushed the weigl/javaastgen branch from 0742879 to be38311 Compare June 28, 2026 18:30
@wadoon
wadoon force-pushed the weigl/javaastgen branch 2 times, most recently from b0ba757 to b06d4d8 Compare July 18, 2026 19:29
@wadoon
wadoon force-pushed the weigl/javaastgen branch 5 times, most recently from 6ef3ac7 to 7ecc12e Compare August 20, 2026 15:00
Fix the java-ast metamodel and the generator so :key.ncore.java:classes
builds with zero errors (previously ~1250 unresolved-type/sealed/hierarchy
errors; only MethodReference.java:137 surfaced because javac aborted on a
syntax error).

Metamodel (src/adt/java-ast.java):
- declare missing types: JavaProgramElement, Operator, Name, Comment,
  AbstractIntegerLiteral, Branch (+Case/Default), Catch, JMLModifiers,
  ObserverFunction, JAbstractSortedOperator, JOperatorSV
- import de.uka.ilkd.key.java.ast.abstraction.Type
- make concrete-extends-concrete parents abstract so generator-emitted
  sealed interfaces stay valid: ClassDeclaration, MethodDeclaration,
  StatementBlock, VariableSpecification, VariableReference, FieldReference,
  ProgramElementName
- Statement/Expression extend ProgramElement so ProgramTransformer.body
  getters in children override covariantly
- strip duplicate FieldSpecification fields, fix ReattachLoopInvariant
  field typo, drop MethodReference list wildcard, remove dead
  Declaration/Expression methods and unused placeholder types
- wire marker interfaces faithfully via abstract classes only
  (Statement, MemberDeclaration, TypeReference*, Reference*, ProgramPrefix,
  IProgramVariable, IProgramMethod, ProgramConstruct, ...)

Generator (src/generator/...):
- PostSteps.sealing: drop SEALED for types without permitted subtypes
  (IGuard, LoopInitializer, IExecutionContext, ... become plain interfaces)
- NodeSteps.setPackage: merge declared implements into extends for
  converted interfaces (JavaParser prints invalid 'interface ... implements')
- NodeSteps.addBuilder: unwrap wildcard element types in list-append methods
- NodeSteps.addWiths: skip constant fields when building withXyz constructors
- NodeSteps.addHashCode: emit 'return 0;' for classes without non-@EQEX fields
- PostSteps visitors: correct generic accept helpers, generate only
  builder fields (skip @internal hashCode and leaf types), use
  ImmutableList.collector instead of removed RoList.collector

Support: MatchHelper gets a generic Object fallback overload so match()
type-checks fields of external types (Type, SchemaVariable, ...).

Regenerated code in src/generated/java/org/key_project/java/ast/.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Java Pull requests that update Java code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant