Repository navigation
Preserve override argument bindings and runtime representations - #602
Conversation
Overrides can change which arguments reach their parameters or lose representations declared by a base method. A valid call through the base can then fail at runtime, and direct and virtual Wasm calls can disagree about their signature. Validate the source override contract before inheriting representations, then check Wasm's additional calling-convention restrictions after lowering. Unannotated override parameters and results inherit the base member's representations. In this example, `Child.take` keeps `Int32` for both its input and result, and the direct and virtual calls together return `42` on Wasm: ```mlscript abstract class Base() with fun take(value: Int32): Int32 class Child() extends Base() with override fun take(value) = value fun dispatch(receiver: Base, value: Int32): Int32 = receiver.take(value) wasm.i32.add(dispatch(Child(), wasm.i32.const(40)), Child().take(wasm.i32.const(2))) ``` Explicit annotations must accept the inherited input representations and produce results that fit the inherited result representation. JavaScript permits wider accepted inputs. Wasm also checks the reverse direction: widening a base parameter from `Int` to `Any` is rejected because direct calls of the override still use the base method's physical slot. Method bodies and returns use that shared physical signature, including abstract declarations and narrowed reference results. Source layout checks reject changes that redistribute inherited arguments, even for unused or abstract members: ```mlscript private class RestBase() with open fun collect(...values: Array[Any]): Any = values private class StolenRest() extends RestBase() with override fun collect(first?: Any, ...values: Array[Any]): Any = values ``` For a call such as `collect(1, 2)`, the added `first` parameter would change the inherited rest binding from `[1, 2]` to `[2]`. The compiler now reports that the override must preserve the arguments bound to inherited parameter `values`. Renaming parameters and adding an optional suffix remain valid when every permitted base call preserves its bindings. An inherited `drop` parameter may be omitted only when the remaining bindings stay intact. The implementation also: - Checks optional, rest, trailing, curried, and contextual parameter layouts, including declared callable aliases and imported declarations. Unsupported callable layouts receive a diagnostic. - Preserves omitted arguments through ordinary call lowering so each callee binds its own defaults. - Updates finger-tree and lazy-collection overrides to honor their inherited variadic `slice` and `concat` contracts. Regressions cover omitted bounds and zero or multiple concatenation arguments through direct and base-class calls. - Documents the override contract and includes the updated inline golden outputs. This combines E14 from hkust-taco#575 with the override-specific work from `LPTK/proper-overriding-checks`. The Scala diff against `hkmc2` is **+519 / -56 lines**. Validation: Metals compilation, focused override and optional-argument regressions, and **`hkmc2AllTests/test`: all 1,170 tests passed**. The full pass used `set Global / concurrentRestrictions += Tags.limit(Tags.Test, 1)`, matching CI's test-task serialization; the initial parallel run hit the existing timeout on `FingerTreeList_StressTest`.
Integrate hkmc2 through 7baee98, including the generic IterableBase interfaces and public LazyIterable contract. Resolve the four collection conflicts using the upstream implementations. For example, slice(start?, end?) now follows the base method's optional bounds, and finger-tree concat(...xs) folds over every supplied sequence. Keep the override regressions against those implementations and update the slice-default comment to describe the current contract. Validation: Metals compilation and hkmc2AllTests/test, with all 1,189 tests passing and test tasks serialized as in CI. The test run produced no new golden-output changes.
Explain the purpose, inputs, results, and phase requirements of OverrideLayout, OverrideChecks, and TargetOverrideChecks, including their parameter correspondence and representation rules. Document the erasure helpers and the Wasm parameter and return conventions used by function generation. Examples cover optional and rest bindings, declared callable arrows, and why (Int) -> Any can be overridden by (Any) -> Int at source level while Wasm rejects the wider input. Documentation only. Verified that all six changed files retain identical executable code, indentation, and blank-line whitespace. Metals compilation passed.
Run the erasure phases in the constructor and expose the validated override edges as an immutable value. Remove the separate run method and the mutable completion and edge fields, retaining the source-root identity check used by lowering. Pass validated edges explicitly to signature computation. For example, an unannotated override still reads its inherited result only after its base has completed erasure; that dependency no longer uses a mutable field. Use one ordered identity set for statement traversal and derive the rest parameter set from the collected declarations. Freeze traversal results before subsequent phases. Preserve visitation and parameter order, and retain local graph caches that prevent cycles and repeated traversal of shared ancestors and type interpretations. Validation: Metals compilation and hkmc2AllTests/test passed. No golden outputs changed. Reviewed the whole branch diff for accidental whitespace changes.
An imported legacy getter whose type is a non-callable alias, such as `declare fun name: Name` with `type Name = Str`, is rejected when it is overridden: the legacy arrow fallback treats every alias reference at a declaration's head as a potentially callable type. Record this as a :fixme in OverrideLayouts.mls. Its golden output also shows the error naming the overriding member while pointing at the inherited declaration. Restore Wasm super-call coverage for unannotated base methods and getters in newres/wasm/Super.mls, next to the annotated variants that replaced the original blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 0.2.x legacy resolver will be retired soon. Agents should keep its existing tests passing but not spend effort fixing, extending, or testing behavior that only the legacy resolver exhibits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LPTK
left a comment
There was a problem hiding this comment.
[Claude Opus 5.5] Review of the override binding and representation checks; details are inline.
Two issues are deliberately left out: the Wasm rejection of overrides that make a required boxed parameter optional (intended, since Int will compile to a non-nullable slot) and an issue specific to legacy resolution (the legacy resolver is being sunset). Regression tests are in 5842e11.
| * Parameters that already fail the source representation check produce no additional error here. | ||
| * This method has no effect for other targets. | ||
| */ | ||
| def apply(edges: Ls[(TermSymbol, TermSymbol)])(using Config, Ctx, State, Raise): Unit = |
There was a problem hiding this comment.
[Claude Opus 5.5] Wasm override constraints are now enforced in three places that overlap: OverrideChecks (source representations), this widening check, and WatBuilder.checkOverrideSignature (primitive slot equality and arity). The same mismatch is reported twice, once in erased-type terms and once in Wasm-type terms. See the M2 and N2 blocks in wasm/VirtualMethods.mls, e.g. "Parameter 'm' … is represented as 'Int' …" followed by "Overriding method g declares parameter 1 as (ref null i31), but the method it overrides declares i32".
There was a problem hiding this comment.
[Claude Opus 5.5] Not resolved yet.
Any ErrorReport raised during WatBuilder's predeclaration phase breaks out of module generation. So checkOverrideSignature is also what stops body generation for a slot it cannot represent. When I suppressed its primitive checks for members that have a source override edge, codegen continued into the bodies of wasm/VirtualMethods.mls (M2) and crashed in castConserve (require(expr.resultTypes.size == 1)).
Resolving this needs a way for WatBuilder to stop generating a module without raising its own diagnostic, such as an aborted status on CompiledWasmModule that the diff harness honors. That is a backend API decision, so I left both diagnostics in place for now.
There was a problem hiding this comment.
For the record, the proper fix will be to treat those backend errors that refer to internal implemnetation details (such as WASM types) as internal errors, and make sure that they do not actually reach the backend (aborting compilation of that particular definition early on).
This will be done in a separate PR.
`writtenCallable` and `legacyCallableHead` only test whether a term is a function type, so match `_: Term.FunTy` rather than a three-underscore positional pattern, as AGENTS.md recommends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Erasure collected the traversed term definitions twice: once for `OverrideChecks`, which deduplicated them by symbol, and again for `eraseDefinition`. Compute the deduplicated list once and pass it to `OverrideChecks`, so both phases work on the same declarations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`TargetOverrideChecks` copied `within`, `describe`, and the "overridden member is defined here" report from `OverrideChecks`. Move them to the `OverrideChecks` companion so the source and Wasm checks use one compatibility predicate and one diagnostic shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`OverrideChecks` recomputed `OverrideLayout.signature` for the inherited declaration of every overriding edge, and erasure recomputed it again for declared members. A base with N overrides had its callable type, including aliases and type-argument bindings, traversed N+1 times. Cache layouts per declaration identity inside `OverrideChecks`, and let `erasedDefinitionSignature` read declared layouts from the same cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Erasure looked up the inherited result of every unannotated definition by scanning the whole list of validated override edges, which costs definitions x edges per program. Group the edges by overriding member once in `OverrideChecks.validatedBases` and look them up directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`Inheritance.validateMember` already rejects an override that turns a field, getter, or method into another kind of member. `OverrideChecks` then compared the pair's parameter lists anyway, so `override fun foo = 1` over `fun foo() = 1` also reported "must preserve the number of parameter lists". Extract the kind test into `Inheritance.changesMemberKind` and skip such pairs in `OverrideChecks`: they have no corresponding parameters, so they yield neither a second error nor a validated edge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the inherited declaration's callable type had no determinable layout, `OverrideChecks` reported "Overriding member 'pick' has a callable type whose parameter layout cannot be determined", with the primary location inside the inherited declaration, often in another file. The override was blamed for its base's type. Distinguish the two sides. A failure in the inherited layout is now reported at the override as "cannot be checked against the member it overrides, which has ...", and the note points at the overridden member. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`Return` cast every returned value to the function's registered result type with `castConserve`, which emits a checked `ref.cast` whenever the value's reference type is a supertype of that result. This applied to every function. Lowering already inserts every representable coercion, and override checking ensures that an override's results fit the slot it inherits, so such a downcast only hid a missing IR cast behind a runtime trap. `checkBodyRetType` stopped reporting these mismatches; for example, `wasm/TailRecOptCasts.mls` lost its `sccF` and `sccG` result errors. Restore the previous return emission, which restores the IR value's own erased type and leaves mismatches to `checkBodyRetType`. No test needed the downcast; overrides that narrow a reference result still return a subtype of their slot. Remove the `resultType` parameter that `FunctionCtx`, `genFuncBody`, and `setupFunction` carried only for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reference permits overrides to add optional parameters when they cannot consume inherited arguments, but the Wasm backend rejects any override that changes the number of parameters of the method that introduced its virtual slot, e.g. `override fun f(x, y?) = x` over `open fun f(x: Int): Int`. State that restriction next to the other Wasm calling-convention rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unannotated override took its result representation from the first erased base, even when that base declared no result and a later base did. A member can have several direct bases through a structural intersection (`MemberLookup.All`), so the override's result erased to `Unknown` and representation checking then reported a mismatch against the annotated base. Inherit from the first base that declares a result; representation checking still reports any other base whose result the override does not fit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Overrides inherit the representations of their base member's parameters and result. An unannotated exposed base such as `open fun f(x) = x + 1` therefore gives every override in another compilation unit unknown representations that erasure cannot recover. Report an error when an overridable member (`open`, or abstract) of a class that `InterfaceExposure` reaches from the unit's exported definitions lacks a parameter or result annotation. A separate full signature also satisfies the requirement. Open members of private or local classes, which only their own unit can override, and members of objects, which cannot be extended, need no annotations. The checked members are recorded in `NewResolverState`, so a worksheet block that re-exposes an earlier class does not report it again. Annotate the members this now rejects in the compilation-test library (`ObjectBuffer.buf`, `SealedClasses.Abstract.value`, and `FingerTreeList_StressTest`'s `toString`) and in strict-resolution diff tests. Make classes private where a test is about unannotated members, and add `slop/strictres/ExposedOpenSignatures.mls` and a reference paragraph for the rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The override-signature work annotated `SuperBase` in `newres/Overrides.mls` instead of adding an annotated variant, so no JavaScript test exercised a super call through an unannotated base. Add that case back next to the annotated block. Exposed open members must now be annotated, so its classes are private to the worksheet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exposed-open-member rule required annotations on every open member reachable from the unit's exports, including members of classes that other units cannot extend and overrides that inherit a typed signature. Restrict it to overridable members of `open` or `abstract` classes and of `open` objects, which may become extensible. `Inheritance.isExtensible` now states this condition for `validateParent` as well. Accept a parameter or result that an override inherits from a specified overridden signature: inherited signatures are reused unchanged, not refined, so they need not be restated. Overridden members of other compilation units count as specified, since their own unit enforces the rule on the members it lets others override. Inheritance needs the parameter correspondence that only erasure computes. `InterfaceExposure` therefore records the exposed members and their exposure paths in `NewResolverState`, and `OverrideChecks.checkExposedSignatures` reports unspecified parts during erasure. Revert the test annotations and private modifiers the broader rule needed for non-extensible classes, and update `ExposedOpenSignatures.mls` and the reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Private members are never overridden: a subclass member with the same name is a distinct member. Their `open` modifier therefore has no effect. Warn about it in `Inheritance.validateMember`, and mention the warning in the reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adding `checkExposedSignatures` stripped the two-space indentation from the blank lines around it, which separate members inside the class. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
[Claude Opus 5.5] About 02ce875, refined by 0b4321a and 9861609 Why. This PR makes unannotated overrides inherit their bases' parameter and result representations. When the base is itself unannotated, there is nothing to inherit. For example, an exposed The rule. The compiler reports an error for an overridable member (
A type is specified by any of:
Overridden members of other compilation units count as specified, because their own unit enforces the rule. open class Typed() with
open fun scale(x: Int, y: Int): Int = x * y
open class Inheriting() extends Typed() with
open override fun scale(x, y) = x + y // OK: inherits (Int, Int) -> Int
open class Extended() extends Typed() with
open override fun scale(x, y, factor?) = x * y // error: 'factor' has no type to inheritExempt:
Implementation. Fallout:
The rule is covered by Choices worth a second look:
|
Clarify that private and local classes receive the exposure-based exemption only while they remain unexposed. For example, an exported function can expose such a class by returning its constructor.
Use the same declaration layouts and inherited parameter correspondence for source representation checks and the additional Wasm restrictions. Check source compatibility first, then the reverse input compatibility required by direct Wasm calls. Remove TargetOverrideChecks and the override-edge lists passed through Erasure to lowering and prelude compilation. For example, (Int, Any?) -> Int can override (Int) -> Int on JavaScript but must be rejected for Wasm. Check this arity restriction, dropped parameters, and variadic signatures even when declarations have no IR bodies. Add regressions and commit their generated golden output. Validation: Metals compilation, ctest, cwtest, sdtest Override, and hkmc2AllTests/test (1191 tests passed).
`OverrideChecks.layout` cached layouts under `Identity[TermDefinition]` keys, which allocated a wrapper for every lookup. Key the cache by the member's `TermSymbol`. A handler member is elaborated again without its resumption parameter list under the same symbol, so one symbol can have two declarations. Nothing overrides a handler member, so only its traversed declaration should reach the cache. Each entry retains the declaration its layout was computed from, and a `softAssert` checks that later lookups pass the same one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`Inheritance.validateMember` rejected an override that turned a function without parameter lists into one with a parameter list as a change between a "getter" and a "method". Whether such a function compiles to a JavaScript getter is an implementation detail; in source terms the two functions differ in their number of parameter lists. Compare only the kind of definition (`fun`, `val`, or `mut val`) in `changesMemberKind`, and report a mismatch in source terms, e.g. "Overriding member 'value' is a function, but the member it overrides is a value". Parameter lists are compared by the override layout check, which reports "must preserve the number of parameter lists" for `override fun foo = 1` over `fun foo() = 1`. The external-declaration exemption only concerned the parameter-list comparison, which layouts now perform for declared callable types too. A written method can therefore override a declared member such as `declare fun pick: (Int) -> Int`, which has the same single parameter list. `slop/OverrideLayouts.mls` now covers both that case and the rejected parameterless override. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test for a written method overriding `declare fun pick: (Int) -> Int` used a declared class with no runtime definition, so it showed only that compilation accepts the override; its output was a runtime error. Define the declared base class at run time and call `pick` through it, which returns the overriding method's result (40, not the base's 5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A handler function whose only parameter list is the continuation, as in `fun lol(k) = k(())`, was rejected with "Handler function cannot be a getter". Whether a function compiles to a JavaScript getter is an implementation detail. Report "Handler function must have a parameter list before its continuation parameter list" instead, using the reference's term for `k`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An override can drop an inherited rest parameter, as with `(Int)` overriding `(Int, drop ...Array[Any])`. On Wasm the override was then reported as "cannot use a variadic signature", although it has no variadic list of its own. Report "overrides a variadic signature, which the Wasm target does not support" when only the inherited signature is variadic, and keep the previous message, located at the rest parameter, for overrides that declare one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A `declare`d `fun` without written parameter lists obtained parameter lists from the arrows in its type, so `declare fun pick: (Int) -> Int` had one list while an abstract `fun pick: (Int) -> Int` had none. Both forms declare a member holding a function value, and both support the explicit `fun pick(_: Int): Int` for a member with a parameter list. A member's parameter lists are now its written ones in both cases. `OverrideLayout` compares written `ParamList`s directly. This removes its recovery of lists from declared callable types: aliases, type argument bindings, curried results, and the legacy fallback for imported declarations. Their diagnostics go too, including "cannot be checked against the member it overrides". Erasure no longer derives physical lists from declared arrows. `OverrideChecks` no longer needs a layout cache, and parameter correspondence pairs plain `Param`s. The type-alias and spread signature registrations that supported this recovery in the elaborator and resolver are reverted. Calls to a parameterless `declare fun f: (A) -> B` now apply the function value it denotes, so their results are not erased to `B`; `codegen/ErasedTypes.mls` uses parameter-list declarations for its typed-result checks. The override tests use parameter-list declarations, and new blocks show that a method cannot override an arrow-typed declared or abstract member while a function-valued member can. The legacy compile-test module and the alias, callback, and generic-result layout tests are removed. Separate signatures followed by implementations, as in `fun f: (Int) -> Int` and `fun f(x) = x`, are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[GPT-6 Astra and Claude Opus 5.5, with some manual updates]
Previously: Overrides could change which arguments reach their parameters or lose representations declared by a base method. A valid call through the base could then fail at runtime, and direct and virtual Wasm calls could disagree about their signature.
In this PR: Validate the source override contracts and check Wasm's additional calling-convention restrictions.
Unannotated override parameters and results inherit the base member's representations. In this example,
Child.takekeepsInt32for both its input and result, and the direct and virtual calls together return42on Wasm:Explicit annotations must accept the inherited input representations and produce results that fit the inherited result representation. JavaScript permits wider accepted inputs. Wasm also checks the reverse direction: widening a base parameter from
InttoAnyis rejected because direct calls of the override still use the base method's physical slot. Method bodies use that shared physical signature for their parameter slots and declared result, including abstract declarations and narrowed reference results. Returned values are not downcast to the slot: a mismatch that lowering did not coerce remains a compilation error.Source layout checks reject changes that redistribute inherited arguments, even for unused or abstract members:
For a call such as
collect(1, 2), the addedfirstparameter would change the inherited rest binding from[1, 2]to[2]. The compiler now reports that the override must preserve the arguments bound to inherited parametervalues. Renaming parameters and adding an optional suffix remain valid when every permitted base call preserves its bindings (Wasm additionally requires an override to keep the introducing method's number of parameters). An inheriteddropparameter may be omitted only when the remaining bindings stay intact.Because overrides inherit these representations, an overridable member (
openor abstract) of anopenorabstractclass, or of anopenobject, must specify its parameter and result types when its compilation unit exposes it, that is, when the resolver's interface-exposure probes reach it from the unit's exports. A type can come from an annotation, a separate full signature, or the corresponding part of an overridden member's signature, which overrides inherit unchanged:Other members, including those of private and local classes, are overridden only within their own unit and need no annotations. Marking a private member
openproduces a warning, since private members are never overridden.The implementation also:
hkmc2, covering omittedslicebounds and zero or multiple concatenation arguments through direct and base-class calls.AGENTS.mdthat the legacy resolver is being sunset; legacy-only issues found in review are left unfixed.This combines E14 from #575 with the override-specific work from
LPTK/proper-overriding-checks. The Scala diff againsthkmc2is +903 / -124 lines (git diff --shortstat, documentation comments included).Known open item: the Wasm backend's override-slot check and the source override checks still report some mismatches twice. (See below.)