Skip to content

Eager Constraint Solving in Polymorphic Flow Analysis - #605

Closed
flandia wants to merge 52 commits into
hkust-taco:hkmc2from
flandia:better-flow
Closed

flandia wants to merge 52 commits into
hkust-taco:hkmc2from
flandia:better-flow

Conversation

@flandia

@flandia flandia commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor
  • Distinguish initializations from reassignments (but it still needs analysis to be cross-block to be useful).
  • Clean the Flow Analysis file for the following improvements.
  • Store bounds in schemes.

LPTK and others added 30 commits October 15, 2026 14:26
Preserve delayed erasure for constructor and placeholder symbols alongside private constructor storage and placeholder source locations. Keep exhaustive loops non-terminating in the ScopedBlocks expectation, and add erasure regressions for private construction, private shadowing, implicit lambdas, and trailing arguments.

Regenerate golden outputs. Validated with Metals compilation, ctest, focused diff tests, and hkmc2AllTests/test.
- Type generator calls as iterators: `@generator` functions return an
  `Iterator`, both in resolution (`listenDefinitionResult`) and in erasure
  (`callableResult`), instead of the payload of their final step.
- Check overrides against the representations of the members they override:
  unannotated override parameters take the inherited representations, override
  results must fit the inherited result, and declared parameters must admit the
  inherited ones (and match them on Wasm, which calls overrides through the
  signature of the overridden method).
- Require type annotations on open members whose parameters are exposed to
  other compilation units, since subclasses there must implement them with the
  same representations. Annotate such members in the library and the tests.
- WORKAROUND: approximate the results of builtin operators from the primitive
  classes of their operands (`BuiltinOperatorResults`), in resolution and in IR
  erasure, until the builtins are declared as overloaded functions. This only
  applies to the JavaScript target, as the Wasm backend implements operators
  with dynamically typed intrinsics.
- Locate representation-conversion diagnostics (`Lowering.castTo` and
  `WatBuilder.castConserve`).
- Document runtime representations in `doc/reference.md`, including
  `undefined` as undefined behavior and the rejection of programs whose only
  problem is a misleading annotation.
- Add regression tests for the inference of representations across worksheet
  blocks, with a :fixme for the Wasm REPL crash when reading a field whose
  representation is another session class.
- Clear `WasmScratch.mls`, moving its unique tests to `BuiltinOperators.mls`,
  `Casts.mls`, and `ReplImports.mls`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The internal errors that `newres/Overrides.mls` records cite their location
in NewResolver.scala, which moved when comments were added there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A backtracking assignment `set x = v in body` saves `x` in an `old` temporary
and restores it in a `finally` block. The temporary had no representation,
so saving and restoring it converted from and back to the representation of
`x`. This was a checked cast that dead-code elimination must keep (on JS),
and an error for an unboxed `Int32` variable (on Wasm).

Erasure now treats the temporaries that elaboration declares with `LetDecl`
(currently only this one) as slots like source variables, inferring their
representations from the values flowing into them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Route assignments to fields (`set o.f = v`, including compound and
  temporary assignments) to the fields they assign: resolve the assigned
  member as selections do, for every class that the receiver can be an
  instance of (including through projections, wildcard opens, annotated
  receivers, and captures), and record the assigned values in each field's
  host (`NewResolver.assignField` and `writeField`).
- Infer the erased type of an unannotated mutable field from its
  initializer and its assignments, replacing the blanket unknown input.
  Record fields are included: a mutable copy `mut {...r}` assigns the
  fields of `r`.
- Make reads of unannotated mutable fields observe their assignments.
  A read used to see only the initializer, so a member could be resolved
  statically in the initializer's class while an instance of another class
  had been assigned (a silent miscompilation on Wasm).
- Interface exposure assigns unknown values to the mutable fields (and
  mutable record fields) that a compilation unit exposes, and
  `Erasure.checkExposedMutableField` requires exposed mutable fields to
  declare their types. Annotate such fields in the library, the apps, and
  the tests.
- Share the projection lookup of `newSel` (`projectedLookup`).
- Document mutable fields in Runtime Representations, and add tests, with
  :fixme regressions for assignments through structural types and for
  private classes used in later Wasm worksheet blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Record fields can be declared mutable with `mut`, in record literals
  (`{mut count: 0, label: "x"}`) and record types (`{mut item: T}`).
  Other fields are immutable, unless their whole record is (`mut {...}`).
  A record literal with a mutable field is allocated unfrozen; resolution
  assumes its other fields are never assigned, which type checking will
  ensure.
- Through a record type, the values it describes can be assigned values of
  the types of its mutable fields: `constrainRecord` assigns such values to
  the actual values' assignable fields, as `constrainFunction` feeds the
  declared parameter types to callbacks. Assignments through record types'
  immutable fields are assumed not to happen, so assignments through record
  types need not be followed. Record type views carry their mutable fields.
- Reads of mutable record fields return their initializer and the values
  assigned to them, instead of values of unknown shape.
- Interface exposure assigns unknown values to the mutable fields of exposed
  records.
- Share the resolution of assignable fields between assignments and record
  constraints (`assignableTargets`, `listenAssignedValues`).
- Update the reference and the tests: the structural-assignment :fixme is
  resolved, and the record :fixmes now only reflect that resolution is not
  flow-sensitive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diagnostics now cite definition headers, and internal-error line numbers
shifted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Integrate the lowered parameter-list and member-key APIs with inferred
representations, mutable-field bounds, flow cast proofs, and Wasm calls.
Coerce dynamically extracted optional/trailing slots and check statically
aligned trailing arguments against their destination representations.

Keep generator/async body payloads separate from inherited callable result
wrappers, and handle literal tests when deforestation fuses through a proved
cast. Add regression coverage for both cases.

Audit branch-added tests, consolidate overlapping optional and mutable-field
cases, and move AI-authored diff files into slop while preserving strict,
loose, JS, and Wasm configurations. Record overwritten-field exposure and
trailing-parameter override gaps as explicit known-failure regressions.
Regenerate and include all affected golden output, adapting older cast
fixtures to the new default resolution mode with explicit Any interfaces.

Validation: ctest; focused erasure and slop tests; hkmc2AllTests/test
(1,128 tests passed).
Respect overwritten record fields and distinguish exposing a frozen spread copy
from admitting external writes into its source. Inherit and check rest/trailing
parameter representations when overriding layouts correspond.

Select fused match arms using ordered subtype tests, specialize default closures
by constructor layout, and require proofs that elided construction is harmless
and projected fields retain their original argument values. Preserve effectful
constructors, superclass effects, qualifiers, and lifted argument layouts.

Consolidate regression cases in slop and include regenerated golden outputs.
Record incompatible override-layout behavior pending a language policy decision.
Remove implicit default overloads from the touched compiler APIs.

Validation: ctest (56 tests), hkmc2AllTests/test (1,128 tests), git diff --check.
Resolve the golden-output conflicts and retain OptionalBinding instead of
restoring the duplicated OptionalNull suite. Adapt consolidated regression
tests to the new strictres configuration, preserving strict and eager
resolution for the Wasm mutable-field tests as well.

Include regenerated golden outputs for upstream rest-parameter inference
and changed diagnostic locations.

Validation: ctest (56 tests), sdtest (31 tests), and hkmc2AllTests/test
(1,131 tests) all pass.
Reimplement the erased-type hierarchy and infrastructure changes from
hkust-taco#560 without introducing rsc/rsc? syntax or semantics.

Separate ErasedFuncSignature from ErasedValueType, sharing only memoized
canonicalization. Replace HasErasedType and broad symbol conversions with
concrete accessors, and preserve delayed initialization checks. Keep
function signatures when copying symbols and adapt specialized signatures
to eliminated parameters.

Update lowering, captures, printing, tail-call rewriting, and Wasm consumers
to distinguish runtime value types from callable signatures. Cover nested
symbol copying and typed Wasm specialization with diff tests, and include
all updated golden outputs.

Validation: hkmc2AllTests/test
Separate local slots and receivers from neighbouring runtime definitions, and
carry those types through scope analysis and lifted parameters. Narrow class
body symbols so synthetic top-level receivers cannot enter these sets.

Build captured paths from the typed capture map, sharing the logic between
class pre-constructors and bodies. Remove the impossible erasure fallback
cases and the now-unreachable top-level symbol refresh paths.

Validation: ctest; lifter diff tests; erased-signature regressions;
hkmc2AllTests/test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
codex and others added 9 commits October 29, 2026 15:05
Preserve erased representations alongside effect summaries, mutability, receiver handling, and statement lowering. Keep effect variables separate from runtime value flows during deforestation and scheme instantiation.

Adapt the exported mutable-field fixture, add an effect/value-flow regression, and refresh golden outputs. Validated with Metals, focused diff tests, and hkmc2AllTests/test.
Merge local PR hkust-taco#592 head 49cb4b9.
Keep inferred value representations separate from definition calling
signatures across erasure, cloning, lowering, and optimizer rewrites.
Preserve operand-dependent builtin results alongside the arity-specific
builtin signatures, as well as undefined and physical return handling.

Unify signature updates for dead-parameter elimination, eta expansion,
and deforestation, retaining both copied and in-place representations.
Include regenerated golden outputs and both incoming regression tests.

Validation: Metals compilation; ctest; cwtest; focused erasure, signature,
effect-flow and eta tests; wdtest WasmScratch; hkmc2AllTests/test
(1,165 tests passed). Reviewed both-parent diffs and full-branch spacing.
Adopt the unified term-erasure initialization API while retaining shape inference, constructor cloning, and body-result erasures. Include the cloned local-class signature regression and its validated golden output.

Validation: Metals compilation and hkmc2AllTests/test (1,165 tests passed). Reviewed the merge diff and whole-PR whitespace changes, preserving intentional contributor formatting.
PR hkust-taco#592 was squash-merged as aa2df37, whose tree is exactly identical to 254b1bf already merged into this branch. Record the squash commit as an ancestor while preserving the existing conflict resolutions and shape-based erasure changes.

Verified identical upstream trees, an unchanged index and worktree, and the original refactor head in our ancestry. This commit has exactly the previously validated tree: Metals compilation and all 1,165 tests passed at 8bfffeb.
Port the assignment-kind distinction from 4b8cafb
on simpler-spreads-in-ir to the current erased-type IR.

Use explicit Assign.init and Assign.reassign constructors. Mark source mutations,
tail-call parameter updates, and handler state transitions as reassignments, and
preserve the flag through assignment rewrites, including lambda rewriting,
lifting, deforestation, reflection, effect checks, and Wasm tail reconstruction.

Add diff tests for assignment kinds through lowering and optimization, and
refresh the affected IR tree snapshots.

Validation: ctest; sdtest AssignReassign; hkmc2AllTests/test.
@flandia

flandia commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor Author

40ca5f4 changed the way of collecting and solving strategies and constraints so the order and debug ID of test outputs also changed.

@flandia

flandia commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor Author

Some benchmark results (performance gain after solving things early)

Input Mono Poly
Option 1.02× 0.86×
Iter 1.04× 1.31×
FingerTreeList 1.20× 1.95×

Update: some refactor worsen the asymptotic complexity of constraint solving and I'm looking into it...

@LPTK

LPTK commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Some benchmark results (performance gain after solving things early)

What are these numbers measuring?

@flandia

flandia commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

What are these numbers measuring?

The time used by flow analysis (pre-analyze + collect + solve)

@LPTK

LPTK commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Do you mean that this PR makes the polymorphic analysis of FingerTreeList twice as slow?

@flandia

flandia commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Do you mean that this PR makes the polymorphic analysis of FingerTreeList twice as slow?

I said performance gain so it's twice as fast

@flandia

flandia commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

LP has spoken; thus we shall make this a separate PR

@flandia flandia changed the title Better Flow Analysis Eager Constraint Solving Nov 5, 2026
@flandia flandia changed the title Eager Constraint Solving Eager Constraint Solving in Polymorphic Flow Analysis Nov 5, 2026
@flandia

flandia commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

I'm still trying to optimize the constraint solving order but the current one is good to review

@flandia
flandia marked this pull request as ready for review November 6, 2026 03:59
@flandia

flandia commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Closed in favor of #621

@flandia flandia closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants