Skip to content

fix(ad): Enzyme rule on single-grid apply_bc! segfaults on Linux x86_64 #40

Closed

Description

@kylebeggs

A custom Enzyme rule on the single-grid apply_bc!(::AbstractArray, ::AbstractGrid) segfaulted on Linux x86_64 (Julia 1.10 threads-1/2 and 1.12 threads-1) while passing cleanly on macOS/aarch64 across every version tried.

[2125] signal (11.128): Segmentation fault
in expression starting at test/autodiff.jl:19
apply_bc! at src/Fields.jl:155 [inlined]
apply! at src/operators/divergence.jl:82 [inlined]
...

Likely cause, and the mitigation taken

The rule bodies allocated: the hit counters were a Dict{Symbol,Threads.Atomic{Int}} looked up by key inside augmented_primal, and the augmented-primal helper took a closure. Rule bodies are compiled into Enzyme's generated code, so a dynamic lookup or GC allocation there is in exactly the wrong place.

Two changes landed together, so the cause is not isolated:

  1. the counters became plain const atomics reached by static field access, and the closures went away;
  2. the single-grid apply_bc!/fold_bc! rules were dropped entirely.

(2) alone would have fixed the crash, since it removes the offending rule. So we have a working tree but not a confirmed root cause.

Dropping (2) was independently justified: single-grid apply_bc! is 2·N·halo slab broadcasts that Enzyme already tapes correctly and cheaply, so a rule there was risk without measurable gain. The descriptor sweeps are where taping actually hurts, and those kept their rules.

Worth doing

  • Reproduce on Linux x86_64 with an allocating rule body to confirm allocation is the trigger, and file an MWE upstream if so — "rule bodies must not allocate" is not documented anywhere I could find, and it fails as a segfault rather than an error.
  • If the cause is confirmed and fixed, reinstating the single-grid rule is still optional; measure before bothering.

Note the platform asymmetry is the same one that makes #26 hard to work on: none of this reproduces on macOS/aarch64.

🤖 Filed by Claude, who would like the record to show it crashed the CI honestly and in public.

Activity

  1. added theissue type on Sep 9, 2026
  2. kylebeggs commented on Sep 30, 2026

    @kylebeggs
    CollaboratorAuthor

    Closing as completed. 9fc1b02 removed the single-grid apply_bc! / fold_bc! Enzyme rules. ext/MatrixFreeOperatorsEnzymeCoreExt.jl now documents that the single-grid path deliberately has no rule: Enzyme tapes its 2·N·halo slab broadcasts cheaply. The rules that remain sit on the raw-storage seams (_exchange_storage! / _bc_storage!), build no closures, and do not allocate. That constraint is recorded in DESIGN §6 and CLAUDE.md, and the ubuntu-latest CI legs are green.

    The root cause was never isolated. The two candidates are allocation in the rule body and the removed rule's mixed-pointer/isbits grid argument (same class as EnzymeAD/Enzyme.jl#2707). That note now lives in #82 in case a single-grid rule is ever wanted again.

    —
    🤖 Claude closing the loop: the segfault didn't get fixed so much as evicted.
    claude-opus-5-5[1m] (high) · claude-code 2.1.285 · unreviewed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions