Skip to content

Fix eigen/eigvals rules for Hermitian StridedMatrix - #853

Open
devmotion wants to merge 9 commits into
mainfrom
dmw/eigen-complex-hermitian
Open

devmotion wants to merge 9 commits into
mainfrom
dmw/eigen-complex-hermitian

Conversation

@devmotion

@devmotion devmotion commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

At a Hermitian A, the eigen/eigvals frules and rrules for StridedMatrix delegated to the rules for Hermitian(A). Their (co)tangents are those of x -> eigen(Hermitian(x)), which reads only the upper triangle of x. That is wrong for unstructured matrices: the rrules returned upper-triangular cotangents and the frules ignored the lower triangle of ΔA (FluxML/Zygote.jl#1369).

This PR:

  • Eigenvalues: tangents uᵢ' ΔA uᵢ and cotangents U Diagonal(Δλ) U', with the type of the primal eigenvalues (so they are only projected onto the reals when the primal eigenvalues are real-typed).
  • Eigenvectors: at Hermitian A, eigen uses the Hermitian solver, whose eigenvectors follow a different phase convention than those of non-Hermitian matrices. They are therefore only differentiable along Hermitian directions, so the rules use the Hermitian part of V' ΔA V for the eigenvector (co)tangents (and the Hermitian phase convention).
  • Both cases now share the code path of the generic rules (V' instead of V \ since V is unitary). The special handling of sortby is no longer needed: the rules call eigen!(A; kwargs...) itself, which sorts exactly as the primal does, and the (co)tangents are computed per eigenpair from the sorted result. _sorteig!_fwd became unused and is removed.
  • Fixes the failures on Julia 1.14 (pre): LinearAlgebra.jl#1717 made eigenvalues of complex matrices complex-typed also on the Hermitian path, so the old rules returned real tangents for complex primals. It also exposed a Core.Box in the eigen pullback (self-recursive thunk method), which made the eigvals rrule non-inferable; the pullback is now a single method.
  • Tests compare against finite differences of the unwrapped eigvals (whenever the eigenvalue type admits all directions) and of eigen ∘ hermitianpart for eigenvectors, with non-Hermitian ΔA, instead of x -> eigen(Matrix(Hermitian(x))). Includes a regression test for Zygote#1369 (∇ sum(eigvals(A)) == I), tests that the (co)tangents follow the order of the eigenpairs for a custom sortby, and that the imaginary parts of cotangents of real-typed eigenvalues are ignored.

Supersedes #841 (thanks @CarloLucibello for the analysis there), which fixed the values-only rrule for real matrices.

CI passes on Julia lts, 1 (incl. x86) and pre, as well as the Zygote and Yota integration tests.

🤖 Generated with Claude Code

At Hermitian `A`, the `eigen`/`eigvals` rules for `StridedMatrix` delegated
to the rules for `Hermitian(A)`. The resulting (co)tangents were those of
`x -> eigen(Hermitian(x))`, which only reads the upper triangle, so they
were wrong for unstructured matrices (FluxML/Zygote.jl#1369).

Now the eigenvalue (co)tangents are `uᵢ' ΔA uᵢ` and `U Diagonal(Δλ) U'`, with
the type of the primal eigenvalues. The eigenvectors are only differentiable
along Hermitian directions (phase convention of the Hermitian solver), so
their (co)tangents use the Hermitian part.

This also fixes the failures on Julia 1.14, where LinearAlgebra.jl#1717 made
eigenvalues of complex matrices complex also for Hermitian matrices, and
removes a `Core.Box` in the `eigen` pullback that broke inference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.64%. Comparing base (e996d60) to head (550778b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #853      +/-   ##
==========================================
+ Coverage   96.57%   96.64%   +0.07%     
==========================================
  Files          28       28              
  Lines        3616     3603      -13     
==========================================
- Hits         3492     3482      -10     
+ Misses        124      121       -3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

devmotion and others added 2 commits October 23, 2026 15:43
Reuse the cotangent buffer for the product with the eigenvectors in the
`eigen` pullback for Hermitian matrices.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Comment thread src/rulesets/LinearAlgebra/factorization.jl Outdated
Cotangents of real-typed eigenvalues are projected onto the reals again, as
in the previous Hermitian code path. Adds tests that (co)tangents follow the
order of the eigenpairs for custom `sortby`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the needs version bump Version needs to be incremented or set to -DEV in Project.toml label Oct 25, 2026
@github-actions github-actions Bot removed the needs version bump Version needs to be incremented or set to -DEV in Project.toml label Oct 25, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant