Skip to content

Relax the dependency graph in Nassau's compute_through_stem - #272

Open
JoeyBF wants to merge 3 commits into
SpectralSequences:masterfrom
JoeyBF:claude/nassau-relaxed-dependency-graph-d69k0s
Open

Relax the dependency graph in Nassau's compute_through_stem#272
JoeyBF wants to merge 3 commits into
SpectralSequences:masterfrom
JoeyBF:claude/nassau-relaxed-dependency-graph-d69k0s

Conversation

@JoeyBF

@JoeyBF JoeyBF commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Implements the relaxation flagged by the long-standing comment in nassau.rs: computing (s, t) only needs (s, t-1) and (s-1, t-1), not (s-1, t) and (s, t-1). The comment noted that "having the dimensions of the modules change halfway through the computation is annoying to do correctly" — this handles that.

What

  • compute_through_stem now uses the relaxed wavefront (s, t) <- (s, t-1), (s-1, t-1) for s >= 2, keeping many t-diagonals (n = t - s fixed) in flight at once. Rows s = 0, 1 stay strict — they're cheap, and step0/step1 read their targets through full matrices.
  • step_resolution_with_subalgebra reads its target C_{s-1} using only generators of degree < t (and C_{s-2} of degree < t-1). By minimality the differentials we lift land in the radical, so the degree-t generators — which (s-1, t) may be adding concurrently — contribute neither to the kernel we quotient by nor to those differentials. The read is thus race-free against those append-only writes, with no locking. New helpers: a generator-degree bound on signature_mask, plus restricted_dimension and restricted_partial_matrix.
  • Differentials are stored in the restricted (radical) basis, and the quasi-inverse save files always take the existing "incomplete information" (Magic::Fix) path, which the secondary machinery already handles.
  • ModuleHomomorphism::apply_to_basis_element now allows a result buffer shorter than the full target dimension (a prefix of the target basis); the matching truncated differential means act never writes past it. This is the only change outside nassau.rs.

Notes

The relaxed graph shortens the critical path to (S, T) from ~S + T to ~T: a whole internal-degree column can compute in parallel once the previous one is committed. Nassau can afford this because each bidegree already computes the kernel it consumes locally (it reduces the incoming differential d_{s-1}), so relaxing shares nothing across bidegrees and duplicates no work.

Remark: the classical resolution (resolution.rs)

The same relaxation is mathematically valid for MuResolutionker(d_{s-1})_t is a column-(t-1) object there too (see the existing get_kernel comment) — but it is not a free win. The classical algorithm reduces the outgoing differential at each bidegree and gets ker(d_{s-1})_t as the byproduct of the single row reduction it does at (s-1, t) to place that row's generators, handing it forward through the kernel cache. That reuse is exactly the (s-1, t) -> (s, t) edge. Relaxing means decoupling "find kernel" (column-(t-1)) from "place generators" (which needs (s-2, t)) and scheduling/caching the kernel independently — the get_kernel stem-edge path is already a special case of this. Done carefully it costs no extra reductions, only holding the reduced matrix live a little longer (memory) plus the scheduling refactor; done naively (cache only the Subspace, recompute the matrix) it doubles the expensive reductions. Left out of this PR since p=2 workloads use Nassau in practice, but noted here for the record.

Test plan

All with --features concurrent (the parallel path; maybe_rayon runs sequentially otherwise):

  • cargo test -p ext --lib nassau::test_restart_stem, plus a new test_stem_concurrent_secondary (save-backed d2 cross-check against the standard resolution, exercising the Magic::Fix quasi-inverse path)
  • cargo test -p ext --test milnor_vs_nassau — existing compare, plus a new wide compare_stem cross-check against the standard resolution (S_2, C2 to n=40/s=30; Joker to n=30/s=20), run 12× at RAYON_NUM_THREADS=16 for determinism
  • full cargo test -p ext — green except the pre-existing save_load_resolution::test_tempdir_lock (unrelated: it relies on read-only directory bits that root bypasses)
  • builds and tests pass in both the default and concurrent feature configurations

🤖 Generated with Claude Code

https://claude.ai/code/session_01MCUtWj6P6suSZqvATCdg6d


Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Added support for calculating partial basis-action results and matrices using caller-defined target dimensions.
    • Added dimension queries that account only for generators below a specified degree, improving consistency during ongoing computations.
    • Added bounded generator-degree filtering to Nassau resolution calculations.
  • Bug Fixes

    • Improved Nassau computation scheduling and preservation of restricted calculation behavior.
  • Tests

    • Added cross-checks comparing Nassau and standard resolutions across multiple degrees and boundary cases.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 7690b9de-6886-4e72-8b5d-c82a42eea843

📥 Commits

Reviewing files that changed from the base of the PR and between 80d614f and 43dc7e9.

📒 Files selected for processing (3)
  • ext/crates/algebra/src/module/free_module.rs
  • ext/crates/algebra/src/module/homomorphism/free_module_homomorphism.rs
  • ext/src/nassau.rs

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The change adds bounded free-module dimension and homomorphism operations, applies them throughout Nassau resolution construction, relaxes concurrent stem scheduling, and adds comparisons against standard-resolution results.

Changes

Bounded basis and matrix contracts

Layer / File(s) Summary
Bounded basis and matrix contracts
ext/crates/algebra/src/module/free_module.rs, ext/crates/algebra/src/module/homomorphism/free_module_homomorphism.rs
Adds bounded generator-dimension queries, restricted basis-element application, and partial matrices with caller-provided target dimensions. Tests cover strict generator bounds and computed offsets.

Restricted resolution computation

Layer / File(s) Summary
Restricted resolution step and persistence
ext/src/nassau.rs
Adds bounded signature masks and matrices. Nassau steps use frozen target dimensions for differentials, correction terms, and quasi-inverse persistence. Saved quasi-inverses use Magic::Fix.

Concurrent stem scheduling

Layer / File(s) Summary
Relaxed concurrent stem scheduling
ext/src/nassau.rs
Updates readiness, wavefront seeding, and successor spawning for same-row and diagonal dependencies.

Resolution validation

Layer / File(s) Summary
Resolution equivalence validation
ext/tests/milnor_vs_nassau.rs, ext/src/nassau.rs
Adds parameterized stem comparisons and a save-backed concurrent-stem test that compares Nassau and standard-resolution results.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: ⚪ Minimal · up to 43dc7

This change enables more concurrent Nassau stem computation while using bounded resolution data; current validation compares its results with the standard resolution, and no merge-blocking risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant compute_through_stem
  participant step_resolution_with_subalgebra
  participant signature_matrix
  participant get_partial_matrix_restricted
  participant resolution_state
  compute_through_stem->>resolution_state: identify ready stem position
  compute_through_stem->>step_resolution_with_subalgebra: compute Nassau step
  step_resolution_with_subalgebra->>signature_matrix: build bounded target matrix
  step_resolution_with_subalgebra->>get_partial_matrix_restricted: build restricted differential
  get_partial_matrix_restricted-->>step_resolution_with_subalgebra: return fixed-dimension matrix
  step_resolution_with_subalgebra-->>resolution_state: save step and quasi-inverse
Loading

Poem

A rabbit bounds the basis bright
Restricted columns fit just right
Nassau hops through stems in rows
Fixed inverses guide the flows
Two resolutions meet and compare
Carrot-tested charts align with care

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 72.73% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 22 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: relaxing the dependency graph in Nassau's compute_through_stem.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ext/crates/algebra/src/module/homomorphism/free_module_homomorphism.rs`:
- Around line 65-71: Enforce the truncated result bound in the homomorphism
action method containing the result-length assertion: ensure both
output_on_generator and target.act write only within result’s provided prefix,
using a bounded action API or a dedicated restricted method that validates the
invariant. Keep the generic full-dimension behavior unchanged and prevent
shorter buffers from causing out-of-bounds writes.

In `@ext/src/nassau.rs`:
- Around line 1052-1055: Update the row-1 readiness branch in the progress check
so the stem-boundary case `(1, max_n + 1)` does not bypass row 0. Compute the
required one-bidegree row-0 dependency halo before this check, then retain
strict readiness validation against the corresponding row-0 progress rather than
treating `t > max_n` as automatically satisfied.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6b01d781-3d1c-4c33-ab32-35cc51bf1ca3

📥 Commits

Reviewing files that changed from the base of the PR and between 4867b30 and 727a2c2.

📒 Files selected for processing (3)
  • ext/crates/algebra/src/module/homomorphism/free_module_homomorphism.rs
  • ext/src/nassau.rs
  • ext/tests/milnor_vs_nassau.rs

Comment thread ext/crates/algebra/src/module/homomorphism/free_module_homomorphism.rs Outdated
Comment thread ext/src/nassau.rs
JoeyBF added a commit to JoeyBF/sseq that referenced this pull request Jul 18, 2026
…Sequences#272's restricted graph

PR SpectralSequences#272's restricted_partial_matrix (degree-bounded columns) had displaced
PR SpectralSequences#271's GPU partial-matrix offload and cross-signature row-reuse. Add a
restricted GPU variant (get_partial_matrix_restricted[_verified]) that keeps the
batched Milnor multiply but sizes output to the frozen prefix and masks bits
beyond it, dispatch to it from restricted_partial_matrix_maybe_gpu, and restore
row-reuse (one full restricted matrix per bidegree, select_rows per signature).

Verified GPU==CPU on H200 NVL (CUDA 12.4): nassau_gpu (14230 rows / 527
bidegrees) and milnor_vs_nassau (9 tests) pass with NASSAU_GPU_VERIFY=1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JoeyBF added a commit to JoeyBF/sseq that referenced this pull request Jul 18, 2026
Under SpectralSequences#272's relaxed dependency graph, many more step jobs are ready at once;
each checked is_in_parallel() and re-queued itself (send_retry) whenever a
guarded rayon region was active, busy-retry-storming the CPU. The heavy nassau
matrix builds are now GPU-offloaded, so the guard's priority-inversion
avoidance no longer pays for itself. The mechanism is a pure scheduling
optimization with no effect on computed results, so removal is safe.

Deletes utils::parallel (ParallelGuard/is_in_parallel/PARALLEL_DEPTH) and the
retry plumbing in both the nassau and generic resolution schedulers.

Verified GPU==CPU still holds on H200 (milnor_vs_nassau 9/9; nassau_gpu 14230
rows/527 bidegrees) with NASSAU_GPU_VERIFY=1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ext/src/utils.rs (1)

598-662: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider a small unit test for thread-local independence.

This subtle invariant — a stolen job's PARALLEL_DEPTH is scoped to its worker thread, not global — has no direct test in this file; it's presently exercised only indirectly through Nassau's concurrent scheduling. A focused test (e.g. spawn two std::threads, hold a ParallelGuard on one, assert is_in_parallel() is false on the other) would cheaply pin down this behavior against future regressions (e.g. accidentally reverting to a shared counter).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ext/src/utils.rs` around lines 598 - 662, Add a focused unit test alongside
ParallelGuard and is_in_parallel that spawns two OS threads, holds a
ParallelGuard on one thread, and verifies is_in_parallel() remains false on the
other thread while the guard is active. Ensure the test synchronizes the threads
so the assertion specifically validates thread-local independence.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@ext/src/utils.rs`:
- Around line 598-662: Add a focused unit test alongside ParallelGuard and
is_in_parallel that spawns two OS threads, holds a ParallelGuard on one thread,
and verifies is_in_parallel() remains false on the other thread while the guard
is active. Ensure the test synchronizes the threads so the assertion
specifically validates thread-local independence.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 679a693a-61e7-4998-a8ba-af03976cc2b8

📥 Commits

Reviewing files that changed from the base of the PR and between 5beaf22 and 80d614f.

📒 Files selected for processing (2)
  • ext/src/nassau.rs
  • ext/src/utils.rs

JoeyBF added a commit to JoeyBF/sseq that referenced this pull request Jul 23, 2026
…tiply path

The zero-signature image matrix (d_s applied to the zero-sig source basis,
column-masked to the zero-sig target) was the last per-bidegree Milnor multiply
still on the CPU — a serial per-row apply_to_basis_element_restricted, ~17% of
frontier wall time in the perf profile. But it is the *same* restricted multiply
as the QI-source `full_matrix` already built via restricted_partial_matrix_maybe_gpu,
just on d_s = differentials[b.s()] instead of d_{s-1}, and its target
mask/dimension are exactly the `target_mask`/`target_dim` already computed for the
bidegree (d_s and d_{s-1} share the target module modules[b.s()-1]). So route it
through the same GPU-offloaded, work-gated, already-verified path and apply the
column mask on CPU; drop the serial `signature_matrix` method. (Reinstates the
"signature_matrix offload" win from the original nassau_gpu branch, lost in the
SpectralSequences#272 relaxed-graph merge.) row_reduce stays on CPU — the signature-masked matrices
are very flat (~100 x 100000), a poor RREF target for the GPU.

Correctness: GPU Ext chart byte-identical to CPU-only through (100,152);
NASSAU_GPU_VERIFY passes at stem 130.

This shrinks the serial tail that Amdahl-capped the GPU:CPU ratio, so the
arithmetic-intensity advantage finally shows through and the gap WIDENS with stem
(S_2, s<=152, 16-core H200 box, w=32):

  band       GPU    CPU    ratio
  130->140   159s   206s   1.30x
  140->150   278s   423s   1.52x
  cum 0->150 596s   771s   1.29x   (was 723s, a near-tie)

Memory stays bounded by the same byte-budget/block machinery (this path reuses
multiply_batch_on_gpu). Next lever: the full-reuse-matrix readback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@JoeyBF
JoeyBF force-pushed the claude/nassau-relaxed-dependency-graph-d69k0s branch from 540b31f to 97f28f3 Compare August 29, 2026 20:59
JoeyBF and others added 3 commits September 6, 2026 01:43
compute_through_stem resolved bidegrees in a strict wavefront, so a
bidegree waited on every bidegree of the previous stem rather than on the
two it actually depends on. Track per-row progress instead and spawn each
bidegree as soon as its same-row and diagonal predecessors are done,
letting independent bidegrees resolve concurrently.

The scheduler runs on the calling thread via in_place_scope, so it never
holds a ParallelGuard and cannot read the per-thread is_in_parallel flag
to sense saturation. A job stolen onto a worker that is already a blocked
guard holder bounces back a retry; park those bidegrees and re-spawn them
on each completion or a short recv_timeout tick.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VuHvdkS1dUru8SfCnyEou8
Give FreeModuleHomomorphism a dedicated apply restricted to a subspace of
generators rather than overloading the general path, and cover the stem
edges in milnor_vs_nassau.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VuHvdkS1dUru8SfCnyEou8
Address the comment-style review of the relaxed dependency graph.

The minimality argument justifying the truncated reads was written out in
full in three places. Keep it where the truncation is actually chosen, in
step_resolution_with_subalgebra, and reduce signature_mask,
restricted_dimension, restricted_partial_matrix and compute_through_stem to
pointers at it.

restricted_partial_matrix claimed it did not read the concurrently growing
target dimension, which the length assert in
apply_to_basis_element_restricted in fact does. State what it does instead:
it takes the target dimension from the caller.

The doc on apply_to_basis_element_restricted linked "Nassau's algorithm" to
crate::module::homomorphism. There is no path from algebra to ext::nassau,
so that link could never resolve; say it in plain text.

Give that assert a message so an error-path test can name it, and assert the
d2 chart is non-empty in test_stem_concurrent_secondary, which otherwise
compares two empty strings if both guards stop matching. Verified against
the real computation: the test compares 14 differentials.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EULwqf8R7n2rgF24Vph4o9
@JoeyBF
JoeyBF force-pushed the claude/nassau-relaxed-dependency-graph-d69k0s branch from 97f28f3 to 43dc7e9 Compare September 6, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant