Repository navigation
Sync KohakuFA-CuTe forward tuning and launch-grid fixes - #3303
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sync the vendored CuTe implementation with merged upstream revision
1afb72698c74b3a1b5958d3b80948458c9bb6e84from KohakuFA-CuTe.This imports specialized Ada/Hopper causal GQA forward paths, split forward attention for short cross-attention queries, and flattened launch grids when batch × heads exceeds 65,535. Native extension build names now include the C++ source hash, preventing stale binaries when copied checkouts preserve modification times.
Port numerical, saved-statistics, cache/stride, graph replay, launch-boundary, and architecture-specific forward regressions. The comparison script reuses SimpleTuner's existing centered FP64 reference. Update the vendoring notice and all OPTIONS translations.
Validation:
.venv/bin/python -m unittest discover -v -f tests/— 7,154 tests completed successfully, with 345 skips.The CPU CI job installs PyTorch-family wheels from the official CPU index and verifies Torch, TorchAudio, and TorchVision imports before test discovery, avoiding mixed CUDA runtime builds in a CPU test environment.