Repository navigation
Add SM86 Ampere support and improve Ada CuTe attention - #3295
Merged
Merged
Conversation
Sync KohakuFA-CuTe c0786dd, enable SM86 in native and automatic dispatch, and extend architecture, graph replay, and gradient regressions. Update the attention option documentation and translations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SM86 Ampere GPUs were rejected by SimpleTuner's attention controller and native entry point, while Ada retained the older wide-head layouts. Sync KohakuFA-CuTe c0786dd and enable the existing CuTe backend for SM86 in both strict and automatic training modes.
Validation:
.venv/bin/python -m unittest discover -v -f tests/: 7,091 tests run, 323 skipped, no failures.kernels.pymatches the pinned upstream file byte-for-byte. Native C++ matches after namespace adaptation, and imported regression methods match after import-path integration. Pre-commit formatting, import sorting, lint, and whitespace checks passed.These are correctness and dispatch checks; this PR does not claim newly measured attention latency or full training throughput.