-
Notifications
You must be signed in to change notification settings - Fork 105
refactor(runtime)!: adopt the modern Infini stack #506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
voltjia
wants to merge
52
commits into
main
Choose a base branch
from
refactor/adopt-modern-infini-stack
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
04fcda0
refactor(runtime)!: adopt modern Infini stack
voltjia e9496ca
docs: define Infini stack repository boundaries
voltjia 05c92ba
refactor: own Infini stack integration build
voltjia bca98df
docs: document the InfiniLM stack build
voltjia f46c074
ci: validate the modern NVIDIA stack
voltjia dab847b
fix: keep static graph cache metadata dynamic
voltjia d78dacb
refactor(ops): use canonical InfiniOps APIs
voltjia d49427f
fix(build): select required linked InfiniOps providers
voltjia f128747
fix(build): limit compiled models to Qwen3
voltjia 0cbc125
feat(distributed): add eager point-to-point wrappers
voltjia 9f40398
feat(ops): restore prepacked linear execution
voltjia 28cc18b
fix(ops): match canonical Gemm call schema
voltjia 7dd8ac6
fix(sampling): support scalar sample outputs
voltjia 80af271
fix(nn): pass RoPE output handles by value
voltjia 3af4355
fix(attention): include variable-length MHA declaration
voltjia 0cb41af
fix(engine): use local static graph assertions
voltjia 6fcd5b7
fix(config): reject unsupported Qwen3 linear bias
voltjia 4e36168
test: cover modern runtime contracts
voltjia 354372a
docs: update modern Qwen3 support boundary
voltjia 1a89775
refactor(build): select InfiniOps implementations from JSON
voltjia 2c2d94b
style: format InfiniOps attention adapters
voltjia baf3315
fix(graph): update paged replay metadata through tensors
voltjia 1551767
fix(graph): fall back to eager paged decode under tensor parallelism
voltjia c76b105
fix: restore modern InfiniCore API compatibility
voltjia 9d64e14
fix(runtime): sync post-migration InfiniCore updates
voltjia 5964127
style: format migrated sources
voltjia e4a549b
fix(runtime): keep receive buffers mutable
voltjia 1ed454f
fix(build): exclude unsupported MXFP4 path
voltjia 33f0409
fix(packaging): preload Torch shared libraries
voltjia 89a0fdb
refactor(runtime): replace deprecated causal softmax backend
voltjia df6f2a3
style: format causal softmax adapter
voltjia 8c2d3f5
feat(runtime): unlock validated InfiniOps capabilities
voltjia 57e9d01
fix(runtime): select available InfiniOps implementations
voltjia 24f9d46
fix(runtime): enable available Moore InfiniOps paths
voltjia 7f958b7
fix(runtime): accept native MetaX device name
voltjia 1de38df
fix(core): validate Cat inputs against tensor rank
voltjia 63147e7
feat(models): enable validated MiniCPM Eagle path
voltjia 9eb004a
style(runtime): group ModelRunner imports
voltjia dcfbebc
fix(graph): enable safe paged decode capture under TP
voltjia 5e9be02
feat(moore): enable modern flash attention
voltjia d2766a2
style(moore): format attention integration
voltjia 4093b0b
build(moore): configure the modern stack
voltjia bae5a89
feat(cambricon): enable tensor-parallel flash attention
baominghelly 5de9a02
fix(distributed): avoid transient runtime teardown
voltjia 88e4c6e
Merge pull request #556 from InfiniTensor/feat/cambricon-tp-flash-att…
voltjia 7c39376
feat(iluvatar): enable modern Infini stack inference (#557)
gongchensu 64fac2f
feat(hygon): enable the modern Infini stack (#564)
gongchensu 6966281
feat(ascend): enable InfiniOps flash attention (#558)
baominghelly 3faa1db
perf(nvidia): reduce refactored inference overhead (#565)
voltjia b6fbf94
fix(ascend): use CommInitAll for single-node TP (#566)
baominghelly 62d48c2
fix(ascend): keep unsafe operators out of device graphs (#568)
baominghelly f21b8f9
fix(allocator): reuse pinned free blocks (#569)
voltjia File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| repo: | ||
| url: https://github.com/InfiniTensor/InfiniLM.git | ||
| branch: main | ||
|
|
||
| github: | ||
| status_context_prefix: "ci/infinilm" | ||
|
|
||
| platforms: | ||
| nvidia: | ||
| image: | ||
| dockerfile: images/nvidia/ | ||
| build_args: | ||
| BASE_IMAGE: nvcr.io/nvidia/pytorch:25.12-py3 | ||
| CUDA_ARCH: sm_80,sm_86,sm_89,sm_90 | ||
| APT_MIRROR: https://mirrors.tuna.tsinghua.edu.cn/ubuntu | ||
| PIP_INDEX_URL: https://pypi.org/simple | ||
| InfiniCore_BRANCH: __Branch_Name__ | ||
| docker_args: | ||
| - "--user=root" | ||
| - "--network=host" | ||
| - "--privileged" | ||
| - "--cap-add=ALL" | ||
| - "--pid=host" | ||
| - "--ipc=host" | ||
| - "--workdir=/workspace" | ||
| volumes: | ||
| - /data:/data | ||
| - /data-aisoft:/data-aisoft | ||
| - /data-aisoft/artifacts/CI_nvidia_test/__WORKSPACE__:/artifacts | ||
| setup: pip install .[dev] --no-build-isolation | ||
| jobs: | ||
| gpu_inferencetest: | ||
| type: inferencetest | ||
| resources: | ||
| ngpus: [1, 4] | ||
| gpu_style: nvidia | ||
| shm_size: 64g | ||
| timeout: 3600 | ||
| stages: | ||
| - name: test | ||
| run: python InfiniLM/examples/test_infer.py --device nvidia --model=/data-aisoft/mechdancer/models/9g_8b_thinking/ | ||
| gpu_benchtest: | ||
| type: benchtest | ||
| resources: | ||
| gpu_style: nvidia | ||
| shm_size: 64g | ||
| timeout: 3600 | ||
| env: | ||
| TEST_PARAM: ['default'] | ||
| stages: | ||
| - name: test | ||
| run: python InfiniLM/examples/bench.py --device nvidia --model=/data-aisoft/mechdancer/models/9g_8b_thinking/ --input-len=256,1024 --output-len=256,1024 --batch-size=8 <TEST_PARAM> | ||
| gpu_accuracytest: | ||
| type: accuracytest | ||
| resources: | ||
| gpu_style: nvidia | ||
| shm_size: 64g | ||
| timeout: 3600 | ||
| env: | ||
| TEST_PARAM: ['--bench mmlu'] | ||
| stages: | ||
| - name: test | ||
| run: python InfiniLM/test/bench/test_benchmark.py --device nvidia --model /data-aisoft/mechdancer/models/9g_8b_thinking/ --bench mmlu --backend cpp --max-new-tokens 5 --cache-dir /data-aisoft/pepe/datasets/ --split=val <TEST_PARAM> | ||
| gpu_servicetest: | ||
| type: servicetest | ||
| resources: | ||
| shm_size: 64g | ||
| env: | ||
| MODEL_LIST: 9g_8b_thinking | ||
| ENGINE: InfiniLM | ||
| TEST_PARAM: ['default'] | ||
| stages: | ||
| - name: test | ||
| run: python InfiniLM/scripts/test_perf.py --verbose |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
直接指定 stack-builder unittest 文件的方式在 Python 3.13 下存在解析歧义
python -m unittest test/scripts/test_build_infini_stack.py -v可能把test解析成标准库的 test 包,然后报ModuleNotFoundError: No module named 'test.scripts'。改用测试发现机制是稳定的:python -m unittest discover -s test/scripts -p 'test_build_infini_stack.py' -v建议把文档命令改为测试发现形式。在本次审查环境中,该形式下 24 个测试全部通过。