fix(kunlun): memset fallback, graph-op destructor UB, missing selectlast_token_hidden kernel - #1544
Open
rubik-hua wants to merge 1 commit into
Open
fix(kunlun): memset fallback, graph-op destructor UB, missing selectlast_token_hidden kernel#1544rubik-hua wants to merge 1 commit into
rubik-hua wants to merge 1 commit into
Conversation
Collaborator
Collaborator
|
另外static cache的推理流程中有cpu交互,预期就是不能跑graph。需要补充paged attention或者flash attention相关的支持。 |
Author
paged attention或者flash attention 我在实现,后面我单独提PR。 不过没编译graph,运行也没指定 --enable-graph情况下,会coredump在graph代码中,这个肯定是bug,需要修复的。 上面agent给的检视意见我会过一遍修复掉。 |
…last_token_hidden kernel Kunlun inference was broken by three independent issues; this commit fixes all of them. 1. Kunlun memset support (src/infinirt/kunlun/infinirt_kunlun.cc) memsetDevice/memsetDeviceAsync returned INFINI_STATUS_NOT_IMPLEMENTED. This surfaced after fafd723 (issue/1170), which made Tensor::zeros actually zero the buffer via memset; previously the buffer from empty() was left untouched, so the stub was never exercised. XRE exposes no memset API, so emulate it with chunked host-to-device copies from a 16 MiB host staging buffer, keeping host memory bounded regardless of fill size. The async variant falls back to the sync copy, consistent with mallocAsync/freeAsync already falling back to sync XRE calls. 2. Uninitialized DispatchableGraphOperator members (include/infinicore/graph/graph.hpp) planned_meta_/runner_/deleter_ had no default initializers. When a derived constructor throws (e.g. an operator unsupported on the current device), stack unwinding still runs the base destructor, which read the garbage deleter_ and jumped through it - a silent SIGSEGV that masked the real "operator not supported" error and made the process exit with no traceback. Default-initialize all three to nullptr so the existing null check in the destructor skips cleanup safely. No functional change for already-supported devices: every constructor overwrites the three members via INFINICORE_GRAPH_OP_DISPATCH before they are read. 3. Kunlun kernel for select_last_token_hidden for already-supported devices: every constructor overwrites the three members via INFINICORE_GRAPH_OP_DISPATCH before they are read. 3. Kunlun kernel for select_last_token_hidden Add an XPU kernel (f16/bf16/f32) and register it in the infiniop dispatcher, unblocking the static-cache path on Kunlun. The build picks the new files up through the existing ops/*/kunlun/*.xpu glob; no build change needed. Note that paged_caching/paged_attention still have no Kunlun kernels, so --enable-paged-attn remains unsupported on this platform for now. Also fix an out-of-scope index in a cat.cc assert (tensors[i] -> tensor, where no such i exists in that loop), which broke debug builds.
rubik-hua
force-pushed
the
kunlun-graph-bugfix
branch
from
September 3, 2026 10:22
77d883b to
b1ebc99
Compare
Author
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Kunlun inference was broken by three independent issues; this commit
fixes all of them.
memsetDevice/memsetDeviceAsync returned INFINI_STATUS_NOT_IMPLEMENTED.
This surfaced after fafd723 (issue/1170), which made Tensor::zeros
actually zero the buffer via memset; previously the buffer from empty()
was left untouched, so the stub was never exercised. XRE exposes no
memset API, so emulate it with chunked host-to-device copies from a
16 MiB host staging buffer, keeping host memory bounded regardless of
fill size. The async variant falls back to the sync copy, consistent
with mallocAsync/freeAsync already falling back to sync XRE calls.
planned_meta_/runner_/deleter_ had no default initializers. When a
derived constructor throws (e.g. an operator unsupported on the current
device), stack unwinding still runs the base destructor, which read the
garbage deleter_ and jumped through it - a silent SIGSEGV that masked
the real "operator not supported" error and made the process exit with
no traceback. Default-initialize all three to nullptr so the existing
null check in the destructor skips cleanup safely. No functional change
for already-supported devices: every constructor overwrites the three
members via INFINICORE_GRAPH_OP_DISPATCH before they are read.
Kunlun kernel for select_last_token_hidden for already-supported devices: every constructor overwrites the three members via INFINICORE_GRAPH_OP_DISPATCH before they are read.
Kunlun kernel for select_last_token_hidden
Add an XPU kernel (f16/bf16/f32) and register it in the infiniop
dispatcher, unblocking the static-cache path on Kunlun. The build picks
the new files up through the existing ops//kunlun/.xpu glob; no build
change needed. Note that paged_caching/paged_attention still have no
Kunlun kernels, so --enable-paged-attn remains unsupported on this
platform for now.
Also fix an out-of-scope index in a cat.cc assert (tensors[i] -> tensor,
where no such i exists in that loop), which broke debug builds.
当前最新代码仓上昆仑芯推理报错:

还会有跟graph相关的coredump,堆栈如下:


修复后验证:
python examples/test_infer.py --device kunlun --model=/mnt/geogpt-doc-new/default/infinilm-models/Qwen3-0.6B --tp 1 --batch-size 1 --prompt "who are you"