open_jev: add native Rust/CUDA text worker - #55
Draft
hsliuustc0106 wants to merge 7 commits into
Draft
hsliuustc0106 wants to merge 7 commits into
hsliuustc0106 wants to merge 7 commits into
Conversation
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
5 of 7 tasks
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
1 of 8 tasks
4 tasks
1 task done
4 tasks
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Closes #54
Serve Open-Jev-27B-v1.1 through the existing Rust frontend with a native Rust/CUDA text worker. It compiles and tokenizes independent candidate prompts, applies the trained scalar decision head and saved temperature, and returns typed
choice,scoreandnoulanswers.The CUDA ABI is now 4; rebuild the shared library and both native workers together.
Test Plan
System1-Omni Version / Commit: base
1be7d41eb74d5b49fd25194042041319950e5dce; head40062bc12ac4b0d6364f61f0db963a49a1fde4f9.Open-Jev tests and fixtures live under the repository-level
tests/open_jev/; shared Qwen JSON/configuration and CUDA tests live undertests/qwen3_5/. Cargo explicitly registers the integration tests, and private unit tests load their files from that same top-level tree. Frontend CPU mock-worker coverage is tracked separately in issue #46 and PR #58.Check prompts, typed answers and token IDs against pinned Open-Jev fixtures. Run the workspace checks and reserved-GPU kernel/worker validation. The recorded performance comparison fixes cached RMSNorm in both native variants and measures scalar versus packed SiLU on 74 single-candidate JevBench requests: one excluded feasibility pass and two measured passes per configuration. Hardware, revisions, controls and acceptance gates are in the linked results.
Test Result
Passed locally after cleanup:
cargo fmt --all --check cargo clippy --workspace --locked --all-targets -- -D warnings cargo test --workspace --locked cargo build --workspace --release --locked61b83b3. The CUDA source and GPU test contents are unchanged; their location and registration changed, and device checks were not repeated. SM89 compilation passed previously; SM89 device execution and full Cua-S1 checkpoint inference were not checked.The comparison covers single-candidate, eager text inference. Fast's preparation variability prevents claiming a stable aggregate winner, and the measured outputs do not establish native/Fast accuracy parity. Multi-candidate prefix sharing and Open-Jev CUDA Graph replay remain unvalidated. Nsight Systems supplied timelines because Nsight Compute counters were denied.
Self-review
Contributor checklist from CONTRIBUTING.md, left for completion before requesting review: