Repository navigation
Conversation
Immediate queues retained all submitted kernels until queue finish, keeping their programs alive as distinct kernels accumulated. Limit retention to 10,000 distinct kernels per immediate command list. Before the next kernel launch, synchronize the list and release its retained kernels. Synchronize before submission so an error cannot cause an already submitted kernel to be retried. Keep captured kernels separate from ordinary submissions and classify capture after launch to handle implicit fork/join capture. Propagate capture-query errors before submission; afterward, log errors and conservatively retain kernels without failing a submitted launch. Preserve deferred retention for replayable command lists. Add regression coverage for in-order and out-of-order queues, the retention limit, synchronization, kernel reuse, explicit and implicit graph capture, and graph replay after limit cleanup. Fixes: Jira URT-1346 Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The capture-active guard lacks regression coverage with pre-existing pending kernels, so tests would not catch synchronization during capture that could retire captured kernels.
1 open issue
What changed in this PR
This PR bounds kernel retention in the Level Zero v2 adapter’s immediate queues while preserving kernels needed for graph capture and replay.
Changes:
- Synchronize before the next launch once an immediate command list retains 10,000 distinct kernels, then release those kernels.
- Separate ordinary submissions from captured and replayable kernels, with capture-query error handling before and after submission.
- Add regression coverage for retention limits, synchronization, kernel reuse, capture, and graph replay.
| File | Description |
|---|---|
unified-runtime/test/adapters/level_zero/v2/deferred_kernel.cpp |
Adds reclamation and graph-capture regression tests. |
unified-runtime/source/adapters/level_zero/v2/queue_immediate_out_of_order.cpp |
Marks out-of-order command list managers as immediate. |
unified-runtime/source/adapters/level_zero/v2/queue_immediate_in_order.cpp |
Marks in-order command list managers as immediate. |
unified-runtime/source/adapters/level_zero/v2/command_list_manager.hpp |
Declares immediate-list tracking and pending-kernel cleanup. |
unified-runtime/source/adapters/level_zero/v2/command_list_manager.cpp |
Implements bounded retention and capture-aware kernel cleanup. |
Review effort: Balanced
Fill each immediate command list with 10,000 distinct ordinary kernels before beginning graph capture. Verify capture preserves the pending references and skips synchronization at the retention limit. After capture, trigger ordinary kernel cleanup and verify captured kernels remain retained and the graph can still be replayed. Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Retention-limit synchronization holds the kernel mutex, creating a critical cross-queue deadlock risk; both launch paths must synchronize before acquiring that mutex while preserving the command-list lock.
1 open issue
1 resolved since last review
Review effort: Balanced
Comment on lines
+253
to
+254
| ZE2UR_CALL(zeCommandListHostSynchronize, | ||
| (getZeCommandList(), UINT64_MAX)); |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Immediate queues retained all submitted kernels until queue finish,
keeping their programs alive as distinct kernels accumulated.
Limit retention to 10,000 distinct kernels per immediate command list.
Before the next kernel launch, synchronize the list and release its
retained kernels. Synchronize before submission so an error cannot
cause an already submitted kernel to be retried.
Keep captured kernels separate from ordinary submissions and classify
capture after launch to handle implicit fork/join capture. Propagate
capture-query errors before submission; afterward, log errors and
conservatively retain kernels without failing a submitted launch.
Preserve deferred retention for replayable command lists.
Add regression coverage for in-order and out-of-order queues, the
retention limit, synchronization, kernel reuse, explicit and implicit
graph capture, and graph replay after limit cleanup.
Fixes: Jira URT-1346