Skip to content

Sync RenderDoc v1.x through 5ad608809 - #4

Merged
dotm5 merged 134 commits into
dgcore-mainfrom
sync/renderdoc-v1.x-20261117
Nov 16, 2026
Merged

dotm5 merged 134 commits into
dgcore-mainfrom
sync/renderdoc-v1.x-20261117

Conversation

@dotm5

@dotm5 dotm5 commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Upstream integration

Merge baldurk/renderdoc v1.x at 5ad6088. Start from dgcore-main and use e99d699 as the functional merge base because PR #3 had already incorporated it as a squash. The merge commit records the official upstream parent.

Preserve the RenderDox product and DComp API contracts, capture-access replay worker rules, action visibility, and Vulkan device-address resource and barrier handling. Repair the Windows Qt moc platform flag, functional Python CLI paths, Vulkan demo comment encoding, and Bootstrap export-check tool discovery. Use VirtualAlloc zero initialization for the shim path buffer to remove the ClangCL memset link failure while preserving the shim's Kernel32-only imports.

Local validation

  • Windows x64 MSVC Release with Bootstrap passed. Checked DComp PE metadata, DCOMP_GetAPI and COM exports, replay marker, Vulkan layer manifest, embedded DXIL and Bootstrap exports.
  • RTX 4070 Laptop GPU captured frame 16 in D3D12_Simple_Triangle and VK_Simple_Triangle using the built commit. Opened both captures, verified the RGB output at the triangle and background, and verified that disabling the draw removed the triangle and clearing the filter restored it.
  • The DCOMP_GetAPI(10700) negotiation, dgcorecmd functional runner help, and 38 source and public-policy tests passed.
  • Rebuilt and replayed both GPU cases at e87d094. The standalone ClangCL x64 shim linked and dumpbin reported only Kernel32.dll.

Aidan Raymond and others added 30 commits October 24, 2026 04:35
* We could put everything onto the replay thread but that's unnecessary back-
  and-forth pingponging for a local capture.
* This is safe to use on the UI thread if it's local, but if it's remote it uses
  the same connection as the replay controller so must happen on the replay
  thread.
* We add a new helper (not exposed to swig) that handles this check, and for
  python we wrap the returned ICaptureAccess in an invoker that ensures we are
  on one of those threads.
* By default on linux some send() errors will kill the process instead of
  returning an error code.
* Previous check was gated behind vendor extensions being used accidentally.
* This was never fully supported by RenderDoc, certainly not when debugging was
  added, and it's unclear if vendors ever supported this either.
* shaderc doesn't build with VS2015 so can't be linked in.
* Creating it late when we're uploading the address map means it gets included
  as a non-internal resource in the replay resources list.
* Unclear why this was blocked on OS X clang as it clearly fires, but this means
  this warning is no longer OS X only which makes iterating on normal machines easier.
* We always just hold one version of the bytecode, in the shader container
  (possibly with an extra debug blob on D3D where that is necessary for DXBC).
* We also remove any attempt to fall back to the original shader blob if we have
  debug search paths found and we get a debug replacement blob (mostly relevant
  for Vulkan but legacy separate debug info on DXBC also does whole-blob
  replacement). If the debug search paths change and we reload, but fail to find
  a debug blob, we continue using the previous debug blob if we had one.
Zorro666 and others added 26 commits November 11, 2026 12:41
- Serialise_vkCmdBindPipeline
- Serialise_vkCmdBindDescriptorSets
- Serialise_vkCmdBindDescriptorSets2
- Serialise_vkCmdBindVertexBuffers
- Serialise_vkCmdBindVertexBuffers2
- Serialise_vkCmdBindIndexBuffer
- Serialise_vkCmdSetVertexInputEXT
- Serialise_vkCmdBindShadersEXT
- Serialise_vkCmdBindIndexBuffer2
- Serialise_vkCmdBindIndexBuffer3KHR
- Serialise_vkCmdBindVertexBuffers3KHR
- Serialise_vkCmdBindDescriptorBuffersEXT
- Serialise_vkCmdSetDescriptorBufferOffsetsEXT
- Serialise_vkCmdBindDescriptorBufferEmbeddedSamplersEXT
- Serialise_vkCmdSetDescriptorBufferOffsets2EXT
- Serialise_vkCmdBindDescriptorBufferEmbeddedSamplers2EXT
- Serialise_vkCmdPushConstants
- Serialise_vkCmdPushConstants2
- Serialise_vkCmdPushDescriptorSet
- Serialise_vkCmdPushDescriptorSetWithTemplate
- Serialise_vkCmdPushDescriptorSet2
- Serialise_vkCmdPushDescriptorSetWithTemplate2
- Serialise_vkCmdCopyQueryPoolResults
- Serialise_vkCmdCopyQueryPoolResultsToMemoryKHR
- Serialise_vkCmdBindTransformFeedbackBuffersEXT
- Serialise_vkCmdBeginTransformFeedbackEXT
- Serialise_vkCmdEndTransformFeedbackEXT
- Serialise_vkCmdBeginTransformFeedback2EXT
- Serialise_vkCmdBindTransformFeedbackBuffers2EXT
- Serialise_vkCmdEndTransformFeedback2EXT
- Serialise_vkCmdBeginConditionalRenderingEXT
- Serialise_vkCmdEndConditionalRenderingEXT
- Serialise_vkCmdBeginConditionalRendering2EXT
- Serialise_vkCmdBuildAccelerationStructuresIndirectKHR
- Serialise_vkCmdBuildAccelerationStructuresKHR
- Serialise_vkCmdCopyAccelerationStructureKHR
- Serialise_vkCmdCopyMemoryToAccelerationStructureKHR
- Inlined D3D12CommandData::AddResourceUsage
- Inlined D3D12CommandData::AddEventResourceUsage
- Delete state in copy assignment operator
- Added a move assignment operator
- Replays with EI containing state changes would have leaked memory
During loading replay state:
- D3D12CommandData::InRerecordRange() returns true
- D3D12CommandData::IsPartialCmdList() returns false
- Serialise_ResourceBarrier
- Serialise_IASetPrimitiveTopology
- Serialise_RSSetViewports
- Serialise_RSSetScissorRects
- Serialise_OMSetBlendFactor
- Serialise_OMSetStencilRef
- Serialise_SetDescriptorHeaps
- Serialise_IASetIndexBuffer
- Serialise_IASetVertexBuffers
- Serialise_SOSetTargets
- Serialise_SetPipelineState
- Serialise_OMSetRenderTargets
- Serialise_SetComputeRootSignature
- Serialise_SetComputeRootDescriptorTable
- Serialise_SetComputeRoot32BitConstant
- Serialise_SetComputeRoot32BitConstants
- Serialise_SetComputeRootConstantBufferView
- Serialise_SetComputeRootShaderResourceView
- Serialise_SetComputeRootUnorderedAccessView
- Serialise_SetGraphicsRootSignature
- Serialise_SetGraphicsRootDescriptorTable
- Serialise_SetGraphicsRoot32BitConstant
- Serialise_SetGraphicsRoot32BitConstants
- Serialise_SetGraphicsRootConstantBufferView
- Serialise_SetGraphicsRootShaderResourceView
- Serialise_SetGraphicsRootUnorderedAccessView
- Serialise_SetMarker
- Serialise_BeginEvent
- Serialise_EndEvent
- Serialise_DrawInstanced
- Serialise_DrawIndexedInstanced
- Serialise_Dispatch
- Serialise_ExecuteBundle
- Serialise_ClearDepthStencilView
- Serialise_ClearRenderTargetView
- Serialise_ClearUnorderedAccessViewUint
- Serialise_ClearUnorderedAccessViewFloat
- Serialise_DiscardResource
- Serialise_CopyBufferRegion
- Serialise_CopyTextureRegion
- Serialise_CopyResource
- Serialise_ResolveSubresource
- Serialise_CopyTiles
- Serialise_AtomicCopyBufferUINT
- Serialise_AtomicCopyBufferUINT64
- Serialise_OMSetDepthBounds
- Serialise_SetSamplePositions
- Serialise_ResolveSubresourceRegion
- Serialise_SetViewInstanceMask
- In Serialise_BeginRenderPass during Loading also process any Resolves to match Replay behaviour

- Serialise_BeginRenderPass
- Serialise_EndRenderPass
- Serialise_EmitRaytracingAccelerationStructurePostbuildInfo
- Serialise_CopyRaytracingAccelerationStructure
- Serialise_SetPipelineState1
- Serialise_DispatchRays
- Serialise_RSSetShadingRate
- Serialise_RSSetShadingRateImage
- Serialise_OMSetFrontAndBackStencilRef
- Serialise_RSSetDepthBias
- Serialise_IASetIndexBufferStripCutValue
Use e99d699 as the functional merge base because PR #3 already incorporated its changes as a squash. Retain the public RenderDox identity and replay worker safeguards while integrating upstream capture access, shared command loading and resource usage accounting.
Use the fork platform definition for the new Qt moc target, resolve the Python binding and runner names from their existing public interfaces, and make the Vulkan test comments valid UTF-8.
Pass the resolved dumpbin directly from the release build to the export checker. Reuse the public toolchain resolver when the checker runs on its own.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T16:48:38.058750Z 11a57f6 PR opened
🔒 Security Review ✅ Completed 2026-11-16T16:52:30.220016Z 11a57f6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11a57f6376

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}
else if(IsCaptureD3D12())
{
return m_D3D11->rasterizer.state.forcedSampleCount;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Read the D3D12 state in the D3D12 sample-count branch

For a loaded D3D12 capture, PipeState::SetState(const D3D12Pipe::State *) explicitly sets m_D3D11 to null, so this branch dereferences a null pointer. The new pixel-debug dialog calls this method from PixelDebugSelector::UpdateTarget(), meaning opening that dialog on a D3D12 draw crashes the application before debugging starts. Return the forced sample count from m_D3D12 instead.

Useful? React with 👍 / 👎.

}
}

ui->sample->setMaximum(pipe.GetOverrideSampleCount() - 1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the resolved sample count for the sample selector

When a D3D11 draw uses an MSAA render target without a forced sample count, GetOverrideSampleCount() returns zero. The preceding loop correctly derives sampleCount from the output textures, but this line discards that result and subtracts one from the zero override instead. This gives the spin box an invalid maximum and hides the selector, preventing users from selecting individual MSAA samples. Set the maximum using the resolved sampleCount, with a lower bound of one before subtraction.

Useful? React with 👍 / 👎.

Comment on lines +58 to +59
ui->viewLabel->setVisible(ui->view->maximum() > 1);
ui->view->setVisible(ui->view->maximum() > 1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Show disambiguation controls when two choices are available

For a Vulkan render pass with exactly two multiview views, the spin-box maximum is one, so these checks hide the view selector even though view 1 is valid. Users therefore cannot select the second view for pixel debugging. The sample visibility checks below have the same off-by-one error for 2× sampling. Since these maxima are zero-based indices, show the controls when the maximum is greater than zero, not greater than one.

Useful? React with 👍 / 👎.

m_LastCmdBufferID = GetResID(commandBuffer);

if(IsActiveReplaying(m_State))
if(InRerecordRange(m_LastCmdBufferID))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore the command-buffer ID before checking the replay range

For captures using vkCmdCopyQueryPoolResultsToMemoryKHR, removing m_LastCmdBufferID = GetResID(commandBuffer) causes this check to use a null resource ID: the replay loop in vk_core.cpp resets that member before every ContextProcessChunk() call. During active replay without an outside command buffer, the null ID matches neither the partial stack nor m_RerecordCmds, so this command is silently skipped and its destination retains stale query results. Restore the assignment before the range check.

Useful? React with 👍 / 👎.

Comment on lines +927 to +930
if(!origpreload.empty())
{
direct_setenv(PRELOAD_ENV_VAR, origpreload.c_str(), true);
direct_setenv("DCOMP_ORIGPRELOAD", "", true);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore empty original hooking environment values

On Linux, launching a capture with an originally unset or empty LD_PRELOAD saves an empty DCOMP_ORIGPRELOAD in GetHookingEnvMods(). This new guard then prevents ResetHookingEnvVars() from removing the injected preload when library_loaded() calls it. Consequently, subsequent child executions inherit the capture library even with hookIntoChildren disabled, since GetUnhookedEnvp() removes only the Vulkan-layer variable. The equivalent guard for the library path also leaves injected search paths behind. Distinguish an absent saved variable from a saved empty value rather than treating both as a reason to skip restoration.

Useful? React with 👍 / 👎.

Comment on lines +229 to +232
HOOK_EXPORT EGLDisplay EGLAPIENTRY eglGetPlatformDisplayEXT_renderdoc_hooked(
EGLenum platform, void *native_display, const EGLAttrib *attrib_list)
{
return eglGetPlatformDisplay_renderdoc_hooked(platform, native_display, attrib_list);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the EXT EGL attribute-list element width

On 64-bit platforms, applications calling eglGetPlatformDisplayEXT with a non-null attribute list supply 32-bit EGLint elements, as declared in official/eglext.h:899, but this hook forwards that memory to the core function as pointer-sized EGLAttrib elements. The driver consequently combines adjacent entries into invalid attributes and can read past the terminator, breaking display creation or crashing the captured application. The new eglCreatePlatformWindowSurfaceEXT hook has the same problem. Use the actual EXT signatures and entry points, or explicitly convert the attribute lists before forwarding to the core functions.

Useful? React with 👍 / 👎.

{
// expand out members of renderdoc and qrenderdoc using the stubs
if(!m_StubRD || !m_StubQRD)
if(m_DeferredInit == 0)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Check the stub pointers after deferred initialization completes

If deferred initialization cannot find a valid pystubs path or either stub import fails, its worker still sets m_DeferredInit to one while leaving a stub pointer null. Opening the Python shell then calls makeHelpContext(), which now skips the pointer checks and passes that null pointer to PyDict_SetItemString(), causing a native crash instead of gracefully disabling help as before. Keep the wait conditional on initialization state, but validate both stub pointers unconditionally before using them.

Useful? React with 👍 / 👎.

The shim reserves and commits a new VirtualAlloc region for the wide path. Its final character is already zero when GetModuleFileNameW uses the preceding characters. Remove the redundant SecureZeroMemory call that ClangCL lowered to an unresolved CRT memset.
@dotm5
dotm5 merged commit 368a254 into dgcore-main Nov 16, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants