Skip to content

packages: add nvidia-dra-driver-gpu - #1035

Open
maherthomsi wants to merge 2 commits into
bottlerocket-os:developfrom
maherthomsi:dra-changes
Open

maherthomsi wants to merge 2 commits into
bottlerocket-os:developfrom
maherthomsi:dra-changes

Conversation

@maherthomsi

@maherthomsi maherthomsi commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Description of changes:
Adds the nvidia-dra-driver-gpu package, which builds the NVIDIA GPU DRA driver (gpu-kubelet-plugin) and installs it as a host systemd service. The package is added to the bottlerocket-core-kit workspace and included in the kit manifest so it is available to consuming variants.

Bump settings sdk for this change to include bottlerocket-os/bottlerocket-settings-sdk#148

The DRA driver (nvidia-dra-driver-gpu.service) and the kubernetes device plugin (nvidia-k8s-device-plugin.service) both allocate the same GPUs, so only one runs at a time. This is enforced without a systemd Conflicts=: the DRA unit gates on ConditionFileNotEmpty=/etc/nvidia-dra-driver-gpu/enabled, and the marker is rendered run (non-empty) only when DRA is enabled and the device plugin is disabled. DRA is therefore opt-in and off by default.

Both settings present

kubelet-dra-drivers.nvidia.enabled kubelet-device-plugins.nvidia.enabled DRA service device-plugin service GPU publisher
enabled disabled running off DRA (ResourceSlice)
enabled enabled off (defers) running device plugin (nvidia.com/gpu)
disabled enabled off running device plugin
disabled disabled off off none

Settings extension not present (failure to adopt the DRA settings API)

Condition DRA marker DRA service device-plugin service GPU publisher
kubelet-dra-drivers extension absent not rendered off running (its own default) device plugin

What "unset" means per setting

Setting If unset Effect
kubelet-dra-drivers.nvidia.enabled treated as disabled (default "false"; shipped default is false) DRA off
kubelet-device-plugins.nvidia.enabled treated as enabled (default "true"; classic behavior) device plugin runs

Out-of-the-box default (both unset): DRA off, device plugin running.

Guarantees: exactly one GPU publisher at any time; the device plugin wins when both are enabled (deterministic, no boot-time race); with no settings API the DRA driver stays off

Testing done:

  • Built the kits for x86_64 and aarch64
  • Launched a custom ami with the driver added to the variant and ensured that the DRA driver was loaded.
    • Tested on aarch64
    • Tested on x86_64

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.spec Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.spec Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.spec Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.spec Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.spec Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.service Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu-exec-start-conf Outdated
@maherthomsi

Copy link
Copy Markdown
Contributor Author

Added settings-sdk changes to this PR

@maherthomsi
maherthomsi force-pushed the dra-changes branch 4 times, most recently from 63c65b5 to 5dc5941 Compare September 11, 2026 22:06
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu-exec-start-conf Outdated
@maherthomsi

maherthomsi commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Addresses review feedback on the nvidia-dra-driver-gpu package:

  • spec: add Requires: %{_cross_os}nvidia-container-toolkit - the rendered unit invokes /usr/bin/nvidia-cdi-hook, which that package ships.
  • service (base unit): replace the ExecStart=/usr/bin/false + Type=simple + Restart=always placeholder with an inert no-op (Type=oneshot, RemainAfterExit=true, ExecStart=/usr/bin/true). It no longer retries forever / logs to console before the drop-in renders, and the ExecStartPre checks are no longer duplicated between the base unit and the drop-in.
  • exec-start-conf (drop-in): reference the optional settings.kubernetes.hostname-override only inside {{#if_not_null}}, so an unset value no longer fails the strict render. Move the daemon's Type=simple/Restart=always/RestartSec/RemainAfterExit=false into the enabled branch (previously inherited from the base) and drop the redundant {{else}} branch since the inert base now covers the disabled/unset case.

Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu-exec-start-conf Outdated
Comment thread packages/nvidia-dra-driver-gpu/nvidia-dra-driver-gpu.service
@maherthomsi
maherthomsi force-pushed the dra-changes branch 2 times, most recently from 93457dd to 1919b69 Compare September 12, 2026 01:02
@maherthomsi
maherthomsi force-pushed the dra-changes branch 2 times, most recently from 2421dee to c2b24e5 Compare September 15, 2026 20:22
Add the nvidia-dra-driver-gpu package, which builds the NVIDIA GPU DRA
driver (gpu-kubelet-plugin) and installs it as a host systemd service.
Include it in the bottlerocket-core-kit workspace and kit manifest.

Signed-off-by: Maher Homsi <maherhom@amazon.com>
Signed-off-by: Maher Homsi <maherhom@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants