Skip to content

Remove backwards symlinks - #1044

Draft
maherthomsi wants to merge 2 commits into
bottlerocket-os:developfrom
maherthomsi:remove-backwards-symlinks
Draft

maherthomsi wants to merge 2 commits into
bottlerocket-os:developfrom
maherthomsi:remove-backwards-symlinks

Conversation

@maherthomsi

Copy link
Copy Markdown
Contributor

Description of changes:
Make the NVIDIA /usr/lib/nvidia/tesla backwards-compatibility symlinks opt-in so EKS k8s 1.37+ nvidia variants can drop them, while k8s 1.36 and below keep them. The FHS library-path normalization previously created these legacy symlinks for every variant; we don't want to carry that forward on new variants, but must not change behavior for existing ones.

This changes the defaults to clean (no symlinks) and ships -compat templates that downstream variants render to restore the old behavior.

  • nvidia-container-toolkit: generate-cdi-specs.service no longer passes --additional-symlinks /usr/lib/nvidia/tesla by default. Its ExecStart can now be overridden by a rendered drop-in, and a new generate-cdi-specs-exec-start-conf-compat template restores the flag. The unit is ordered After=settings-applier.service so a rendered drop-in is present before the oneshot runs.
  • nvidia-k8s-device-plugin: the default exec-start template drops --cdi-enabled-hooks create-lib-symlinks; a new nvidia-k8s-device-plugin-exec-start-conf-compat template retains it.

Testing done:
Built an aws-k8s-1.36-nvidia x86_64 AMI consuming this kit and booted it as a node on a live EKS 1.36 cluster (nvidia.com/gpu allocatable = 1). Verified both paths:

  • Clean defaults (build with no downstream overrides): device-plugin exec-start.conf had no create-lib-symlinks; /etc/cdi/nvidia.json contained 0 nvidia/tesla references - confirming the intended 1.37+ behavior.
  • Compat templates (build with downstream overrides applied): device-plugin exec-start.conf = --cdi-enabled-hooks create-lib-symlinks; generate-cdi-specs ran with --additional-symlinks; /etc/cdi/nvidia.json contained 30 nvidia/tesla references; nvidia-smi worked (driver 580.178.04); host /usr/lib/nvidia/tesla present (49 symlinks).
  • Confirmed generate-cdi-specs.service keeps RefuseManualStart=true and that a daemon-reload (not a restart) applies the drop-in before the oneshot runs.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

The FHS library-path normalization left generate-cdi-specs generating
`/usr/lib/nvidia/tesla` backwards-compatibility symlinks for every variant.
Starting with EKS k8s 1.37, nvidia variants should not create these symlinks.

Signed-off-by: Maher Homsi <maherhom@amazon.com>
…mlinks

The default device-plugin exec-start template no longer passes
`--cdi-enabled-hooks create-lib-symlinks`, so containers do not receive the
legacy `/usr/lib/nvidia/tesla` symlinks on EKS k8s 1.37+ nvidia variants. Add a
-compat exec-start template that retains the flag for k8s 1.36 and below.

Signed-off-by: Maher Homsi <maherhom@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant