Skip to content

fix(controller): render the RBAC the metrics proxy needs to answer a scrape - #216

Merged
duckhawk merged 1 commit into
mainfrom
feat/controller-rbac-proxy-binding
Sep 8, 2026
Merged

duckhawk merged 1 commit into
mainfrom
feat/controller-rbac-proxy-binding

Conversation

@duckhawk

@duckhawk duckhawk commented Sep 7, 2026

Copy link
Copy Markdown
Member

What

helm_lib_module_controller_rbac now renders the ClusterRoleBinding that the kube-rbac-proxy sidecar needs in order to answer a scrape, gated on a new optional controllerMetricsProxyPort key in its config. Chart bumped to 1.72.22.

Why

The two controller helpers are meant to be used together and disagree about who grants what:

  • helm_lib_module_controller_manifests adds a kube-rbac-proxy sidecar in front of the manager's metrics port whenever controllerMetricsProxyPort is set. The proxy authenticates and authorises every request by creating a TokenReview and a SubjectAccessReview.
  • helm_lib_module_controller_rbac renders the ServiceAccount and its bindings — and neither of those two verbs is among them.

So every module using the pair has to know to add a binding to d8:rbac-proxy by hand, and the ones that do not get a proxy that fails closed with 401 on every scrape.

The failure mode is what makes this worth fixing in the library rather than in each module. Nothing looks broken: the manager publishes its metrics, the ServiceMonitor selects the Service, the pod is Ready, and the controller's own log is silent. The only symptom is up == 0 on the module's job and a TargetDown alert that has been firing since the module was installed. Found in sds-object after TargetDown had been up for two weeks; the same binding is missing today in eight more storage modules that call these helpers.

How

{{- $rbacConfig := dict
  "controllerMetricsProxyPort" (int .Values.myModule.internal.controllerMetricsProxyPort)
  "roleRules" (list ...)
  "clusterRoleRules" (list ...)
}}
{{ include "helm_lib_module_controller_rbac" (list . $rbacConfig) }}

Pass the same value already given to helm_lib_module_controller_manifests. It is not used as a port — it is how the RBAC helper learns that a sidecar exists. Gating on it rather than rendering unconditionally keeps a controller without a metrics proxy from gaining the ability to review tokens, and keeps the rendered output of every current caller byte-identical until it opts in.

The binding is named d8:<module>:<fullname>:rbac-proxy, the convention the core modules (cert-manager, cni-cilium, node-manager, …) already follow. A module that carries this binding by hand today can therefore drop its copy and set the key in the same commit: the object keeps its identity and Helm adopts it rather than replacing it. Doing only one of the two would render the binding twice under one name, which Helm rejects.

Tests

Three cases added to helm_lib_module_controller_rbac_test.yaml: no binding without the key (5 documents, as before), the binding with the key (6 documents, correct roleRef and subject), and the naming with a custom fullname.

make ci/tests/unit: 407 tests in 86 suites pass. Snapshots unchanged, make doc/diff clean.

…scrape

helm_lib_module_controller_manifests puts a kube-rbac-proxy in front of the
manager's metrics port whenever controllerMetricsProxyPort is set. That proxy
authenticates and authorises every scrape by creating a TokenReview and a
SubjectAccessReview, and neither verb is in the RBAC its twin,
helm_lib_module_controller_rbac, renders. The binding was left to the module,
and modules kept forgetting it: one helper hands out the sidecar, the other
hands out the ServiceAccount, and nothing connects the two.

The failure is quiet in the worst way. The manager publishes its metrics, the
ServiceMonitor selects the Service, the pod is Ready, and the proxy fails
closed with 401 on every request. The only symptom is up == 0 on the module's
job and a TargetDown that has been firing since the module was installed; the
controller's own log says nothing at all.

Render the ClusterRoleBinding to d8:rbac-proxy — the ClusterRole Deckhouse
ships for exactly this pair of verbs — from the same helper, gated on
controllerMetricsProxyPort so that a controller without the sidecar does not
gain the ability to review tokens. The name matches the convention the core
modules already use, d8:<module>:<fullname>:rbac-proxy, so a module that
carries this binding by hand today can drop its copy on upgrade without the
object changing identity.

Bump the chart to 1.72.22.

Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
@duckhawk
duckhawk merged commit 9ae9cb1 into main Sep 8, 2026
4 checks passed
@duckhawk
duckhawk deleted the feat/controller-rbac-proxy-binding branch September 8, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants