fix(controller): render the RBAC the metrics proxy needs to answer a scrape - #216
Merged
Merged
Conversation
…scrape helm_lib_module_controller_manifests puts a kube-rbac-proxy in front of the manager's metrics port whenever controllerMetricsProxyPort is set. That proxy authenticates and authorises every scrape by creating a TokenReview and a SubjectAccessReview, and neither verb is in the RBAC its twin, helm_lib_module_controller_rbac, renders. The binding was left to the module, and modules kept forgetting it: one helper hands out the sidecar, the other hands out the ServiceAccount, and nothing connects the two. The failure is quiet in the worst way. The manager publishes its metrics, the ServiceMonitor selects the Service, the pod is Ready, and the proxy fails closed with 401 on every request. The only symptom is up == 0 on the module's job and a TargetDown that has been firing since the module was installed; the controller's own log says nothing at all. Render the ClusterRoleBinding to d8:rbac-proxy — the ClusterRole Deckhouse ships for exactly this pair of verbs — from the same helper, gated on controllerMetricsProxyPort so that a controller without the sidecar does not gain the ability to review tokens. The name matches the convention the core modules already use, d8:<module>:<fullname>:rbac-proxy, so a module that carries this binding by hand today can drop its copy on upgrade without the object changing identity. Bump the chart to 1.72.22. Signed-off-by: v.oleynikov <vasily.oleynikov@flant.com>
krpsh123
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
helm_lib_module_controller_rbacnow renders the ClusterRoleBinding that thekube-rbac-proxysidecar needs in order to answer a scrape, gated on a new optionalcontrollerMetricsProxyPortkey in its config. Chart bumped to 1.72.22.Why
The two controller helpers are meant to be used together and disagree about who grants what:
helm_lib_module_controller_manifestsadds akube-rbac-proxysidecar in front of the manager's metrics port whenevercontrollerMetricsProxyPortis set. The proxy authenticates and authorises every request by creating aTokenReviewand aSubjectAccessReview.helm_lib_module_controller_rbacrenders the ServiceAccount and its bindings — and neither of those two verbs is among them.So every module using the pair has to know to add a binding to
d8:rbac-proxyby hand, and the ones that do not get a proxy that fails closed with 401 on every scrape.The failure mode is what makes this worth fixing in the library rather than in each module. Nothing looks broken: the manager publishes its metrics, the ServiceMonitor selects the Service, the pod is Ready, and the controller's own log is silent. The only symptom is
up == 0on the module's job and aTargetDownalert that has been firing since the module was installed. Found insds-objectafterTargetDownhad been up for two weeks; the same binding is missing today in eight more storage modules that call these helpers.How
Pass the same value already given to
helm_lib_module_controller_manifests. It is not used as a port — it is how the RBAC helper learns that a sidecar exists. Gating on it rather than rendering unconditionally keeps a controller without a metrics proxy from gaining the ability to review tokens, and keeps the rendered output of every current caller byte-identical until it opts in.The binding is named
d8:<module>:<fullname>:rbac-proxy, the convention the core modules (cert-manager, cni-cilium, node-manager, …) already follow. A module that carries this binding by hand today can therefore drop its copy and set the key in the same commit: the object keeps its identity and Helm adopts it rather than replacing it. Doing only one of the two would render the binding twice under one name, which Helm rejects.Tests
Three cases added to
helm_lib_module_controller_rbac_test.yaml: no binding without the key (5 documents, as before), the binding with the key (6 documents, correctroleRefand subject), and the naming with a customfullname.make ci/tests/unit: 407 tests in 86 suites pass. Snapshots unchanged,make doc/diffclean.