fix: retry provisioning and add IPv6, bucket policy, hostUsers

CHANGELOG.md

@@ -22,6 +22,23 @@ All notable changes to RustFS Operator are documented in this file. The format i
 
 ## [Unreleased]
 
+### Added
+
+- Tenant `spec.network` for Service IP families and IPv6 listen addresses, plus dual-stack binds
+  for operator observability, STS, and Console sockets.
+- Tenant `spec.hostUsers` and OpenShift `hostUsers: false` defaults for `restricted-v3`.
+- Tenant bucket canned anonymous access and ConfigMap-sourced bucket policies.
+
+### Fixed
+
+- Provisioning now requeues transient RustFS admin/S3 and Kubernetes failures instead of leaving
+  policies, users, and buckets failed until an unrelated object change.
+
+### Changed
+
+- Documented that distinct-physical-disk erasure failures and a separate data-plane operator are
+  outside this controller's scope.
+
 ## [0.0.6] - 2026-08-22
 
 ### Added

deploy/rustfs-operator/README.md

@@ -59,9 +59,10 @@ allowed runtime identity. Keep `openshift.enabled=false` and omit the Tenant
 fields on generic Kubernetes so the RustFS defaults remain in effect.
 
 This profile provides SCC-compatible manifests but does not by itself imply
-OpenShift certification or OperatorHub distribution. Support is currently
-limited to `restricted-v2`; the `restricted-v3` requirement to set
-`spec.hostUsers: false` is not implemented.
+OpenShift certification or OperatorHub distribution. Chart-managed Deployments
+set `hostUsers: false` when `openshift.enabled=true`, and Tenant workloads do
+the same for an explicit empty security-context pair or `spec.hostUsers: false`,
+covering the OpenShift `restricted-v3` host-user-namespace control.
 
 The RustFS server image is an independent prerequisite. It must support an
 arbitrary SCC-assigned UID: writable image-layer directories, including
@@ -109,6 +110,7 @@ The following table lists the configurable parameters of the RustFS Operator cha
 | `operator.prometheusRule.enabled` | Create Prometheus alert rules for operator and tenant storage health | `false` |
 | `operator.tenantMonitor.enabled` | Poll RustFS tenant storage health and capacity metrics | `true` |
 | `operator.tenantMonitor.intervalSeconds` | Tenant storage monitor interval | `300` |
+| `operator.bindAddress` | Optional literal IPv4/IPv6 bind address for operator HTTP sockets; empty prefers `::` then `0.0.0.0` | `""` |
 | `clusterDomain` | Kubernetes cluster DNS domain used for Tenant peer URLs, generated TLS SANs, and operator STS auto TLS | `cluster.local` |
 | `operator.env` | Environment variables | `[{name: RUST_LOG, value: info}]` |
 | `operator.nodeSelector` | Node selector for pod placement | `{}` |
@@ -236,7 +238,9 @@ The generated ClusterRole grants only `get`, `list`, and `watch` for Secrets and
 
 | Parameter | Description | Default |
 |-----------|-------------|---------|
-| `openshift.enabled` | Omit chart-managed Deployment security contexts and delegate runtime identity to OpenShift SCC | `false` |
+| `openshift.enabled` | Omit chart-managed Deployment security contexts, set `hostUsers: false`, and delegate runtime identity to OpenShift SCC | `false` |
+| `network.ipFamilyPolicy` | Optional Service `ipFamilyPolicy` for chart-managed Services | `""` |
+| `network.ipFamilies` | Optional Service `ipFamilies` for chart-managed Services | `[]` |
 | `namespace` | Namespace to deploy to | `""` (uses release namespace) |
 | `commonLabels` | Labels to add to all resources | `{}` |
 | `commonAnnotations` | Annotations to add to all resources | `{}` |

deploy/rustfs-operator/crds/tenant-crd.yaml

@@ -30,6 +30,14 @@ spec:
                 description: Buckets that should exist in the RustFS tenant.
                 items:
                   properties:
+                    anonymous:
+                      description: Canned anonymous access for this bucket. Mutually exclusive with `policy`.
+                      enum:
+                      - Private
+                      - Download
+                      - Upload
+                      - Public
+                      type: string
                     deletionPolicy:
                       enum:
                       - Retain
@@ -45,12 +53,36 @@ spec:
                     objectLock:
                       nullable: true
                       type: boolean
+                    policy:
+                      description: Custom bucket policy document sourced from a ConfigMap. Mutually exclusive with `anonymous`.
+                      nullable: true
+                      properties:
+                        configMapKeyRef:
+                          properties:
+                            key:
+                              maxLength: 253
+                              minLength: 1
+                              type: string
+                            name:
+                              maxLength: 253
+                              minLength: 1
+                              type: string
+                          required:
+                          - key
+                          - name
+                          type: object
+                      required:
+                      - configMapKeyRef
+                      type: object
                     region:
                       nullable: true
                       type: string
                   required:
                   - name
                   type: object
+                  x-kubernetes-validations:
+                  - message: bucket policy and anonymous access are mutually exclusive
+                    rule: '!(has(self.policy) && has(self.anonymous) && self.anonymous != ''Private'')'
                 maxItems: 1024
                 type: array
                 x-kubernetes-list-map-keys:
@@ -357,6 +389,14 @@ spec:
                   - name
                   type: object
                 type: array
+              hostUsers:
+                description: |-
+                  Pod `hostUsers` for generated RustFS workloads.
+
+                  `false` isolates the user namespace and satisfies OpenShift `restricted-v3`.
+                  When omitted, an OpenShift-style empty security-context pair also renders `hostUsers: false`.
+                nullable: true
+                type: boolean
               image:
                 nullable: true
                 type: string
@@ -572,6 +612,32 @@ spec:
                 default: /data
                 nullable: true
                 type: string
+              network:
+                description: |-
+                  Tenant Service IP family policy and RustFS listen addresses.
+                  When omitted, generated Services inherit the cluster default and RustFS listens on IPv4.
+                nullable: true
+                properties:
+                  ipFamilies:
+                    items:
+                      description: Kubernetes Service IP family values.
+                      enum:
+                      - IPv4
+                      - IPv6
+                      type: string
+                    maxItems: 2
+                    type: array
+                    x-kubernetes-list-type: set
+                  ipFamilyPolicy:
+                    description: Kubernetes Service IP family policy values.
+                    enum:
+                    - SingleStack
+                    - PreferDualStack
+                    - RequireDualStack
+                    - null
+                    nullable: true
+                    type: string
+                type: object
               podDeletionPolicyWhenNodeIsDown:
                 description: |-
                   Controls how the operator handles Pods when the node hosting them is down (NotReady/Unknown).

deploy/rustfs-operator/templates/_helpers.tpl

@@ -76,6 +76,22 @@ Name of the namespaced Role used by STS auto TLS.
 {{- printf "%s-sts-tls" (include "rustfs-operator.fullname" .) | trunc 63 | trimSuffix "-" }}
 {{- end }}
 
+{{/*
+Optional Service ipFamilyPolicy / ipFamilies from values.network.
+*/}}
+{{- define "rustfs-operator.serviceNetwork" -}}
+{{- $network := default dict .Values.network -}}
+{{- if $network.ipFamilyPolicy }}
+ipFamilyPolicy: {{ $network.ipFamilyPolicy }}
+{{- end }}
+{{- if $network.ipFamilies }}
+ipFamilies:
+{{- range $network.ipFamilies }}
+- {{ . }}
+{{- end }}
+{{- end }}
+{{- end }}
+
 {{/*
 Create the name of the console service account to use
 */}}

deploy/rustfs-operator/templates/console-deployment.yaml

@@ -7,6 +7,7 @@
   "CONSOLE_LOGIN_ADMISSION_MAX_IN_FLIGHT" "console.loginAdmission.maxInFlight"
   "CONSOLE_LOGIN_ADMISSION_BODY_LIMIT_BYTES" "console.loginAdmission.bodyLimitBytes"
   "CONSOLE_LOGIN_ADMISSION_TIMEOUT_SECONDS" "console.loginAdmission.timeoutSeconds"
+  "CONSOLE_BIND_ADDRESS" "console.bindAddress"
 -}}
 {{- range $env := .Values.console.env }}
 {{- if hasKey $reservedConsoleEnv $env.name -}}
@@ -48,6 +49,9 @@ spec:
         checksum/secret: {{ include (print $.Template.BasePath "/console-secret.yaml") . | sha256sum }}
     spec:
       serviceAccountName: {{ include "rustfs-operator.consoleServiceAccountName" . }}
+      {{- if $openShiftEnabled }}
+      hostUsers: false
+      {{- end }}
       {{- with .Values.console.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
@@ -71,6 +75,10 @@ spec:
                 secretKeyRef:
                   name: {{ include "rustfs-operator.fullname" . }}-console-secret
                   key: jwt-secret
+          {{- with .Values.console.bindAddress }}
+            - name: CONSOLE_BIND_ADDRESS
+              value: {{ . | quote }}
+          {{- end }}
           {{- with $consoleLoginAdmission }}
             - name: CONSOLE_LOGIN_ADMISSION_REQUESTS_PER_SECOND
               value: {{ .requestsPerSecond | quote }}

deploy/rustfs-operator/templates/console-frontend-deployment.yaml

@@ -21,6 +21,9 @@ spec:
         {{- include "rustfs-operator.selectorLabels" . | nindent 8 }}
         app.kubernetes.io/component: console-frontend
     spec:
+      {{- if $openShiftEnabled }}
+      hostUsers: false
+      {{- end }}
       {{- with .Values.console.frontend.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}

deploy/rustfs-operator/templates/console-frontend-service.yaml

@@ -9,6 +9,7 @@ metadata:
     app.kubernetes.io/component: console-frontend
 spec:
   type: ClusterIP
+  {{- include "rustfs-operator.serviceNetwork" . | nindent 2 }}
   ports:
     - port: {{ .Values.console.frontend.service.port }}
       targetPort: http

deploy/rustfs-operator/templates/console-service.yaml

@@ -13,6 +13,7 @@ metadata:
   {{- end }}
 spec:
   type: {{ .Values.console.service.type }}
+  {{- include "rustfs-operator.serviceNetwork" . | nindent 2 }}
   {{- if and (eq .Values.console.service.type "LoadBalancer") .Values.console.service.loadBalancerIP }}
   loadBalancerIP: {{ .Values.console.service.loadBalancerIP }}
   {{- end }}

deploy/rustfs-operator/templates/deployment.yaml

@@ -24,6 +24,7 @@
   "OPERATOR_STS_TLS_AUTO" "sts.tls.auto"
   "OPERATOR_TENANT_MONITOR_ENABLED" "operator.tenantMonitor.enabled"
   "OPERATOR_TENANT_MONITOR_INTERVAL_SECONDS" "operator.tenantMonitor.intervalSeconds"
+  "OPERATOR_BIND_ADDRESS" "operator.bindAddress"
   "POD_NAME" "the Pod metadata.name field"
 -}}
 {{- range $env := .Values.operator.env }}
@@ -56,6 +57,9 @@ spec:
         app.kubernetes.io/component: operator
     spec:
       serviceAccountName: {{ include "rustfs-operator.serviceAccountName" . }}
+      {{- if $openShiftEnabled }}
+      hostUsers: false
+      {{- end }}
       {{- with .Values.operator.imagePullSecrets }}
       imagePullSecrets:
         {{- toYaml . | nindent 8 }}
@@ -137,6 +141,10 @@ spec:
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.namespace
+          {{- with .Values.operator.bindAddress }}
+            - name: OPERATOR_BIND_ADDRESS
+              value: {{ . | quote }}
+          {{- end }}
           {{- if .Values.sts.enabled }}
             - name: OPERATOR_STS_PORT
               value: {{ .Values.sts.port | quote }}

deploy/rustfs-operator/templates/operator-metrics-service.yaml

@@ -13,6 +13,7 @@ metadata:
   {{- end }}
 spec:
   type: {{ .Values.operator.metrics.service.type }}
+  {{- include "rustfs-operator.serviceNetwork" . | nindent 2 }}
   {{- if and (eq .Values.operator.metrics.service.type "ClusterIP") .Values.operator.metrics.service.clusterIP }}
   clusterIP: {{ .Values.operator.metrics.service.clusterIP }}
   {{- end }}

deploy/rustfs-operator/templates/operator-sts-service.yaml

@@ -16,6 +16,7 @@ metadata:
   {{- end }}
 spec:
   type: {{ .Values.sts.service.type }}
+  {{- include "rustfs-operator.serviceNetwork" . | nindent 2 }}
   {{- if and (eq .Values.sts.service.type "ClusterIP") .Values.sts.service.clusterIP }}
   clusterIP: {{ .Values.sts.service.clusterIP }}
   {{- end }}

deploy/rustfs-operator/values.yaml

@@ -5,10 +5,22 @@ clusterDomain: cluster.local
 
 # OpenShift installation compatibility. When enabled, the chart omits Pod and
 # container securityContext fields from its own Deployments so the namespace
-# SecurityContextConstraints (SCC) can assign an allowed UID and FSGroup.
+# SecurityContextConstraints (SCC) can assign an allowed UID and FSGroup, and
+# sets hostUsers: false for restricted-v3.
 openshift:
   enabled: false
 
+# Optional IP family policy for chart-managed Services. Empty inherits the
+# cluster default. Example dual-stack:
+# network:
+#   ipFamilyPolicy: PreferDualStack
+#   ipFamilies:
+#     - IPv4
+#     - IPv6
+network:
+  ipFamilyPolicy: ""
+  ipFamilies: []
+
 # Operator deployment configuration
 operator:
   # Number of operator replicas
@@ -61,6 +73,10 @@ operator:
     enabled: true
     intervalSeconds: 300
 
+  # Empty binds the IPv6 unspecified address (::) then falls back to 0.0.0.0.
+  # Set to a literal IPv4 or IPv6 address to pin the listen address.
+  bindAddress: ""
+
   # Basic process probes. Override these for stricter platform-specific checks.
   livenessProbe:
     httpGet:
@@ -175,6 +191,9 @@ console:
   # Console server port
   port: 9090
 
+  # Empty binds the IPv6 unspecified address (::) then falls back to 0.0.0.0.
+  bindAddress: ""
+
   # Log level for console (trace, debug, info, warn, error)
   logLevel: info
 

docs/operator-user-guide.md

@@ -100,7 +100,10 @@ namespace SecurityContextConstraints (SCC) to assign an allowed UID and
 FSGroup. This is manifest compatibility, not an OpenShift certification claim.
 Generic Kubernetes installations must keep the default
 `openshift.enabled=false` behavior. The current target is `restricted-v2`;
-`restricted-v3` also requires `spec.hostUsers: false`, which is not yet covered.
+`restricted-v3` also requires `spec.hostUsers: false`. The Operator sets that
+field on chart-managed Deployments when `openshift.enabled=true`, and on Tenant
+StatefulSets when `spec.hostUsers` is set or when an OpenShift-style empty
+security-context pair delegates identity to SCC.
 
 SCC-compatible manifests are insufficient when the server image assumes UID
 `10001`. Before deploying a Tenant, use an arbitrary-UID-compatible image whose
@@ -544,6 +547,8 @@ Useful Tenant-level fields:
 | `podDeletionPolicyWhenNodeIsDown` | Node-down pod deletion behavior. |
 | `securityContext` | Pod SecurityContext overrides for all RustFS Pools. |
 | `containerSecurityContext` | RustFS container SecurityContext overrides for all Pools. |
+| `hostUsers` | Optional Pod `hostUsers`. `false` isolates the user namespace (`restricted-v3`). An OpenShift empty security-context pair also defaults this to `false`. |
+| `network` | Optional Service `ipFamilyPolicy`/`ipFamilies` and RustFS listen addresses. Omitted Tenants keep IPv4 `0.0.0.0` listen addresses. IPv6 or dual-stack clusters should set `ipFamilies: [IPv6]` or `ipFamilyPolicy: PreferDualStack` so Services and `RUSTFS_ADDRESS` use `[::]`. |
 
 Both fields are also available on each `spec.pools[]` entry. Pool values are
 merged over Tenant values, which are merged over the Operator's defaults. By
@@ -563,6 +568,7 @@ MinIO Operator contract:
 
 ```yaml
 spec:
+  hostUsers: false
   pools:
     - name: pool-0
       securityContext: {}
@@ -850,7 +856,9 @@ The operator can create RustFS policies, users, and buckets after the Tenant wor
 - `spec.credsSecret` for RustFS admin credentials.
 - `spec.policies` for canned policies sourced from ConfigMaps.
 - `spec.users` for regular users. Each user must have at least one direct policy mapping.
-- `spec.buckets` for buckets and optional object lock.
+- `spec.buckets` for buckets, optional object lock, canned anonymous access (`Private`, `Download`, `Upload`, `Public`), or a custom bucket policy ConfigMap. `anonymous` and `policy` are mutually exclusive. When both are omitted, the operator does not change a live bucket policy.
+
+Transient Kubernetes and RustFS admin/S3 failures (timeouts, 429, 5xx, connection errors, TLS not ready) leave provisioning items `Pending` and requeue instead of marking the Tenant `Failed`. Permanent 4xx configuration errors still fail and wait for a spec or object change.
 
 ConfigMaps and user Secrets must live in the Tenant namespace. The Operator indexes references from Tenant specs, so creating or updating a referenced object enqueues every referencing Tenant without requiring or mutating labels or requiring write access to that object.
 
@@ -919,10 +927,13 @@ spec:
   buckets:
     - name: app-data
       objectLock: true
+      anonymous: Download
 ```
 
 Deletion behavior is conservative: provisioned resources are retained when removed from the Tenant spec.
 
+This operator provisions those objects onto the Tenant's RustFS cluster. A separate data-plane operator is out of scope.
+
 ### 7.9 Pool Lifecycle
 
 `spec.poolLifecycle` controls explicit pool lifecycle requests. The current PVC retention policy is `Retain`.
@@ -1175,6 +1186,15 @@ kubectl logs -n <namespace> -l rustfs.tenant=<tenant>
 
 Check PVC binding, StorageClass availability, image pull errors, node selectors, tolerations, and resource requests.
 
+### Distinct physical disks
+
+RustFS requires each local erasure endpoint to map to a distinct physical disk. The Operator only
+creates one PVC per `volumesPerServer` entry; if several PVCs land on the same node disk, the
+server exits with `local erasure endpoints must use distinct physical disks`. That is a storage
+topology / StorageClass issue, not an Operator provisioning bug. Fix the volume placement, or use
+a StorageClass that provisions independent disks. `RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true` is a
+RustFS data-plane escape hatch and is not applied by the Operator.
+
 ### S3 API is not reachable
 
 Verify the Tenant S3 service and endpoints: