docs(operator): document native cold storage (spec.coldStorage)

documentation/concepts/cold-storage.md

@@ -166,7 +166,7 @@ A legitimate change of version, after restoring onto a different host or migrati
 - **No materialized views.** A materialized view accepts no storage policy at all, so it cannot be tiered to object storage. It uses [TTL](/docs/query/sql/alter-mat-view-set-ttl/) for retention instead.
 - **No downgrade after the first upload.** Enabling remote upload is a one-way version change.
 - **Upgrade replicas first.** Older instances and open source nodes do not understand the seal event and suspend WAL apply rather than skipping it.
-- **Operator-driven manager failover.** The manager role moves at runtime, but nothing detects manager death or elects a replacement, and a hot-switched role does not survive a restart.
+- **Operator-driven manager failover.** The manager role moves at runtime, but nothing detects manager death or elects a replacement, and a hot-switched role does not survive a restart. On Kubernetes, the [Enterprise Kubernetes Operator](/docs/enterprise-kubernetes-operator/configuration/#cold-storage) declares the manager in `spec.coldStorage.manager` and re-applies it across restarts; electing a replacement is still a human decision.
 - **Backups do not contain cold data.** A backup captures a cold partition's local metadata and symbol indexes, but not its `data.parquet` bytes, which exist only in the object store. The database backup and the object store prefix have to be kept, and recovered, as one set. See [Backup and restore](/docs/operations/backup/#cold-storage-partitions).
 - **No idle warm cache.** Chunks are shared only while an active read holds a lease.
 - **Fixed key layout.** Object key templates are not configurable.

documentation/enterprise-kubernetes-operator/configuration.md

@@ -69,10 +69,12 @@ isolation boundary:
 - `spec.replication.root` defaults to the identity-scoped
   `db/<namespace>/<cluster>/`. Leave it unset unless deliberately adopting an
   existing WAL stream.
+- `spec.coldStorage.objectStoreRef.root` defaults to the identity-scoped
+  `cold/<namespace>/<cluster>/` and is immutable once the cluster is created.
 
 A provider-level `root` on `QuestDBObjectStore` does **not** add a base prefix;
-the per-use backup or replication root overrides it. Scope cloud permissions to
-the effective per-use prefixes.
+the per-use backup, replication, or cold-storage root overrides it. Scope cloud
+permissions to the effective per-use prefixes.
 
 ## Backup and HA
 
@@ -118,6 +120,69 @@ runbooks for routing and failover.
 The WAL cleaner is enabled by default. Do not disable it unless another process
 owns replication-WAL retention; otherwise WAL grows without bound.
 
+## Cold storage
+
+Native [cold storage](/docs/concepts/cold-storage/) tiers sealed partitions to
+object storage. On QuestDB Enterprise 4.0.0 or later, enable it through
+`spec.coldStorage`:
+
+```yaml
+spec:
+  coldStorage:
+    objectStoreRef:
+      name: questdb-cold-store
+      # root: cold/questdb/   # defaults to cold/<namespace>/<cluster>/
+    # manager: 1              # default
+```
+
+`objectStoreRef` names a same-namespace `QuestDBObjectStore`. The operator
+resolves it into a connection Secret named `<cluster>-cold-storage-store`,
+mounts that Secret on every database Pod, and enables cold storage on every
+instance; QuestDB pods perform all object-store I/O. Recognizable QuestDB
+Enterprise image tags below 4.0.0 are rejected at admission. Unknown custom
+tags and digest-only images are allowed and fail closed at runtime if the
+engine cannot answer cold-storage observations.
+
+`manager` selects the 1-based instance serial that holds the cold-storage
+[manager role](/docs/concepts/cold-storage/#roles); every other instance is a
+refresher. Changing `manager` requests an engine-level handoff: the operator
+demotes the old manager, verifies it settled as a refresher, then promotes the
+replacement — never two managers at once. Do not run
+[`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/)
+against operator-managed instances; change `spec.coldStorage.manager` and
+follow the
+[manager-move runbook](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager).
+
+Run the manager on a replica. The default `manager: 1` normally selects the
+primary, and a `Planned` promotion requires the manager settled on an instance
+other than the departing primary — otherwise the promotion fails with
+`ColdManagerMoveRequired`. See
+[Planned prechecks](/docs/enterprise-kubernetes-operator/high-availability/#planned-prechecks).
+
+Watch the `ColdStorageHealthy` condition and `status.coldStorage`:
+
+```sh
+kubectl get questdbcluster <name> -n <namespace> \
+  -o jsonpath='{range .status.conditions[?(@.type=="ColdStorageHealthy")]}{.type}{"="}{.status}{" reason="}{.reason}{" message="}{.message}{"\n"}{end}{.status.coldStorage}{"\n"}'
+```
+
+`True/ManagerReady` with `status.coldStorage.currentManager` naming the
+desired instance and a positive `managerTerm` is the healthy steady state.
+While a handoff is in flight, `handoffSource` names the demoted manager until
+the replacement is observed stable. `False/ManagerHandoffBlocked` fails closed
+— for example while the current manager is unreachable — rather than risking
+two managers.
+
+The operator configures and observes cold storage; it never reads or deletes
+cold objects, never assigns
+[storage policies](/docs/concepts/storage-policy/) to tables, and never
+garbage-collects the prefix. Assign policies with
+[`ALTER TABLE SET STORAGE POLICY`](/docs/query/sql/alter-table-set-storage-policy/).
+`spec.coldStorage` may be added to a running cluster but cannot be removed,
+and `objectStoreRef` is immutable: changing the store or root would re-point
+live cold data. `manager` must not exceed `instances`, so lower `instances`
+only after moving the manager to a remaining serial.
+
 ## Storage
 
 Each instance receives its own persistent volume:
@@ -330,9 +395,16 @@ Object-store keys are **not** rejected, but operator-provided primary
 
 Never put access keys, passwords, or credential-bearing connection strings in
 `spec.config`: the custom resource and rendered ConfigMap are plaintext.
-Additional backup destinations (`backup.object.store.1` through `.9`) and
-`cold.storage.object.store` are suitable only for credential-free,
-ambient-identity settings until a Secret-backed mechanism is available.
+Additional backup destinations (`backup.object.store.1` through `.9`) are
+suitable only for credential-free, ambient-identity settings until a
+Secret-backed mechanism is available.
+
+For cold storage, use [`spec.coldStorage`](#cold-storage): its connection is
+Secret-backed. When `spec.coldStorage` is absent, the legacy
+`cold.storage.enabled`, `cold.storage.object.store`, and `cold.storage.role`
+keys remain supported for credential-free settings. When it is present, those
+three keys are operator-owned and rejected, while advanced `cold.storage.*`
+and `storage.policy.*` tuning stays user-managed.
 
 The `qwp.udp.*` receiver keys are also operator-owned: `qwp.udp.enabled` and
 `qwp.udp.bind.to` are set through [`spec.protocols.qwp.udp`](#qwp-udp), and
@@ -375,11 +447,18 @@ I/O even though the operator preserves the single-writer gate.
 Important immutable choices include:
 
 - `spec.objectStoreRef` once set;
+- `spec.coldStorage` in presence once set, and `spec.coldStorage.objectStoreRef`
+  in value;
 - `spec.storage.storageClassName`;
 - `spec.bootstrap` in presence and value;
 - `spec.protocols.pgwire.tls` in presence;
 - `spec.replication.root` in presence and value; and
-- `QuestDBObjectStore.spec.provider`.
+- `QuestDBObjectStore.spec.provider` and its physical coordinates: the S3
+  bucket, region, and endpoint, and the Azure container, account name, and
+  endpoint. Credential references, Secret contents, and non-coordinate
+  transport options remain mutable, so
+  [credential rotation](#rotate-static-object-store-credentials-safely) is
+  unaffected.
 
 Storage size is expand-only. For exact transition rules and less common fields,
 use the

documentation/enterprise-kubernetes-operator/high-availability.md

@@ -52,7 +52,11 @@ It does not use the database's in-place
 [role switch](/docs/high-availability/failover/) (`SWITCH ROLE`,
 `POST /lifecycle/switch`). Do not run those against instances managed by the
 operator: `status.currentPrimary` and the PVC role labels would no longer
-describe the cluster.
+describe the cluster. The same applies to
+[`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/):
+change
+[`spec.coldStorage.manager`](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager)
+instead.
 
 :::
 
@@ -84,6 +88,27 @@ kubectl get questdbcluster <name> -n <namespace> \
   -o jsonpath='{.status.replication.activePromotion}{"\n"}'
 ```
 
+On a cluster with
+[`spec.coldStorage`](/docs/enterprise-kubernetes-operator/configuration/#cold-storage),
+also require settled cold-manager ownership away from the departing primary:
+current-generation `ColdStorageHealthy=True/ManagerReady`,
+`status.coldStorage.currentManager` equal to `spec.coldStorage.manager` and
+not the current primary, a positive `managerTerm`, and an empty
+`handoffSource`:
+
+```sh
+kubectl get questdbcluster <name> -n <namespace> \
+  -o jsonpath='manager={.spec.coldStorage.manager}{" current="}{.status.coldStorage.currentManager}{" term="}{.status.coldStorage.managerTerm}{" handoffSource="}{.status.coldStorage.handoffSource}{"\n"}'
+```
+
+Otherwise a non-no-op `Planned` promotion fails fast with
+`ColdManagerMoveRequired`. The promotion never moves the cold manager itself:
+[move `spec.coldStorage.manager`](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager)
+to a ready replica, wait for `ManagerReady`, and create a new promotion
+object. Moving it to the promotion target is allowed but leaves the manager on
+the new primary after the cutover. An active promotion pauses new cold-manager
+handoffs, so move the manager before creating the promotion.
+
 The defaults reflect two different costs:
 
 - `catchUpTimeoutSeconds: 900` bounds pre-drain catch-up while the old primary
@@ -190,6 +215,15 @@ promotion, the fenced loss state must have zero ready `<name>-rw` EndpointSlice
 endpoints; afterward the selected target is the only live primary. Record the
 accepted recovery point and any expected lost-write window.
 
+Emergency promotion never waits on cold-storage ownership. If the lost primary
+was also the cold manager, `ColdStorageHealthy` reports
+`False/ManagerHandoffBlocked` and the handoff fails closed rather than risking
+two managers: refreshers continue serving already-offloaded partitions, while
+new uploads and remote garbage collection wait. Cold health recovers through
+the ordinary handoff once the fenced instance returns as a replica. To place
+the manager elsewhere afterward,
+[move `spec.coldStorage.manager`](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager).
+
 ## If promotion stalls or fails
 
 Read the promotion's status first:

documentation/enterprise-kubernetes-operator/known-limitations.md

@@ -51,6 +51,23 @@ Increasing `replication.primary.keepalive.interval` lengthens this
 stale-direct-client window. Leave its 10-second default unless QuestDB advises
 otherwise.
 
+### Planned promotion requires the cold manager off the departing primary
+
+With
+[`spec.coldStorage`](/docs/enterprise-kubernetes-operator/configuration/#cold-storage),
+a `Planned` promotion starts only when the cold-storage manager is settled on
+an instance other than the departing primary; otherwise it fails fast with
+`ColdManagerMoveRequired`, and there is no override. The default `manager: 1`
+normally selects the primary, so
+[move the manager](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager)
+to a replica before the first `Planned` cutover. The operator never elects a
+replacement manager on its own: changing `spec.coldStorage.manager` is the
+only path, and an unreachable current manager blocks the handoff fail-closed
+(`ManagerHandoffBlocked`) rather than risking two managers. `Emergency`
+promotion never waits on cold ownership; while the manager is lost, new
+uploads and remote garbage collection pause and refreshers continue serving
+already-offloaded partitions.
+
 ### Promotion can be unbounded
 
 A live but hung final upload can hold `Draining`: the operator cannot

documentation/enterprise-kubernetes-operator/operations/database.md

@@ -304,6 +304,49 @@ writer-health or separate follower contract, and the expected retained or
 deleted replica PVCs. Confirm each current primary/replica PVC's
 `questdb.io/role` label matches that role.
 
+## Move the cold-storage manager
+
+With
+[`spec.coldStorage`](/docs/enterprise-kubernetes-operator/configuration/#cold-storage),
+the cold-storage [manager role](/docs/concepts/cold-storage/#roles) is moved
+by changing `spec.coldStorage.manager`, never by running
+[`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/)
+against instances. Run the manager on a replica: that moves upload, manifest,
+and garbage-collection work off the primary, and a `Planned` promotion
+requires the manager settled away from the departing primary.
+
+Before the change, require current generation,
+`ColdStorageHealthy=True/ManagerReady`, and an empty `handoffSource`:
+
+```sh
+kubectl get questdbcluster <name> -n <namespace> \
+  -o jsonpath='{range .status.conditions[?(@.type=="ColdStorageHealthy")]}{.type}{"="}{.status}{" reason="}{.reason}{" observed="}{.observedGeneration}{"\n"}{end}manager={.spec.coldStorage.manager}{" current="}{.status.coldStorage.currentManager}{" term="}{.status.coldStorage.managerTerm}{" handoffSource="}{.status.coldStorage.handoffSource}{"\n"}'
+```
+
+Choose the serial of a ready replica (`<name>-3` has serial `3`) and patch:
+
+```sh
+kubectl patch questdbcluster <name> -n <namespace> --type merge \
+  -p '{"spec":{"coldStorage":{"manager":3}}}'
+```
+
+The operator demotes the old manager, verifies it settled as a refresher,
+then promotes the replacement — never two managers at once. While the handoff
+is in flight, `status.coldStorage.handoffSource` names the demoted instance.
+Uploads and remote garbage collection pause during the short managerless
+interval; reads continue everywhere. Wait for `True/ManagerReady` with
+`currentManager` naming the target and a positive `managerTerm`.
+
+`False/ManagerHandoffBlocked` fails closed rather than risking two managers —
+for example while the current manager is unreachable. Recover the named
+instance; do not work around the block with direct SQL. Two interactions to
+plan around:
+
+- An active promotion pauses new cold-manager handoffs. Finish or resolve the
+  cutover first.
+- `manager` must not exceed `instances`, so scale in only after moving the
+  manager to a remaining serial.
+
 ## Grow storage
 
 Storage is expand-only, and `spec.storage.storageClassName` is immutable.

documentation/operations/cold-storage.md

@@ -132,6 +132,17 @@ WHERE timestamp IN '2026-02-10';
 
 The manager role moves between instances at runtime, with no restart. Role switching requires database administrator (system admin) privileges. See [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) for the full syntax.
 
+:::note
+
+On a cluster managed by the
+[Enterprise Kubernetes Operator](/docs/enterprise-kubernetes-operator/), do not
+run these statements directly. Change `spec.coldStorage.manager` and follow the
+[operator's manager-move runbook](/docs/enterprise-kubernetes-operator/operations/database/#move-the-cold-storage-manager):
+the operator performs this same demote-then-promote sequence and would treat a
+hand-switched role as drift.
+
+:::
+
 The supported handoff is two steps, in this order:
 
 ```questdb-sql title="1. On the current manager"