From 806cbe4ae63273522065c665cc422906a6792d10 Mon Sep 17 00:00:00 2001 From: Seer <2774413+chideat@users.noreply.github.com> Date: Mon, 7 Sep 2026 06:35:55 -0400 Subject: [PATCH] docs(redis): correct the cross-datacenter replication API surface (#59) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Validated docs/en/functions/95-disaster-recovery/ against a live business cluster and against redis-operator release-5.1. Several documented API names and procedures no longer match the product, and the setup guides could not be followed as written. Peer authentication now binds the instance's default account, per 207d1c3c ("accept the default account as a peer-auth credential") and 77851bdc. Both setup guides used to open by creating a dedicated custom RedisUser with the username aapeerrepl and its own password Secret; aapeerrepl was removed from the product together with secretName, its only password source, and never shipped in a released build. Step 1 of each guide now confirms the default-account RedisUser — drc-acl--default on cluster, rfr-acl--default otherwise — which needs no provisioning, and states the requirement that follows: default passwords are generated per instance, so every member of a group must be given the same one. Binding a custom account stays documented as the narrower alternative. API corrections: - spec.activeRedis.redisUserName does not exist; the field is redisUserRef. A patch using the old name is a silent no-op: the API server strips it with only a warning and reports "patched (no change)". - ActiveRedisConnection.spec.secretName was removed. Manifests carrying it are rejected outright, which made the whole Redis 6.0 setup procedure unexecutable. A 6.0 link takes the same instance-level binding. - Port 7379 is added to the proxy Service only in Active-Active mode. A Disaster Recovery link uses the proxy RESP endpoint alone, so the upgrade guide's prerequisite to open 7379 between datacenters was for a port nothing listens on in that mode. - spec.peerPort has no 7379 default; it defaults to the port of addresses[0] and is meant to be left unset. Also corrected: the peer-auth RedisUser may be a custom or a default account (only system is refused), the password-Secret exclusivity rule applies when either RedisUser involved is bound as a peer-auth credential, the automatic 6.0 path binds the existing default-account RedisUser rather than minting one, the failover sample no longer shows pause: true as an upstream failure, and the Redis 6.0 downstream example no longer reuses the upstream's serviceID under a warning that it must be unique. The Redis 6.0 procedure gains a step to wait for the operator-written binding, which the connection would otherwise outrun. Corrected examples were re-validated by server-side dry-run. (cherry picked from commit 386e5273d64802da1c8f41a2d85b2711fdf1985c) --- TERMINOLOGY_CANDIDATES.md | 32 ++++++ .../95-disaster-recovery/10-intro.mdx | 16 ++- .../20-disaster-recovery/20-setup.mdx | 105 ++++++++---------- .../20-disaster-recovery/30-failover.mdx | 4 +- .../40-upgrade-6.0-to-7.2.mdx | 68 ++++++------ .../30-active-active/20-setup.mdx | 49 ++++---- .../95-disaster-recovery/90-limitations.mdx | 2 +- docs/en/release_notes.mdx | 4 +- 8 files changed, 149 insertions(+), 131 deletions(-) diff --git a/TERMINOLOGY_CANDIDATES.md b/TERMINOLOGY_CANDIDATES.md index d4c5e45..4072cb2 100644 --- a/TERMINOLOGY_CANDIDATES.md +++ b/TERMINOLOGY_CANDIDATES.md @@ -97,6 +97,38 @@ and in `doc/backup-restore-rwo.md` are deliberately kept out of customer docs. | **bundle version** (API `spec.upgradeOption.crVersion`) | `redis-operator` `api/middleware/v1/redis_types.go:32-36` (`CRVersion`, `AutoUpgrade`) and `:339` (`printcolumn` name `Bundle Version`); annotation `middleware.instance/crVersion` in `internal/config/keys.go:44`; tier selection in `pkg/actor/manager.go:107-140` (`Search` takes the highest actor whose `Version()` is not greater than the annotation, defaulting to the operator's own version) | Not new — `docs/en/functions/10-create-instance.mdx:261` already documents the `BUNDLE VERSION` column as "the version number of the operator managing the instance". This is the first place the docs pair that display name with the `crVersion` / `autoUpgrade` API fields, and the first place a docs statement depends on a specific value. `crVersion` is the operator's own version (`internal/vc/generator.go:60,163`, from `REDIS_OPERATOR_VERSION`), so the column is product-version-shaped. **The published threshold `5.1.0` is deliberately conservative**: the code boundary is `4.1.2`, because the frozen `v5.0` tier (`Version()` `4.1.2`, serving `[4.1.2, 5.1.0)`, introduced by `7a7ebc3c` as a verbatim copy of a canonical builder that already carried the network-delivered restore) also skips the mount-based path — so instances pinned at `4.1.2`–`5.0.x` do not in fact need `ReadWriteMany`. Wording chosen by the maintainer 2026-09-07; revisit if the frozen tier changes. | | **transient in-cluster source** | `internal/ops/restoresource/source.go`; `internal/builder/restorebuilder/source.go`; wired by the base and `v5.0` ensure-resource actors (`internal/ops/{cluster,failover}/actor[/v5.0]/actor_ensure_resource.go`) | **Coined for these docs.** The implementation is a versitygw S3 gateway; the docs deliberately name neither the project nor S3, to avoid confusing it with the **S3 Backup** method on the same page. Confirm the phrase. | +## Cross-datacenter replication API correction (added 2026-09-07) + +Terms and phrasings introduced while validating +`docs/en/functions/95-disaster-recovery/` against a live business cluster +(operator `v5.1.0-rc.43.g2e4ef16f`, an ancestor of `release-5.1` whose only +intervening commits are e2e tests) and against `release-5.1` source. Repository +paths are relative to the sibling checkout `redis-group/redis-operator`. + +| Term as used in docs | Provenance | Note for the reviewer | +|:---|:---|:---| +| **module peer port** (`7379`) | `internal/config/keys.go:54-57` — `ActiveRedisPeerPort = 7379`, "the module-owned peer listener port"; the Service port is appended only for mesh in `internal/builder/activeredis/service.go:66-80` | Replaces the docs' former "dedicated peer port", which implied a separate transport that Disaster Recovery links use. They do not: the port is a Service-level **alias onto the same proxy listener**, added only in Active-Active mode. Confirm the phrase, and confirm that "alias" is the right word for customer docs. | +| **default-account `RedisUser`** | `internal/builder/activeredis/peerauth.go:26-48` (`DefaultAccountRedisUserRef`); names generated by `internal/builder/clusterbuilder/acl.go:74-76` (`drc-acl--default`) and `internal/builder/failoverbuilder/acl.go:74-76` (`rfr-acl--default`) | **Coined for these docs.** Names the `RedisUser` object holding an instance's `default` account. As of `207d1c3c` (*accept the default account as a peer-auth credential*) this is the binding both setup guides now document for Redis `7.2`, and the one the operator writes for Redis `6.0`. The code has no customer-facing name for it. Confirm the phrase, and confirm that spelling the generated resource names out in the docs is wanted. | +| **peer-auth binding** | `api/middleware/redis/v1alpha1/activeredis_types.go:63-84` (`RedisUserRef`); `internal/controller/middleware/redis_peerauth_provision.go` | The docs already used *peer-auth credential* for the value. *Binding* is now used for the `spec.activeRedis.redisUserRef` reference itself, so that "the credential" and "the field naming it" stop being the same word. Confirm both are wanted. | +| **one endpoint carries both transports** | `internal/builder/activeredis/service.go:49-56` — "ONE proxy endpoint carries both transports. The proxy has a single listener and picks the backend port per session" | **Coined for these docs.** Describes why a Disaster Recovery link needs only the proxy RESP address. Confirm the phrasing; the alternative is to describe the per-session routing explicitly, which is more accurate but longer. | + +Resolved by this change, and recorded here because the wrong names were +published: `spec.activeRedis.redisUserName` does not exist (the field is +`redisUserRef`, `api/middleware/redis/v1alpha1/activeredis_types.go:84`), and +`ActiveRedisConnection.spec.secretName` has been removed +(`internal/controller/middleware/activeredisconnection_controller.go:205-217`). +Both were verified against the live API by server-side dry-run. `aapeerrepl` is +gone from the product as well — `77851bdc` removed the mint along with +`secretName`, its only password source — so the docs no longer name it; it +survives only as a test constant in `tests/framework/activeredis/deployer.go:53`. + +Checked and left unchanged: `status.upstreamPeer.service_metadata` is real — +`internal/controller/middleware/redis/activeredis_args.go:47-50` injects +`instance=/` on both generations, plus `version=` on the new +module. The live connection shows it empty only because +`activeredisconnection_controller.go:433-439` records `upstreamPeer` once, from +the first shard to report peer info, and that shard reported none. + ## Known inconsistencies to resolve 1. **"Source side" / "Target side" (Web Console) vs. "upstream" / "downstream" diff --git a/docs/en/functions/95-disaster-recovery/10-intro.mdx b/docs/en/functions/95-disaster-recovery/10-intro.mdx index aeeb1ad..ed55346 100644 --- a/docs/en/functions/95-disaster-recovery/10-intro.mdx +++ b/docs/en/functions/95-disaster-recovery/10-intro.mdx @@ -72,8 +72,8 @@ The replication module ships in two generations. Which one an instance loads is | Active-Active (`mesh`) | Not supported | Supported | | Replicated command set | Smaller — commands introduced after Redis 6.0 are not replicated | Full; see [Command Support](./60-commands.mdx) | | Command guards | None — `MOVE`, `SWAPDB`, `RESTORE`, `FLUSHALL`, and `FLUSHDB` are replicated rather than refused | `MOVE`, `SWAPDB`, and `RESTORE` are always refused; `FLUSHALL` and `FLUSHDB` are refused while peers are attached | -| Peer authentication | None at the module level; the link credential comes from the `ActiveRedisConnection` `spec.secretName` | Mandatory. The instance binds a `RedisUser` through `spec.activeRedis.redisUserName`; `spec.secretName` is ignored | -| Peer transport | Proxy Redis Serialization Protocol (RESP) port (`6379`) | Dedicated peer port (`7379`) fronted by the proxy | +| Peer authentication | None at the module level. The link authenticates as the instance's default account, which the operator binds for you through `spec.activeRedis.redisUserRef` | Mandatory. The instance binds a `RedisUser` through `spec.activeRedis.redisUserRef` — normally its own default account, which must then carry the same password in every datacenter | +| Peer transport | Proxy Redis Serialization Protocol (RESP) port (`6379`) | The same proxy RESP port (`6379`) — one endpoint carries both transports. Active-Active additionally exposes the module peer port (`7379`) | | Persistence | RDB recommended | RDB only — `appendonly=yes` is rejected by admission | | Database count | Standard | `databases` must be `<= 16` | @@ -97,11 +97,15 @@ A proxy is deployed for both module generations. Enabling cross-datacenter repli | Port | Name | Purpose | Applies to | |:-----|:-----|:--------|:-----------| -| `6379` | `proxy` | RESP endpoint. Pre-flight inspection, and — on Redis `6.0` — the replication transport. Also the address used in mesh seed entries. | All versions | -| `7379` | `peer` | The peer port. On the new module this is the **only** peer transport: it carries both Disaster Recovery replication and Active-Active gossip. | Redis `7.2` | +| `6379` | `proxy` | RESP endpoint, and the replication transport for both module generations. The proxy tells a peer session apart from a client session and routes it accordingly, so one endpoint carries both. This is the address recorded in mesh seed entries and in an `ActiveRedisConnection`'s `spec.addresses`. | All versions | +| `7379` | `peer` | The module peer port, aliased onto the same proxy listener. It is added to the proxy Service **only in Active-Active mode**, because a mesh member gossips its own local peer port and something must answer on it. | Redis `7.2`, Active-Active only | - -When an external load balancer or firewall fronts the proxy of a Redis `7.2` instance, it must expose port `7379` as `7379`. The module always advertises its own local peer port for the data plane, so remapping it breaks replication. Only the RESP port may be remapped, through `announcePort` in Active-Active mode or `spec.peerPort` on an `ActiveRedisConnection`. + +The proxy Service of a `peerof` instance carries no `7379` port, and none needs to be opened between datacenters. A Disaster Recovery link addresses the proxy RESP endpoint alone — the address recorded in the downstream's `ActiveRedisConnection`. + + + +When an external load balancer or firewall fronts the proxy of an Active-Active member, it must expose port `7379` as `7379`. The module always advertises its own local peer port for that traffic, so remapping it breaks the mesh. Only the RESP port may be remapped, through `announcePort`. ### Instance ID: Service ID diff --git a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/20-setup.mdx b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/20-setup.mdx index 571e71d..ae18325 100644 --- a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/20-setup.mdx +++ b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/20-setup.mdx @@ -22,8 +22,8 @@ The setup procedure differs between the two module generations. Follow the secti | Redis version | Procedure | Link authentication | |:--------------|:----------|:--------------------| -| `7.2` | [Redis 7.2 — new module](#redis-72--new-module) | A `RedisUser` bound on each instance | -| `6.0` | [Redis 6.0 — legacy module](#redis-60--legacy-module) | A Secret referenced by the connection | +| `7.2` | [Redis 7.2 — new module](#redis-72--new-module) | The instance's `default` account, bound explicitly | +| `6.0` | [Redis 6.0 — legacy module](#redis-60--legacy-module) | The instance's `default` account, bound automatically | Replication between Redis `6.0` and Redis `7.2` is not supported. See [Module generations and version applicability](../10-intro.mdx#module-generations-and-version-applicability). @@ -31,59 +31,44 @@ Replication between Redis `6.0` and Redis `7.2` is not supported. See [Module ge ## Redis 7.2 — new module \{#redis-72--new-module} -The new module requires a **peer-auth credential**: every instance in the replication group binds a `RedisUser` through `spec.activeRedis.redisUserName`. The operator pushes that credential to every node, and both inbound and outbound peer links authenticate with it. Without it the module rejects all inbound peer replication. +The new module requires a **peer-auth credential**: every instance in the replication group binds a `RedisUser` through `spec.activeRedis.redisUserRef`. The operator pushes that credential to every node, and both inbound and outbound peer links authenticate with it. Enabling replication without a binding is rejected at admission, and the module rejects all inbound peer replication. - -The module accepts an inbound peer only when the username it presents equals the local peer-auth username. Every instance in the group must therefore bind a `RedisUser` with the **same `spec.username` and the same password value**. The `RedisUser` resource names may differ, but the credential values may not. +Bind the instance's own **default account**. Every instance already has one — its own reconcile creates it — so nothing has to be provisioned; the binding only names it. + + +The module accepts an inbound peer only when the credential it presents matches the local peer-auth credential. Binding the default account therefore requires every member of the group to carry the **same `default` password**. + +This is not automatic: default passwords are generated per instance. Set the same default password on every member before you wire the links. -The peer-auth `RedisUser` must satisfy all of the following, otherwise link creation is rejected: +### Step 1: Confirm the default account in each datacenter \{#step-1-create-the-peer-auth-credential-in-each-datacenter} -* `spec.redisName` references the local instance. -* `spec.accountType` is `custom` — the `default` and `system` accounts are refused, because binding them would let any client of that account act as a replication peer. -* `spec.passwordSecrets` lists at least one Secret carrying a `password` key. -* The Secret is **dedicated** to this `RedisUser`; sharing a password Secret across `RedisUser` resources is rejected by admission for peer-auth users. -* `status.phase` is `Success`. +The default-account `RedisUser` is named after the instance: -### Step 1: Create the peer-auth credential in each datacenter \{#step-1-create-the-peer-auth-credential-in-each-datacenter} +| Instance architecture | `RedisUser` name | +|:----------------------|:-----------------| +| Cluster | `drc-acl--default` | +| Sentinel, standalone | `rfr-acl--default` | -Perform these steps in **both** datacenters, using the same username and the same password value. +Check that it is provisioned in **both** datacenters — the binding is rejected unless its phase is `Success`: ```bash -# The dedicated password secret for the peer-auth account -$ kubectl -n default create secret generic s72-dc1-peer-secret --from-literal=password='Peer@Repl123' - -# The peer-auth RedisUser -$ cat << EOF | kubectl -n default create -f - -apiVersion: redis.middleware.alauda.io/v1 -kind: RedisUser -metadata: - name: s72-dc1-peer -spec: - accountType: custom - arch: sentinel - redisName: s72-dc1 - username: aapeerrepl - passwordSecrets: - - s72-dc1-peer-secret -EOF +$ kubectl -n default get redisusers rfr-acl-s72-dc1-default +NAME INSTANCE USERNAME PHASE AGE +rfr-acl-s72-dc1-default s72-dc1 default Success 3m ``` -> `spec.arch` must match the architecture of the referenced instance (`sentinel`, `cluster`, or `standalone`). - -Wait for the user to be applied: +The binding itself is set in Step 2, along with the rest of the replication configuration. -```bash -$ kubectl -n default get redisusers s72-dc1-peer -NAME INSTANCE USERNAME PHASE AGE -s72-dc1-peer s72-dc1 aapeerrepl Success 12s -``` + +Binding the default account widens what that password grants: anything holding it can act as a replication peer. A `custom` `RedisUser` narrows that, and lets the replication credential be rotated independently of the instance password. It must reference the local instance, carry a password Secret of its own that no other peer-auth `RedisUser` uses, reach `status.phase: Success`, and carry the **same username and the same password value in every datacenter**. A `system` account is always refused. + ### Step 2: Enable replication on the upstream instance ```bash $ kubectl -n default patch redis s72-dc1 --type=merge \ - --patch='{"spec": {"activeRedis": {"serviceID": 0, "mode": "peerof", "redisUserName": "s72-dc1-peer"}}}' + --patch='{"spec": {"activeRedis": {"serviceID": 0, "mode": "peerof", "redisUserRef": "rfr-acl-s72-dc1-default"}}}' ``` @@ -110,21 +95,23 @@ Each instance creates a proxy Service named `activeredis-proxy-`. ```bash $ kubectl -n default get svc activeredis-proxy-rfr-s72-dc1 -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -activeredis-proxy-rfr-s72-dc1 LoadBalancer 10.96.145.32 192.168.1.10 6379:31234/TCP,7379:32100/TCP 45s +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +activeredis-proxy-rfr-s72-dc1 LoadBalancer 10.96.145.32 192.168.1.10 6379:31234/TCP 45s ``` - -The proxy Service of a Redis `7.2` instance exposes two ports: `6379` (RESP) and `7379` (peer). Replication rides the peer port exclusively, so the downstream must be able to reach **both**. Port `7379` must be exposed unchanged — the module always advertises its own local peer port. Connection admission performs a TCP dial against the peer port and rejects the connection if it is unreachable. + +The proxy Service of a `peerof` instance exposes a single port, `6379`. The proxy carries the RESP control plane and the replication stream on it, telling the two apart per session, so the downstream needs to reach only that one address — the one you put in `spec.addresses`. No `7379` port is created in Disaster Recovery mode, and none has to be opened between datacenters. + +Connection admission dials the endpoint it resolves for the replication stream and rejects the connection if it is unreachable. With `spec.peerPort` left unset, that is the address in `spec.addresses[0]` itself. ### Step 4: Enable replication on the downstream instance -Repeat Step 1 in the downstream datacenter (same username, same password value), then enable replication with a **different** `serviceID`: +Confirm the downstream's default account as in Step 1, and make sure its default password matches the upstream's. Then enable replication with a **different** `serviceID`: ```bash $ kubectl -n default patch redis s72-dc2 --type=merge \ - --patch='{"spec": {"activeRedis": {"serviceID": 1, "mode": "peerof", "redisUserName": "s72-dc2-peer"}}}' + --patch='{"spec": {"activeRedis": {"serviceID": 1, "mode": "peerof", "redisUserRef": "rfr-acl-s72-dc2-default"}}}' ``` ### Step 5: Create the connection on the downstream side @@ -141,7 +128,6 @@ spec: instance: s72-dc2 addresses: - 192.168.1.10:6379 - peerPort: 7379 teardownPolicy: Detach EOF ``` @@ -149,9 +135,8 @@ EOF | Field | Description | |:------|:------------| | `instance` | The local (downstream) instance name. | -| `addresses` | The upstream proxy's RESP endpoint. The host of the first entry is also used for the peer port dial. | -| `peerPort` | The upstream proxy's peer port. Defaults to `7379`; override it only when an external load balancer exposes that port under a different number. | -| `secretName` | **Ignored on the new module.** The link authenticates with the instance-level peer-auth binding. | +| `addresses` | The upstream proxy's RESP endpoint. The first entry is what the replication stream dials. | +| `peerPort` | **Leave it unset.** The upstream proxy serves the replication stream and the RESP control plane on one endpoint, so the port of `addresses[0]` is used. Set it only when an external load balancer splits the two onto different numbers. There is deliberately no schema default. | | `teardownPolicy` | `Detach` (default) or `Decommission`. See [Removing a connection](#removing-a-connection). | @@ -267,16 +252,19 @@ Each disaster recovery instance will create a Proxy Service with a name that fol ```bash # Enable ActiveRedis - $ kubectl -n default patch redis s6-dest --type=merge --patch='{"spec": {"activeRedis":{"serviceID":0 }}}' + $ kubectl -n default patch redis s6-dest --type=merge --patch='{"spec": {"activeRedis":{"serviceID":1 }}}' ``` The range of `serviceID` is `[0-15]`. In the same disaster recovery cluster, the serviceID cannot be repeated. -#### Create a secret containing the password of the upstream `default` user +#### Wait for the peer-auth binding + + The operator adds the binding on a following reconcile, and the connection is rejected while it is still empty. Wait until this prints a name: ```bash - $ kubectl -n default create secret generic s6-src-auth-cred --from-literal=password=abc@123 + $ kubectl -n default get redis s6-dest -o jsonpath='{.spec.activeRedis.redisUserRef}' + rfr-acl-s6-dest-default ``` #### Configure Disaster Recovery Connection @@ -292,13 +280,14 @@ spec: addresses: - 192.168.1.10:30010 instance: s6-dest - secretName: s6-src-auth-cred EOF ``` - > Note to replace the upstream address and secret. + > Note to replace the upstream address. + + + A Redis `6.0` link authenticates as the instance's **default account**. The operator writes that down for you: the first time it reconciles an instance that has replication enabled and carries no binding, it sets `spec.activeRedis.redisUserRef` to the instance's own default-account `RedisUser` — `drc-acl--default` for a cluster instance, `rfr-acl--default` for a Sentinel one. Nothing is provisioned: that account already exists, and the password in it is the one the link uses. Both datacenters must therefore carry the same `default` password. - - `spec.secretName` is **required** on Redis `6.0` — it carries the credential the link authenticates with. The Secret must contain a `password` key, and may optionally contain a `username` key (the `default` user is assumed otherwise). + There is no credential field on the connection. `spec.secretName` has been removed, and a manifest that still sets it is rejected with `unknown field "spec.secretName"`. #### Check Disaster Recovery Connection Status @@ -325,7 +314,7 @@ spec: - 192.168.1.10:30010 instance: s6-dest pause: false - secretName: s6-src-auth-cred + teardownPolicy: Detach status: instance: s6-dest shards: @@ -380,7 +369,7 @@ Before a connection is accepted, the platform runs a set of pre-flight checks ag | Requirements inspect | Instance resource rule check | Need to ensure that the memory resources of the downstream instance must be greater than or equal to the upstream instance. | | Config inspect | Instance Service ID check | Need to ensure that in the replication group, the ServiceID is unique and within the range `[0-15]`. | -On Redis `7.2` the inspection dials with the peer-auth credential the data path will actually use, so a successful inspection also confirms that both datacenters carry the same credential. It additionally performs a TCP dial against the upstream peer port (`7379`). +On Redis `7.2` the inspection dials with the peer-auth credential the data path will actually use, so a successful inspection also confirms that both datacenters carry the same credential. It additionally performs a plain TCP dial against the endpoint it resolves for the replication stream — the address in `spec.addresses[0]`, unless `spec.peerPort` splits the two. ## Removing a connection \{#removing-a-connection} diff --git a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/30-failover.mdx b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/30-failover.mdx index 56d047c..c04a03c 100644 --- a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/30-failover.mdx +++ b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/30-failover.mdx @@ -76,8 +76,8 @@ spec: addresses: - 192.168.1.11:30011 instance: c6-dest - pause: true - secretName: c6-dest-conn-password + pause: false + teardownPolicy: Detach status: instance: c6-dest message: shard 0 status is Disconnected; shard 1 status is Disconnected; shard 2 diff --git a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/40-upgrade-6.0-to-7.2.mdx b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/40-upgrade-6.0-to-7.2.mdx index 1ed9c83..3f768cb 100644 --- a/docs/en/functions/95-disaster-recovery/20-disaster-recovery/40-upgrade-6.0-to-7.2.mdx +++ b/docs/en/functions/95-disaster-recovery/20-disaster-recovery/40-upgrade-6.0-to-7.2.mdx @@ -12,7 +12,7 @@ title: Upgrade from Redis 6.0 to Redis 7.2 This procedure is a **draft**. It describes the upgrade path implemented in Alauda Cache Service E1 v5.1.0, but it has not yet completed production validation. Rehearse it in a non-production environment, take a verified backup of every member first, and contact Alauda support before running it against production data. -Redis `6.0` carries the frozen legacy replication module. Redis `7.2` carries the new module, which adds Active-Active mode, mandatory peer authentication, rotation-safe credentials, and the dedicated peer-port transport. Moving an existing Disaster Recovery group from `6.0` to `7.2` is an in-place version upgrade of every member, followed by re-wiring the links. +Redis `6.0` carries the frozen legacy replication module. Redis `7.2` carries the new module, which adds Active-Active mode, mandatory peer authentication, and rotation-safe credentials. Moving an existing Disaster Recovery group from `6.0` to `7.2` is an in-place version upgrade of every member, followed by re-wiring the links. ## What this upgrade involves @@ -26,8 +26,8 @@ Changes that take effect with the new module: | Area | Redis `6.0` | Redis `7.2` | |:-----|:------------|:------------| -| Link authentication | The `ActiveRedisConnection` `spec.secretName` | A peer-auth `RedisUser` bound with `spec.activeRedis.redisUserName` | -| Peer transport | Proxy RESP port (`6379`) | Proxy peer port (`7379`) | +| Link authentication | The instance's `default` account, bound automatically through `spec.activeRedis.redisUserRef` | A peer-auth `RedisUser` bound with `spec.activeRedis.redisUserRef`; a dedicated `custom` account is recommended | +| Peer transport | Proxy RESP port (`6379`) | The same proxy RESP port (`6379`) — one endpoint carries both transports | | Persistence | RDB recommended | RDB only — `appendonly=yes` is rejected | | Database count | Standard | `databases` must be `<= 16` | | Active-Active mode | Not available | Available | @@ -50,7 +50,7 @@ Before starting, confirm each of the following for **every** member of the group $ kubectl -n default get redis s6-dc1 -o jsonpath='{.spec.customConfig.databases}' ``` -5. **Port `7379` is reachable between datacenters.** The new module replicates exclusively over the proxy peer port. Update firewall rules and external load balancers to expose `7379` — unchanged, not remapped — on every member's proxy before you begin, so the links can be re-created at the end. +5. **The upstream proxy endpoint stays reachable.** The new module replicates over the same proxy endpoint the Redis `6.0` links already used, so a Disaster Recovery group needs no new port opened. Leave the existing firewall rules and load-balancer entries for the proxy's RESP port in place, so the links can be re-created at the end. 6. **Record the current topology.** Note each member's `serviceID`, and which member holds the `ActiveRedisConnection` pointing at which upstream. You will re-create these links. ```bash @@ -106,30 +106,30 @@ The ordering rule matters for later, within-generation upgrades of a **live** gr ### Step 4: Confirm the peer-auth binding -The new module requires a peer-auth credential on every instance. On the in-place upgrade of an instance that already had cross-datacenter replication enabled, the platform provisions one automatically — see [Automatic peer-auth provisioning](#automatic-peer-auth-provisioning). +The new module requires a peer-auth credential on every instance. On an instance that already had cross-datacenter replication enabled, the platform binds one automatically — see [Automatic peer-auth binding](#automatic-peer-auth-provisioning). Check what each member ended up with: ```bash -$ kubectl -n default get redis s6-dc1 -o jsonpath='{.spec.activeRedis.redisUserName}' -s6-dc1-aa-peer +$ kubectl -n default get redis s6-dc1 -o jsonpath='{.spec.activeRedis.redisUserRef}' +rfr-acl-s6-dc1-default -$ kubectl -n default get redisusers s6-dc1-aa-peer -NAME INSTANCE USERNAME PHASE AGE -s6-dc1-aa-peer s6-dc1 aapeerrepl Success 2m +$ kubectl -n default get redisusers rfr-acl-s6-dc1-default +NAME INSTANCE USERNAME PHASE AGE +rfr-acl-s6-dc1-default s6-dc1 default Success 8d ``` -If the `RedisUser` does not exist, or its phase is not `Success`, the automatic provisioning is on hold. Resolve it as described in [When automatic provisioning is on hold](#when-automatic-provisioning-is-on-hold) before continuing. +If the binding is empty, or the `RedisUser` phase is not `Success`, resolve it as described in [If the binding is not there yet](#when-automatic-provisioning-is-on-hold) before continuing. -The module accepts an inbound peer only when the username it presents matches the local peer-auth username. Automatic provisioning converges on the fixed username `aapeerrepl` in every datacenter, and derives the password from the shared connection Secret the group already used — so members that were wired to each other converge on the same value without coordination. +The module accepts an inbound peer only when the username it presents matches the local peer-auth username. The automatic binding uses each instance's `default` account, so members that were already replicating to each other converge on the same credential — that shared `default` password is exactly what their Redis `6.0` links were authenticating with. If you bound credentials manually on any member, verify that **every** member carries the same username and the same password value before creating any link. ### Step 5: Re-create the connections -Create the `ActiveRedisConnection` in each downstream cluster, now without `secretName` and with the peer port: +Create the `ActiveRedisConnection` in each downstream cluster. There is no credential field any more — the link uses the instance's peer-auth binding — and the upstream address is the same one the Redis `6.0` link used: ```bash $ cat << EOF | kubectl -n default create -f - @@ -141,12 +141,11 @@ spec: instance: s6-dc2 addresses: - 192.168.1.10:6379 - peerPort: 7379 teardownPolicy: Detach EOF ``` -Admission resolves the peer-auth binding, runs the pre-flight inspection with the credential the data path will use, and dials the upstream peer port. A failure at this point names the specific problem. +Admission resolves the peer-auth binding, runs the pre-flight inspection with the credential the data path will use, and dials the upstream endpoint. A failure at this point names the specific problem. ### Step 6: Verify @@ -166,35 +165,40 @@ Confirm that: Then restore normal client traffic. -## Automatic peer-auth provisioning \{#automatic-peer-auth-provisioning} +## Automatic peer-auth binding \{#automatic-peer-auth-provisioning} -The new module makes the peer-auth binding mandatory, which would otherwise block the very update that performs the upgrade. The platform handles this specific transition: +The new module makes the peer-auth binding mandatory, which would otherwise block the very update that performs the upgrade. The platform handles that by writing down the credential the Redis `6.0` link has been using all along. -1. When you change `spec.version` from `6.0` to `7.2` on an instance that **already** has cross-datacenter replication enabled and carries no `spec.activeRedis.redisUserName`, the defaulting webhook fills in the deterministic name `-aa-peer`. -2. The operator then mints a matching `RedisUser` with the fixed username `aapeerrepl`, plus a dedicated password Secret named `-aa-peer-secret`. The password **value** is copied from the instance's existing Redis `6.0` `ActiveRedisConnection` Secret — the credential the replication group already shared — so every datacenter converges on the same username and password independently. +A Redis `6.0` link never carried a username: its credential held a password and nothing else, so the module sent a one-argument `AUTH` that the proxy resolved to the instance's **default account**. That account *is* the link credential. The operator therefore binds the instance's own default-account `RedisUser`: - -Provisioning triggers only on this transition, and only for the deterministic `-aa-peer` name: +| Instance architecture | Bound `RedisUser` | +|:----------------------|:------------------| +| Cluster | `drc-acl--default` | +| Sentinel, standalone | `rfr-acl--default` | + +**Nothing is provisioned.** That account already exists — the instance's own reconcile creates it — and the password in it is the one the link uses, so the binding changes no byte on the wire. + +The binding is applied on the first reconcile of any Redis `6.0` instance that has replication enabled and carries none, so it is normally already in place before you begin. If the version change arrives first, the defaulting webhook fills in the same name. -* a **fresh** instance created on Redis `7.2` with replication enabled must be given a binding explicitly — it is rejected without one; -* a mistyped manual binding surfaces as `RedisUser not found` rather than being silently created. + +* A **fresh** instance created on Redis `7.2` with replication enabled must be given a binding explicitly — it is rejected without one. +* A mistyped manual binding surfaces as `RedisUser not found` rather than being silently created. +* A binding that is already set is never overwritten. -### When automatic provisioning is on hold \{#when-automatic-provisioning-is-on-hold} + +Because it binds the `default` account, every member of the group must carry the **same `default` password** for the links to authenticate — which was already true of the Redis `6.0` links, and rotating that password now also rotates the replication credential. -The mint is skipped, and the binding points at a `RedisUser` that does not exist, in these cases: +To keep the two independent, create a dedicated `custom` `RedisUser` on every member — the same username and the same password value everywhere — and bind that instead; see [Step 1 of the Disaster Recovery setup](./20-setup.mdx#step-1-create-the-peer-auth-credential-in-each-datacenter). Do it before Step 5, and on every member. + -| Situation | Why | Resolution | -|:----------|:----|:-----------| -| The instance has no local `ActiveRedisConnection` carrying a Secret | There is nothing to derive the password from. This is normal for a pure upstream, whose downstreams' connection resources live in other clusters. | Create the peer-auth `RedisUser` manually and set `spec.activeRedis.redisUserName` to it. | -| The instance's connection Secrets diverge | The operator cannot tell which credential the group shares. | Create the peer-auth `RedisUser` manually with the group's intended credential. | -| The source Secret has no `password` key | There is no password to copy. | Create the peer-auth `RedisUser` manually. | +### If the binding is not there yet \{#when-automatic-provisioning-is-on-hold} -In every case the resolution is the same: create the credential yourself, exactly as described in [Step 1 of the Disaster Recovery setup](./20-setup.mdx#step-1-create-the-peer-auth-credential-in-each-datacenter), using the **same username and password value across every member**, and bind it: +The binding is deferred for one reason only: the default-account `RedisUser` does not exist yet. The instance's own reconcile creates it, and the binding lands on a following pass. If it is still empty once the instance is `Ready`, set it yourself: ```bash $ kubectl -n default patch redis s6-dc1 --type=merge \ - --patch='{"spec": {"activeRedis": {"redisUserName": "s6-dc1-peer"}}}' + --patch='{"spec": {"activeRedis": {"redisUserRef": "rfr-acl-s6-dc1-default"}}}' ``` ## After the upgrade diff --git a/docs/en/functions/95-disaster-recovery/30-active-active/20-setup.mdx b/docs/en/functions/95-disaster-recovery/30-active-active/20-setup.mdx index 4f166d9..6c092d9 100644 --- a/docs/en/functions/95-disaster-recovery/30-active-active/20-setup.mdx +++ b/docs/en/functions/95-disaster-recovery/30-active-active/20-setup.mdx @@ -21,42 +21,31 @@ Setting up Active-Active replication has two parts: enabling Active-Active mode * Each member's proxy is reachable from every other member on both port `6379` (RESP, used by seeds) and port `7379` (peer port, which carries both gossip and replication). * **The system clocks of all members are synchronized**, through a time-synchronization daemon (`ntpd`, `chronyd`, or the platform equivalent) on every node of every cluster in the group. This is a hard prerequisite, not a recommendation — see [Clock synchronization](./30-operations.mdx#clock-synchronization-requirement). -## Step 1: Create the peer-auth credential in each datacenter \{#step-1-create-the-peer-auth-credential-in-each-datacenter} +## Step 1: Confirm the default account in each datacenter \{#step-1-create-the-peer-auth-credential-in-each-datacenter} -Every peer link in an Active-Active group authenticates, and every member must present the **same username and the same password value**. Create the credential in each datacenter before enabling the mode. This is the same credential Disaster Recovery uses; if these instances already carry one, skip to Step 2. +Every peer link in an Active-Active group authenticates, and every member must present the same credential. Bind the instance's own **default account**: every instance already has one, created by its own reconcile, so nothing has to be provisioned. This is the same credential Disaster Recovery uses — if these instances already carry a binding, skip to Step 2. -```bash -# The dedicated password secret for the peer-auth account -$ kubectl -n default create secret generic s72-dc1-peer-secret --from-literal=password='Peer@Repl123' +The default-account `RedisUser` is named after the instance: -# The peer-auth RedisUser -$ cat << EOF | kubectl -n default create -f - -apiVersion: redis.middleware.alauda.io/v1 -kind: RedisUser -metadata: - name: s72-dc1-peer -spec: - accountType: custom - arch: sentinel - redisName: s72-dc1 - username: aapeerrepl - passwordSecrets: - - s72-dc1-peer-secret -EOF -``` - -> `spec.arch` must match the architecture of the referenced instance (`sentinel`, `cluster`, or `standalone`). +| Instance architecture | `RedisUser` name | +|:----------------------|:-----------------| +| Cluster | `drc-acl--default` | +| Sentinel, standalone | `rfr-acl--default` | -Wait for the user to be applied: +Check that it is provisioned in each datacenter — the binding is rejected unless its phase is `Success`: ```bash -$ kubectl -n default get redisusers s72-dc1-peer -NAME INSTANCE USERNAME PHASE AGE -s72-dc1-peer s72-dc1 aapeerrepl Success 12s +$ kubectl -n default get redisusers rfr-acl-s72-dc1-default +NAME INSTANCE USERNAME PHASE AGE +rfr-acl-s72-dc1-default s72-dc1 default Success 3m ``` - -The module accepts an inbound peer only when the username it presents equals the local peer-auth username. Every member of the group must therefore bind a `RedisUser` with the **same `spec.username` and the same password value**. The `RedisUser` resource names may differ, but the credential values may not. + +The module accepts an inbound peer only when the credential it presents matches the local peer-auth credential. Binding the default account therefore requires every member of the mesh to carry the **same `default` password**, and default passwords are generated per instance — set them to a common value before enabling the mode. + + + +A `custom` `RedisUser` narrows what the credential grants — anything holding the default password can otherwise act as a mesh peer — and lets the replication credential be rotated independently of the instance password. It must reference the local instance, own its password Secret, reach `status.phase: Success`, and carry the **same username and the same password value in every datacenter**. A `system` account is always refused. ## Step 2: Enable Active-Active mode on each instance @@ -73,7 +62,7 @@ spec: activeRedis: serviceID: 0 mode: mesh - redisUserName: s72-dc1-peer + redisUserRef: rfr-acl-s72-dc1-default proxy: service: type: LoadBalancer @@ -86,7 +75,7 @@ spec: |:------|:------------| | `mode` | `mesh`. | | `serviceID` | Unique within the mesh, range `[0-15]`, immutable. | -| `redisUserName` | The peer-auth `RedisUser` binding. Mandatory. | +| `redisUserRef` | The peer-auth `RedisUser` binding — normally the instance's default account. Mandatory. | | `proxy.service.announceAddress` | The externally-reachable host peers dial to reach this member — the host on the external load balancer or VIP, decoupled from the in-cluster Service. Must be a **bare host**: no scheme, no port, no path. | | `proxy.service.announcePort` | The externally-reachable RESP port for the gossip control plane. Defaults to the proxy Service port `6379`; override it only when an external load balancer exposes that port under a different number. | diff --git a/docs/en/functions/95-disaster-recovery/90-limitations.mdx b/docs/en/functions/95-disaster-recovery/90-limitations.mdx index 935a16a..a6e6ca8 100644 --- a/docs/en/functions/95-disaster-recovery/90-limitations.mdx +++ b/docs/en/functions/95-disaster-recovery/90-limitations.mdx @@ -103,7 +103,7 @@ Run a time-synchronization daemon (`ntpd`, `chronyd`, or the platform equivalent The proxy address itself is not highly available. If you expose the proxy via NodePort, place it behind a front-end load balancer to ensure HA; alternatively, use MetalLB to provide a highly available virtual IP for the proxy. -On Redis `7.2` the address must expose **both** the RESP port (`6379`) and the peer port (`7379`), and the peer port must not be remapped — the module always advertises its own local peer port for replication. +In Active-Active mode on Redis `7.2` the address must expose **both** the RESP port (`6379`) and the peer port (`7379`), and the peer port must not be remapped — the module always advertises its own local peer port for gossip. A Disaster Recovery link uses the RESP endpoint alone. ### Only IP connection addresses are supported (Redis 6.0) diff --git a/docs/en/release_notes.mdx b/docs/en/release_notes.mdx index 00f0f10..296ec13 100644 --- a/docs/en/release_notes.mdx +++ b/docs/en/release_notes.mdx @@ -30,13 +30,13 @@ Redis `7.2` instances now run a redesigned replication module, offering two mode * **Disaster Recovery** (`peerof`, the default) is the hot-standby topology, with directed links from one upstream to up to 15 downstream instances, each ready to be promoted if the upstream datacenter is lost. It remains available on Redis `6.0`. * **Active-Active** (`mesh`) lets every datacenter serve reads and writes locally, with conflicts resolved automatically and membership maintained by a gossip mesh seeded from a handful of addresses. It is available on Redis `7.2` only, and is driven by the new `ActiveRedisMesh` resource. -Replication on Redis `7.2` rides a dedicated peer port (`7379`) exposed by the instance proxy, which carries both replication and gossip. +Replication and gossip on Redis `7.2` ride the instance proxy, which tells the two transports apart on a single endpoint. Active-Active additionally exposes the module peer port (`7379`) on the proxy Service, so that mesh members can gossip. See [Cross-Datacenter Replication](./functions/95-disaster-recovery/10-intro.mdx). #### Mandatory Peer Authentication -Every Redis `7.2` instance with cross-datacenter replication enabled must bind a `RedisUser` as its peer-auth credential through `spec.activeRedis.redisUserName`. Peer links authenticate with this credential, which must carry the same username and password value in every datacenter. +Every Redis `7.2` instance with cross-datacenter replication enabled must bind a `RedisUser` as its peer-auth credential through `spec.activeRedis.redisUserRef` — normally the instance's own default account, which needs no provisioning. Peer links authenticate with this credential, so every datacenter must carry the same username and password value; a dedicated `custom` account may be bound instead. The credential is delivered to the nodes at runtime and is never written into the instance configuration file. Peer links no longer freeze it at wiring time, so rotating the password no longer breaks a link on its next reconnect.