Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ include::third-party:partial$nav.adoc[]
**** xref:learn:clusters-and-availability/xdcr-conflict-logging-feature.adoc[XDCR Conflict Logging]
***** xref:learn:clusters-and-availability/xdcr-viewing-conflict-logs.adoc[Viewing Conflict Logs]
**** xref:learn:clusters-and-availability/xdcr-active-active-sgw.adoc[XDCR Active-Active with Sync Gateway]
**** xref:learn:clusters-and-availability/xdcr-support-for-cng.adoc[XDCR Support for Cloud Native Gateway]
*** xref:learn:clusters-and-availability/groups.adoc[Server Group Awareness]
* xref:learn:security/security-overview.adoc[Security]
** xref:learn:security/authentication.adoc[Authentication]
Expand Down
120 changes: 20 additions & 100 deletions modules/backup-restore/pages/enterprise-backup-restore.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ which is available for both Couchbase Server _Enterprise Edition_ and Couchbase

[NOTE]
====
`cbbackupmgr` is _not_ backward compatible with backups created by means of `cbbackup`.
`cbbackupmgr` is not backward compatible with backups created by means of `cbbackup`.
Comment thread
supritha-kumar marked this conversation as resolved.

In Couchbase Enterprise Server 7.2 and after, `cbbackupmgr` is available in the `Tools` package that must be downloaded.
In Couchbase Enterprise Server 7.2 and later, `cbbackupmgr` is available in the `Tools` package that must be downloaded.
See xref:cli:cli-intro.adoc#server-tools-packages[Server Tools Packages].
====

Expand All @@ -30,10 +30,10 @@ Administrators are therefore recommended to define plans for both https://en.wik

All backup is stored in and recovered from a [.term]_Backup Repository_.
In turn, a [.term]_Backup Repository_ is stored in a Backup Archive on the filesystem.
Each backup job in the [.term]_Backup Repository_ stores its backup in two ways:
Each backup job in the [.term]_Backup Repository_ stores its backup in 2 ways:

* All bucket data is stored in a small, secondary database.
* All bucket creation scripts and configuration files are stored on the file system, as files.
* You can store all bucket data in a small, secondary database.
* You can store all bucket creation scripts and configuration files on the filesystem, as files.

=== What's Backed Up

Expand Down Expand Up @@ -64,15 +64,15 @@ NOTE: Backups that include users contain the user's hashed passwords.
Other flags let you exclude specific metadata, or select a subset of data to back up.
See xref:backup-restore:cbbackupmgr-config.adoc[cbbackupmgr config] for a list of the arguments you can use to control what `cbbackupmgr` backs up.

You can also use command line flags to control how the `cbbackupmgr restore` command restores data.
You can also use command line flags to control how the `cbbackupmgr restore` command restores data.
For example, use `--overwrite-users` to have `cbbackupmgr` overwrite existing users and groups in the database if the backup contains a matching user or group.
By default, `cbbackupmgr` does not overwrite existing users in the database.
Instead, it restores just the users in the backup that do not exist in database.
See xref:backup-restore:cbbackupmgr-restore.adoc[cbbackupmgr restore] for a list of the arguments you can use to control what `cbbackupmgr` restores.

=== Tool Locations

When installed as part of the Couchbase Server install, `cbbackupmgr` tool is stored with all other tools in the following _per platform_ locations:
When installed as part of the Couchbase Server install, `cbbackupmgr` tool is stored with all other tools in the following per platform locations:

.Backup Tool Locations
[cols="1,5"]
Expand Down Expand Up @@ -102,132 +102,52 @@ Each of the subsequent, incremental backups take a fraction of the time taken by
The backup archive is a directory that contains a set of backup repositories as well as logs for the backup client.
The backup directory should be modified only by the backup client, and any modifications that are not done by that client might result in a corruption of backup data.

Only one backup client can access the backup archive at one time.
Only 1 backup client can access the backup archive at 1 time.
If multiple instances of the backup client are running on the same archive at the same time, this might result in corruption.
To prevent such corruption instances, you may be required to create multiple backup archives depending on your use case.

[#version-compatibility]
== Version Compatibility

For 6.5 and all later versions, `cbbackupmgr` can be used to back up data either from a cluster running its own version, or from a cluster running a prior, _compatible_ version.
For example, the 6.6.0 tool can back up data from a cluster running 6.6.0, 6.5.x, 6.0.x, or 5.5.x.
It can also be used to restore _to_ any of those versions data previously backed up _from_ any of those versions.
For 7.2 and later versions, you can use `cbbackupmgr` to back up data either from a cluster running its own version, or from a cluster running a prior, `compatible` version.
You can also use it to restore data to any of these versions from backups created on any of them.

The following table lists the compatible cluster-versions for each version of `cbbackupmgr`.
Unless otherwise specified, backup and restore apply both to _local_ and to _cloud_ data.

.Compatibility Requirements for Backup and Restore
[cols="5,3,3,3,3,3,3,3,3"]
[cols="5,3,3,3"]
|===
| *cbbackupmgr version*
| *8.0*
| *7.6*
| *7.2*
| *7.1*
| *7.0*
| *6.6*
| *6.5.x*
| *6.0.x*
| *5.5.x*

| 7.6
| ✓
| 8.0
| ✓
| ✓
| ✓
| ✓
|
|
|

| 7.2
| 7.6
|
| ✓
| ✓
| ✓
| ✓
|
|
|

| 7.1
|
| 7.2
|
|
| ✓
| ✓
| ✓
| ✓*
|
|


| 7.0
|
|
|
| ✓
| ✓
| ✓*
| ✓*
|


| 6.6.0 and above
|
|
|
|
| ✓
| ✓*
| ✓*
| ✓*


| 6.5
|
|
|
|
|
| ✓
| ✓
| ✓


| 6.0.x
|
|
|
|
|
|
| ✓
|


| 5.5.x
|
|
|
|
|
|
|
| ✓


|===

+*+ For local backup only — _not_ for cloud.

.Restoring metadata and users
****
== Restoring metadata and users

* When restoring metadata to a newer Server version,
if the feature that the metadata applies to no longer exists in the newer Server version, then the metadata may not be restorable.

* If the user roles no longer exist in the version that you wish to restore to, then an error will be logged for the target user.
* If the user roles no longer exist in the version that you want to restore to, then an error is logged for the target user.

* In general, if you can upgrade _directly_ to the new version, then you should be able to restore the users.
* In general, if you can upgrade to the new version, then you should be able to restore the users.
If you cannot upgrade directly, then restoring users may cause errors.
For example, if some of the user roles no longer exist in the newer Server version.

****
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
= XDCR Support for Cloud Native Gateway
:description: pass:q[Learn how to configure and use XDCR with Cloud Native Gateway (CNG) to replicate data across Couchbase clusters using gRPC instead of using direct node-to-node connections.]
:page-edition: Enterprise Edition
:page-toclevels: 3

[abstract]
{description}

This feature allows XDCR to replicate data to a target cluster through its CNG interface, using gRPC over HTTPS instead of direct node-to-node Couchbase connections.

This document covers the concepts, configuration, usage, and behavior of XDCR over CNG for Couchbase Server administrators and operators.
For information about CNG deployment, see xref:cloud-native-gateway::intro/about-cng.adoc[About Cloud Native Gateway].

IMPORTANT: This feature is available in Couchbase Server Enterprise Edition only.
It's not available in Couchbase Server Community Edition.

== Cloud Native Gateway in Couchbase

The Couchbase Cloud Native Gateway (CNG) provides a gRPC interface to Couchbase services so that Couchbase clients do not need to be aware of cluster topology details.
CNG uses a set of RPC network endpoints based on gRPC over HTTP/2, an efficient binary wire protocol suited to the high performance of Couchbase Server.

CNG allows the details of a cluster to be hidden behind a single service endpoint.
Applications and clients connect to a single hostname and port and are load-balanced across that service.
Cloud Native Gateway handles cluster topology changes transparently, so nothing changes within the cluster itself when clients connect through CNG.

== Deploying CNG

In Kubernetes environments, Cloud Native Gateway is deployed as a sidecar image alongside each Couchbase Server node.
The sidecar is managed by the Couchbase Autonomous Operator (CAO).
Kubernetes load balancers and OpenShift routes can be configured to expose the CNG service to clients outside the cluster.

== Using CNG for XDCR

Couchbase clients, including XDCR, are smart clients that are cluster topology-aware and connect directly to the node where data is located.
In cloud environments, this requires exposing multiple ports on every node across an entire cluster, which creates significant complexity and friction in the following scenarios:

* OpenShift and Kubernetes environments:
OpenShift Routes and Kubernetes Ingress objects only support HTTP/HTTPS traffic.
With CNG, instead of a custom solution to expose all of the cluster's nodes externally, you can use the standard approaches for exposing cluster services.
For example, you can use OpenShift Routes and Kubernetes LoadBalancer to just expose CNG.
For more information, see xref:cloud-native-gateway::intro/about-cng.adoc[About Cloud Native Gateway].
* Private Link environments:
Cloud Service Provider (CSP) private link services expect a single load-balanced resource.
The configuration required for direct XDCR node-to-node connections is complex and difficult to maintain.
* General cloud networking:
Any network interface or proxy that supports HTTPS or gRPC works naturally with CNG, whereas direct XDCR connections require custom solutions that are environment-specific and fragile.

CNG allows the XDCR connection to flow over a single gRPC interface, eliminating the need to expose the target cluster topology and simplifying all of the above networking scenarios.

== CNG for Incoming Connections

CNG fronts incoming connections to a cluster.
When a target cluster has CNG deployed, incoming XDCR connections from a source cluster can connect through CNG.

NOTE: For bidirectional XDCR using CNG, both clusters must have CNG installed, since each cluster acts as a target for traffic from the other.

== Version and Environment Requirements

This section outlines the version and environment requirements for deploying and using XDCR over Cloud Native Gateway.

=== Version Requirements

The following version requirements apply to XDCR over CNG:

[cols="1,1,2",options="header"]
|===
|Replication Role |Component |Requirement

|Source cluster
|Couchbase Server
|8.5

|Source cluster
|Cloud Native Gateway
|Not required on source cluster

|Target cluster
|Couchbase Server
|Any version that supports CNG (earlier versions are supported as target; however, both clusters must support Couchbase Server 8.5 or later for bidirectional CNG replication)

|Target cluster
|Cloud Native Gateway
|CNG version 1.2.1 or later
|===

IMPORTANT: The Couchbase Server on the source cluster must be running Couchbase Server 8.5 or later, as the XDCR in the Server must understand the gRPC protocol and support Cloud Native Gateway as a target.
The target cluster must have Cloud Native Gateway version 1.2.1 or later, as 1.2.1 includes XDCR support.

=== One-Way vs Two-Way Replication Requirements

[cols="1,2,2",options="header"]
|===
|Replication Type |Source Cluster Requirements |Target Cluster Requirements

|One-way (unidirectional)
|8.5
CNG not required.
|CNG with XDCR support installed.
Server version can be any version that works with CNG 1.2.1.

|Two-way (bidirectional)
|8.5
CNG with XDCR support installed (acting as a target).
|8.5
CNG with XDCR support installed (acting as a target).
|===

NOTE: In a bidirectional setup, each cluster is both an XDCR source and a target.
As a source, it must be running Couchbase Server version 8.5.
As a target, it must have CNG version 1.2.1 or later installed.

== Limitations

The following limitations apply to the initial release of XDCR support for CNG:

* *HLV-enabled buckets*: Buckets that have `enableCrossClusterVersioning` set to `true` cannot be used with XDCR over CNG.
Attempting to create a replication with an HLV-enabled bucket returns an error.
* *xdcrDiffer*: The `xdcrDiffer` tool does not support CNG in Phase 1.
Attempting to use `xdcrDiffer` with a CNG remote reference returns an error.
* *Conflict Logging*: Conflict logging is not supported for XDCR over CNG since Conflict logging requires cross cluster versioning to be enabled on the bucket.
* *Sync Gateway Interop*: Mobile buckets (Sync Gateway / App Services) are only supported for one-way (uni-directional) XDCR replication with CNG since bidirectional support requires cross cluster versioning to be enabled on the bucket.

== How XDCR over CNG Works

This section explains how XDCR communicates with the target cluster through Cloud Native Gateway, including the connection model, protocol switching capabilities, and error handling mechanisms.

=== Connection Model

When a remote cluster reference is created with the `couchbase2://` prefix, XDCR transmits data over HTTPS using the gRPC protocol.
By default, this traffic uses port 18098 on the Cloud Native Gateway.

The CNG on the target cluster handles load balancing and routes traffic to the appropriate nodes and vBuckets within the target cluster.
Cluster topology changes in the target cluster are handled by CNG transparently.
The source cluster XDCR process does not need to track or update topology maps for the target.

=== Protocol Switching

XDCR supports dynamically switching a remote cluster reference between CNG-based and direct node-to-node connections.
When the remote reference for an existing replication is updated to change between these 2 protocols, for example, switching from `couchbase2://` to a direct cluster hostname or vice versa, the replication resumes using the existing XDCR checkpoints.
Comment thread
supritha-kumar marked this conversation as resolved.
No data needs to be re-replicated from the beginning.
For more information, see xref:manage:manage-xdcr/create-xdcr-reference.adoc#switching-protocols[Switching Protocols].

This means that in a multi-cluster XDCR topology, some replication legs can use CNG while others use direct node-to-node connections.

NOTE: Updating the remote reference's hostname or protocol may require updating the root certificate if the CNG certificate differs from the direct cluster certificate.

=== Error Handling

XDCR returns clear error messages in the following scenarios:

* If `couchbase2://` is specified but CNG is not installed or available on the target cluster, XDCR returns an error stating that CNG was not found on the target and advises trying without the `couchbase2://` prefix if direct node-to-node connections are available.
* If `couchbase2://` is not specified but direct node-to-node connections are failing and the target cluster has CNG enabled, XDCR returns an error suggesting that the user try the `couchbase2://` prefix.

XDCR does not attempt to automatically infer or switch the protocol.
The user is responsible for choosing the correct prefix and for acting on the error messages.

== Deployment Use Cases

This section describes common deployment scenarios for XDCR over Cloud Native Gateway, including self-managed clusters, Kubernetes and OpenShift environments, and hybrid deployments with Capella.

=== Self-Managed to Self-Managed

Two self-managed clusters, both running Couchbase Server 8.5, can replicate data using XDCR over CNG.
This enables bidirectional replication without requiring direct node-to-node network access between the clusters.

Typical requirements for this scenario:

* Both clusters run Couchbase Server 8.5 release or later.
* Both clusters have CNG version 1.2.1 or later installed.
* Remote references are created using the `couchbase2://` scheme and the hostname of the target Cloud Native Gateway.
* Remote references are configured with full TLS encryption (`secureType=full`).
* The clusters do not require alternate addresses or direct node-to-node network connectivity.

=== OpenShift Environment

With Cloud Native Gateway (CNG), instead of a custom solution to expose all of the cluster's nodes externally, you can use the standard approaches for exposing cluster services like OpenShift Routes and Kubernetes LoadBalancer and only expose the CNG service outside of the k8s environment.
For more information, see xref:cloud-native-gateway::intro/about-cng.adoc[About Cloud Native Gateway].

In an OpenShift deployment managed by the Couchbase Autonomous Operator (CAO), CNG is deployed as a sidecar to each Couchbase Server node.
Administrators define OpenShift routes that expose the CNG service to external clients.
XDCR from any Couchbase cluster can replicate into the OpenShift-hosted cluster by creating a remote reference with the `couchbase2://` scheme and the route address.

NOTE: CAO manages the installation and lifecycle of CNG in Kubernetes and OpenShift environments.
When creating XDCR remote references in a CAO-managed cluster, the `couchbase2://` scheme and CNG address should be allowed in the CAO XDCR configuration.

=== Capella to Self-Managed Over Public Network

A Capella cluster running Couchbase Server 8.5 can replicate data to a self-managed cluster using Cloud Native Gateway across the public network or private network using VPC Peering.
To use this scenario:

* The self-managed target cluster must have CNG installed and enabled.
* The self-managed CNG must allow connections from the Capella cluster nodes.
* On the Capella cluster, create a remote reference using the `couchbase2://` prefix and the self-managed cluster CNG hostname.
Loading