Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1001,11 +1001,21 @@ Supported MVP placement values:
| `aws-ap-southeast-1` | AWS | Singapore |
| `alicloud-ap-southeast-1` | Alibaba Cloud | Singapore |
| `gcp-us-east1` | Google Cloud | South Carolina |
| `azure-centralus` | Microsoft Azure | Central US |

The prefix before the first `-` is the cloud provider selector. `aws` maps to
internal provider `aws`; `alicloud` maps to internal provider
`alibaba_cloud`; `gcp` maps to internal provider `gcp`. Keep this mapping
centralized in `internal/config/region`.
`alibaba_cloud`; `gcp` maps to internal provider `gcp`; `azure` maps to
internal provider `azure`. Keep this mapping centralized in
`internal/config/region`.

Filesystem-only placements are marked in
`internal/config/region.filesystemOnly`. `azure-centralus` is the current
Filesystem-only placement: `region.ValidateFilesystem` accepts it for
`ti fs`, while `region.Validate` keeps it out of TiDB Cloud Starter placement
validation so `ti db` commands fail locally with an explicit Filesystem-only
error. The built-in Filesystem endpoint mapping remains the authority on which
placements have a Filesystem endpoint.

Do not store secrets in logs, telemetry, generated docs examples, or test
fixtures.
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,9 @@ Automation should move to `TI_*` and `TIDB_CLOUD_*` environment variables. The v
### Configure

- Authentication: a TiDB Cloud Public Key and a Private Key from the [TiDB Cloud API Keys](https://tidbcloud.com/org-settings/api-keys) console. When either credential is missing, `ti` links to this page in its authentication error before suggesting `ti configure` or the corresponding environment variables.
- Default region: one of aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, alicloud-ap-southeast-1, or gcp-us-east1.
- Regions supporting TiDB Cloud Filesystem: aws-us-east-1, aws-ap-southeast-1, aws-us-west-2, alicloud-ap-southeast-1, or gcp-us-east1. These endpoints are built into `ti`; endpoint resolution does not download a Drive9 region manifest.
- Regions supporting TiDB Cloud Starter: aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or alicloud-ap-southeast-1.
- Default region: one of aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, alicloud-ap-southeast-1, gcp-us-east1, or azure-centralus.
- Regions supporting TiDB Cloud Filesystem: aws-us-east-1, aws-ap-southeast-1, aws-us-west-2, alicloud-ap-southeast-1, gcp-us-east1, or azure-centralus. These endpoints are built into `ti`; endpoint resolution does not download a Drive9 region manifest.
- Regions supporting TiDB Cloud Starter: aws-us-east-1, aws-us-west-2, aws-eu-central-1, aws-ap-northeast-1, aws-ap-southeast-1, or alicloud-ap-southeast-1. `azure-centralus` is a Filesystem-only placement, so `ti db` commands reject it during local placement validation.

Set up a default profile with one command:

Expand Down
2 changes: 1 addition & 1 deletion docs/priciples.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Supported TiDB Cloud Starter placement values are:
| `aws-ap-southeast-1` | AWS | Singapore |
| `alicloud-ap-southeast-1` | Alibaba Cloud | Singapore |

`aws` maps to the internal provider `aws`; `alicloud` maps to `alibaba_cloud`; `gcp` maps to `gcp`. TiDB Cloud Filesystem supports `aws-us-east-1`, `aws-ap-southeast-1`, `aws-us-west-2`, `alicloud-ap-southeast-1`, and `gcp-us-east1` through endpoint mappings built into `ti`.
`aws` maps to the internal provider `aws`; `alicloud` maps to `alibaba_cloud`; `gcp` maps to `gcp`; `azure` maps to `azure`. TiDB Cloud Filesystem supports `aws-us-east-1`, `aws-ap-southeast-1`, `aws-us-west-2`, `alicloud-ap-southeast-1`, `gcp-us-east1`, and `azure-centralus` through endpoint mappings built into `ti`. `azure-centralus` is a Filesystem-only placement: TiDB Cloud Starter commands reject it during local placement validation instead of sending an unsupported region to the Starter API.

## TiDB Cloud Authentication

Expand Down
2 changes: 1 addition & 1 deletion docs/telemetry-backend-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Allowed event fields:
| `exit_code` | integer | yes | 0 to 255. |
| `error_code` | string | no | Stable code only, max 64 bytes; empty string allowed. |
| `duration_ms` | integer | yes | 0 to 86,400,000. |
| `cloud_provider` | string | no | `aws`, `alibaba_cloud`, `gcp`, `unknown`, or empty. |
| `cloud_provider` | string | no | `aws`, `alibaba_cloud`, `gcp`, `azure`, `unknown`, or empty. |
| `region_code` | string | no | Known ti region code, `unknown`, or empty. |
| `cli_version` | string | yes | Version-safe characters only, max 64 bytes. |
| `os` | string | yes | Known Go `runtime.GOOS` value. |
Expand Down
53 changes: 53 additions & 0 deletions e2e/cli_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1077,6 +1077,59 @@ func TestFSRemoteInventoryAndIDCredentialSelectionAcrossCommandFamilies(t *testi
}
}

func TestFilesystemOnlyRegionRoutesFileSystemAndRejectsStarter(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("fake companion build path is covered by unit tests on Windows")
}
bin := tiBinary(t)
home := t.TempDir()
companion := filepath.Join(t.TempDir(), "ti-drive9")
build := exec.Command("go", "build", "-o", companion, "./testdata/fake-drive9.go")
build.Dir = "."
if output, err := build.CombinedOutput(); err != nil {
t.Fatalf("build fake Drive9 companion: %v\n%s", err, output)
}
recordPath := filepath.Join(t.TempDir(), "calls.jsonl")
const fileSystemID = "tenant-azure-sandbox"
token := drive9TestToken(fileSystemID)
env := []string{
"HOME=" + home,
"TI_PROFILE=default",
"TI_REGION_CODE=aws-us-east-1",
"TI_DRIVE9_BIN=" + companion,
"TI_FS_TOKEN=" + token,
"TIDB_CLOUD_PUBLIC_KEY=",
"TIDB_CLOUD_PRIVATE_KEY=",
"TI_ALLOW_TEST_ENDPOINTS=",
"TI_TEST_FS_MANIFEST_URL=http://127.0.0.1:1/must-not-be-used",
"FAKE_DRIVE9_RECORD=" + recordPath,
"DRIVE9_SERVER=https://wrong.example.test",
"DRIVE9_REGION_CODE=aws-us-west-2",
"DRIVE9_API_KEY=wrong-credential",
}
result := runTIWithInput(t, bin, "", env, "--region", "azure-centralus", "fs", "list-files", "--path", "/")
result.wantExitCode(0)
assertFakeDrive9Call(t, readFakeDrive9Calls(t, recordPath), []string{"fs", "ls"}, token, home, "default", fileSystemID, "https://azure-centralus.drive9.ai", "azure-centralus")
for _, name := range []string{"config", "credentials", "fs_credentials"} {
if _, err := os.Stat(filepath.Join(home, ".ti", name)); !os.IsNotExist(err) {
t.Fatalf("token-only access persisted %s: %v", name, err)
}
}

starterEnv := []string{
"HOME=" + t.TempDir(),
"TI_REGION_CODE=",
"TIDB_CLOUD_PUBLIC_KEY=test-public",
"TIDB_CLOUD_PRIVATE_KEY=test-private",
}
starter := runTIWithInput(t, bin, "", starterEnv,
"--region", "azure-centralus",
"db", "create-db-cluster", "--db-cluster-type", "starter", "--db-cluster-name", "ti-e2e-azure", "--dry-run",
)
starter.wantExitCode(2)
starter.wantStderrContains("Filesystem only")
}

func TestFSConfigurationFreeAccess(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("fake companion build path is covered by unit tests on Windows")
Expand Down
4 changes: 2 additions & 2 deletions e2e/installer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ fi
t.Fatalf("installer did not validate companion command surface %q; calls:\n%s", want, companionCallData)
}
}
for _, regionCode := range []string{"aws-us-east-1", "aws-ap-southeast-1", "aws-us-west-2", "alicloud-ap-southeast-1", "gcp-us-east1"} {
for _, regionCode := range []string{"aws-us-east-1", "aws-ap-southeast-1", "aws-us-west-2", "alicloud-ap-southeast-1", "gcp-us-east1", "azure-centralus"} {
if !strings.Contains(string(output), regionCode) {
t.Fatalf("installer did not list ti fs region %q:\n%s", regionCode, output)
}
Expand Down Expand Up @@ -294,7 +294,7 @@ func TestInstallersUseProductOwnedFSRegionsAndHideCompanionPath(t *testing.T) {
if strings.Contains(content, "ti fs companion installed to") {
t.Fatalf("%s still reports the companion installation path", path)
}
for _, regionCode := range []string{"aws-us-east-1", "aws-ap-southeast-1", "aws-us-west-2", "alicloud-ap-southeast-1", "gcp-us-east1"} {
for _, regionCode := range []string{"aws-us-east-1", "aws-ap-southeast-1", "aws-us-west-2", "alicloud-ap-southeast-1", "gcp-us-east1", "azure-centralus"} {
if !strings.Contains(content, regionCode) {
t.Fatalf("%s does not list ti fs region %q", path, regionCode)
}
Expand Down
3 changes: 2 additions & 1 deletion internal/api/endpoints/endpoints.go
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ func (r Resolver) ResolveIAM() (Endpoint, error) {
}

func (r Resolver) ResolveFS(provider, regionCode string) (Endpoint, error) {
if err := region.Validate(provider, regionCode); err != nil {
if err := region.ValidateFilesystem(provider, regionCode); err != nil {
return Endpoint{}, apperr.Wrap("config.invalid_region", "config", 2, err.Error(), err)
}

Expand Down Expand Up @@ -246,6 +246,7 @@ func builtInFSRegionManifest() *FSRegionManifest {
{RegionCode: "aws-us-west-2", Mode: DefaultFSMode, ServerURL: "https://aws-us-west-2.drive9.ai", CloudProvider: "aws", TiDBRegion: "us-west-2"},
{RegionCode: "alicloud-ap-southeast-1", Mode: DefaultFSMode, ServerURL: "https://alicloud-ap-southeast-1.drive9.ai", CloudProvider: "alicloud", TiDBRegion: "ap-southeast-1"},
{RegionCode: "gcp-us-east1", Mode: DefaultFSMode, ServerURL: "https://gcp-us-east1.drive9.ai", CloudProvider: "gcp", TiDBRegion: "us-east1"},
{RegionCode: "azure-centralus", Mode: DefaultFSMode, ServerURL: "https://azure-centralus.drive9.ai", CloudProvider: "azure", TiDBRegion: "centralus"},
},
}
}
Expand Down
24 changes: 24 additions & 0 deletions internal/api/endpoints/endpoints_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ func TestResolveFSUsesBuiltInRegionMapping(t *testing.T) {
{name: "AWS Oregon", provider: region.ProviderAWS, regionCode: "us-west-2", baseURL: "https://aws-us-west-2.drive9.ai", regionName: "aws-us-west-2"},
{name: "Alibaba Cloud Singapore", provider: region.ProviderAlibabaCloud, regionCode: "ap-southeast-1", baseURL: "https://alicloud-ap-southeast-1.drive9.ai", regionName: "alicloud-ap-southeast-1"},
{name: "GCP South Carolina", provider: region.ProviderGCP, regionCode: "us-east1", baseURL: "https://gcp-us-east1.drive9.ai", regionName: "gcp-us-east1"},
{name: "Azure Central US", provider: region.ProviderAzure, regionCode: "centralus", baseURL: "https://azure-centralus.drive9.ai", regionName: "azure-centralus"},
}
resolver := NewResolver()
if resolver.FSManifestURL != "" {
Expand All @@ -178,6 +179,29 @@ func TestResolveFSUsesBuiltInRegionMapping(t *testing.T) {
}
}

func TestResolveStarterRejectsFilesystemOnlyPlacement(t *testing.T) {
resolver := NewResolver()

endpoint, err := resolver.ResolveFS(region.ProviderAzure, "centralus")
if err != nil {
t.Fatalf("ResolveFS failed for the Filesystem-only placement: %v", err)
}
if endpoint.BaseURL != "https://azure-centralus.drive9.ai" || endpoint.RegionName != "azure-centralus" {
t.Fatalf("unexpected fs endpoint: %#v", endpoint)
}

if _, err := resolver.ResolveStarter(region.ProviderAzure, "centralus"); err == nil {
t.Fatal("expected Starter resolution to reject the Filesystem-only placement")
} else {
if got := apperr.ExitCodeFor(err); got != 2 {
t.Fatalf("expected exit code 2, got %d", got)
}
if message := apperr.MessageFor(err); !strings.Contains(message, "azure-centralus") || !strings.Contains(message, "Filesystem only") {
t.Fatalf("unexpected Starter error message %q", message)
}
}
}

func TestResolveFSUnsupportedManifestRegion(t *testing.T) {
resolver := Resolver{
FSManifest: &FSRegionManifest{
Expand Down
29 changes: 29 additions & 0 deletions internal/cli/root_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1147,6 +1147,35 @@ func TestRegionOverrideAllowsEnvironmentCredentialsWithoutEnvRegion(t *testing.T
}
}

func TestFilesystemOnlyRegionRejectedForStarter(t *testing.T) {
withConfigEnv(t)

_, _, err := executeForTest("--region", "azure-centralus", "db", "create-db-cluster", "--db-cluster-type", "starter", "--db-cluster-name", "demo-cluster", "--dry-run")
if err == nil {
t.Fatal("expected the Filesystem-only placement to fail for TiDB Cloud Starter")
}
if got := apperr.ExitCodeFor(err); got != 2 {
t.Fatalf("expected exit code 2, got %d", got)
}
if got := apperr.MessageFor(err); !strings.Contains(got, "azure-centralus") || !strings.Contains(got, "Filesystem only") {
t.Fatalf("unexpected error message %q", got)
}
}

func TestFilesystemOnlyRegionAcceptedForFileSystem(t *testing.T) {
withConfigEnv(t)

stdout, _, err := executeForTest("--region", "azure-centralus", "fs", "create-file-system", "--dry-run")
if err != nil {
t.Fatalf("expected Azure file system dry-run to succeed: %v", err)
}
for _, want := range []string{"azure centralus", "/v1/admin/tenants"} {
if !strings.Contains(stdout, want) {
t.Fatalf("Azure dry-run missing %q:\n%s", want, stdout)
}
}
}

func TestCreateClusterOmitsProjectSelection(t *testing.T) {
home := t.TempDir()
t.Setenv("HOME", home)
Expand Down
48 changes: 46 additions & 2 deletions internal/config/region/region.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,13 @@ const (
ProviderAWS = "aws"
ProviderAlibabaCloud = "alibaba_cloud"
ProviderGCP = "gcp"
ProviderAzure = "azure"

ProviderPrefixAWS = "aws"
ProviderPrefixAlibabaCloud = "alicloud"
ProviderPrefixAlibabaCloudLegacy = "ali"
ProviderPrefixGCP = "gcp"
ProviderPrefixAzure = "azure"
)

type Region struct {
Expand Down Expand Up @@ -45,19 +47,37 @@ var supported = map[string][]Region{
ProviderGCP: {
{Code: "us-east1", Label: "South Carolina"},
},
ProviderAzure: {
{Code: "centralus", Label: "Central US"},
},
}

// filesystemOnly marks placements that serve TiDB Cloud Filesystem without a
// TiDB Cloud Starter placement. TiDB Cloud Starter resolution rejects them
// through Validate, while Filesystem resolution accepts them through
// ValidateFilesystem; the built-in Filesystem endpoint mapping remains the
// authority on whether a Filesystem endpoint exists for a placement.
var filesystemOnly = map[string]struct{}{
placementKey(ProviderAzure, "centralus"): {},
}

func placementKey(provider, regionCode string) string {
return provider + "\x00" + regionCode
}

var providerPrefixes = map[string]string{
ProviderPrefixAWS: ProviderAWS,
ProviderPrefixAlibabaCloud: ProviderAlibabaCloud,
ProviderPrefixAlibabaCloudLegacy: ProviderAlibabaCloud,
ProviderPrefixGCP: ProviderGCP,
ProviderPrefixAzure: ProviderAzure,
}

var providerToPrefix = map[string]string{
ProviderAWS: ProviderPrefixAWS,
ProviderAlibabaCloud: ProviderPrefixAlibabaCloud,
ProviderGCP: ProviderPrefixGCP,
ProviderAzure: ProviderPrefixAzure,
}

func Providers() []string {
Expand Down Expand Up @@ -109,8 +129,11 @@ func DefaultRegion(provider string) (string, error) {
return regions[0].Code, nil
}

// CanonicalCode returns the canonical placement code for an accepted
// provider/region pair. It accepts every placement ti can address, including
// Filesystem-only placements such as azure-centralus.
func CanonicalCode(provider, nativeCode string) (string, error) {
if err := Validate(provider, nativeCode); err != nil {
if err := validatePlacement(provider, nativeCode); err != nil {
return "", err
}
prefix, ok := providerToPrefix[provider]
Expand All @@ -130,7 +153,7 @@ func ParsePlacementCode(code string) (Placement, error) {
if !ok {
return Placement{}, fmt.Errorf("unsupported cloud provider prefix %q in region code %q; supported prefixes: %s", prefix, code, joinProviderPrefixes())
}
if err := Validate(provider, nativeCode); err != nil {
if err := validatePlacement(provider, nativeCode); err != nil {
return Placement{}, err
}
regionLabel := ""
Expand All @@ -151,7 +174,28 @@ func ParsePlacementCode(code string) (Placement, error) {
}, nil
}

// Validate reports whether provider/regionCode is a supported TiDB Cloud
// Starter placement. A placement that only serves TiDB Cloud Filesystem is
// rejected with an explicit Filesystem-only error instead of being reported as
// unknown.
func Validate(provider, regionCode string) error {
if err := validatePlacement(provider, regionCode); err != nil {
return err
}
if _, ok := filesystemOnly[placementKey(provider, regionCode)]; ok {
return fmt.Errorf("placement %q serves TiDB Cloud Filesystem only, not TiDB Cloud Starter", providerToPrefix[provider]+"-"+regionCode)
}
return nil
}

// ValidateFilesystem reports whether provider/regionCode can be routed to a
// TiDB Cloud Filesystem endpoint. Whether an endpoint exists for the placement
// is decided by the built-in Filesystem region mapping.
func ValidateFilesystem(provider, regionCode string) error {
return validatePlacement(provider, regionCode)
}

func validatePlacement(provider, regionCode string) error {
regions, ok := supported[provider]
if !ok {
return fmt.Errorf("unsupported cloud provider %q; supported providers: %s", provider, joinProviders())
Expand Down
Loading
Loading