Repository navigation
fix(fs): support Azure Central US with cloud routing regressions - #76
Conversation
|
PTAL @qiffang |
srstack
left a comment
There was a problem hiding this comment.
Reviewed all 19 changed files across both commits. CI test is green (gofmt, go mod tidy, make test, make e2e, make build). I also applied the patch to a worktree locally: go test ./... passes, the new TestFSCloudRegionsUseBuiltInEndpoints (gcp + azure) and the installer tests pass, and I ran the DB probe below against the freshly built binary. The macOS-only WebDAV test cannot be exercised on my Linux host, so that part is verified by reading only.
Overall the Filesystem-side change is correct and well covered. I would like one decision on the MAJOR item before this merges.
[MAJOR] azure-centralus is now accepted by Starter/DB local validation
region.supported is shared by DB and FS: ResolveFS uses it (internal/api/endpoints/endpoints.go:136) and so does ResolveStarter (:100). Adding azure to supported (internal/config/region/region.go:50) therefore widens DB placement validation too.
Measured on the patched binary:
$ ti db create-db-cluster --db-cluster-type starter --db-cluster-name demo --region azure-centralus --dry-run
"region": { "name": "regions/azure-centralus" } # local validation passed, EXIT=0
$ ti ... --region azure-eastus --dry-run
ti [ERROR]: unsupported region "eastus" for cloud provider "azure"; supported regions: centralus # EXIT=2
README:60 ("Regions supporting TiDB Cloud Starter"), the Starter table in docs/priciples.md:69-80, and the published service matrix in ai/ti/reference/ti-regions-security-and-limitations.md all omit Azure (they also omit GCP). That matrix encodes "Starter supported / Filesystem not supported", so azure-centralus would be the first Filesystem-only case and is currently indistinguishable from a Starter-capable region at the CLI boundary.
To be fair, the drift predates this PR (gcp-us-east1 is in supported but not in the Starter docs), so this is not newly introduced. But Azure makes it permanent. Please pick one and state it in the PR description:
- Filesystem-only (preferred if there is no evidence that Starter has Azure Central US): split validation, e.g. a
region.ValidateFilesystemused byResolveFSandParsePlacementCode, keepingregion.ValidateforResolveStarter. DB commands then fail locally with a clear unsupported-region error whileti configurestill accepts azure. - If Starter really does support it, sync README:60, the Starter table in docs/priciples.md, the AGENTS.md placement table, and the published docs matrix.
[MINOR] No cross-check test between the built-in manifest and the region table
validateFSManifest (endpoints.go:253) only checks non-empty fields and duplicates; it never verifies that CloudProvider + TiDBRegion round-trip to RegionCode, and TestResolveFSUsesBuiltInRegionMapping iterates a hand-written table rather than the manifest itself. So adding a manifest entry without a matching region entry, or a typo (the v0.2.6 gcp-us-east-1 -> gcp-us-east1 class of bug), fails no test. Suggest a test that walks builtInFSRegionManifest().Regions, asserts CanonicalCode(CloudProvider, TiDBRegion) == RegionCode, and asserts each entry resolves through ResolveFS.
[MINOR] Telemetry backend must ship before or with the CLI
The new enum lives in internal/telemetrybackend/event.go (allowedRegions at :75, the oneOf(cloud_provider, ...) at :210), which is the separately deployed ti-telemetry-backend. The column is an unconstrained VARCHAR(32), so no migration is needed, but if the backend is not deployed first, Azure events are rejected as invalid cloud_provider / region_code and dropped (best-effort, no user impact). Please note the ordering in the PR description or release steps.
[MINOR] Published docs not updated
AGENTS.md requires syncing the English ti pages under docs/pingcap-docs/docs/ for user-visible changes, and this PR has no gitlink update. I checked ai/ti/reference/ti-regions-security-and-limitations.md at both the pinned submodule SHA and the current release-8.5 head: its region matrix lists only AWS x5 + Alibaba x1 (GCP is missing too). Either open the companion pingcap/docs change plus submodule bump, or state explicitly that docs ship separately.
[MINOR] mountready_test.go skip is broader than needed (commit 2)
The added os.ReadDir(mp) permission check sits before the two readiness assertions, so an EACCES on macOS skips the whole test. But defaultMountPointActive uses os.Stat + statfs (internal/fs/mountready_darwin.go:15) and needs no directory read permission, and per the comment above the test, catching the "any readable directory counts as ready" regression is exactly its purpose. Prefer tolerating the permission denial only on the final os.ReadDir(mp) assertion and keeping the readiness assertions running. Unrelated to Azure; fine to keep here if noted.
[NIT]
- internal/config/region/region_test.go:35 still uses the literal
{"azure", "eastus"}althoughProviderAzurenow exists. The case's meaning changed from "unknown provider" to "known provider, unknown region", and the added{"unsupported", "centralus"}preserves the old coverage - good. - The installer "Config regions" line (scripts/install.sh:280, scripts/install.ps1:117) now includes
gcp-us-east1, azure-centralus, which matches README:107/108 (an improvement), but that label previously meant the Starter list. Consider renaming it to "Supported placement regions" so it does not imply Starter support.
What looks good
- The routing path is correct end to end: the manifest matches on
cloud_provider/tidb_region->azure-centralus->https://azure-centralus.drive9.ai;fsEndpointputs the native code inRegionCodeand the canonical code inRegionName, andcreate-file-systempersistsendpoint.RegionName(canonical) locally, consistent with credential validation, mount locators, andDRIVE9_REGION_CODE. TestFSCloudRegionsUseBuiltInEndpointsis a strong test: pointingTI_TEST_FS_MANIFEST_URLat an unreachable address while leavingTI_ALLOW_TEST_ENDPOINTSempty proves the built-in mapping is used, and it also asserts that inheritedDRIVE9_*values are stripped and that token-only access persists nothing.- The fscred case covering "stored credential region wins over profile placement without mutating the profile", the
CanonicalCoderound-trip addition inTestParsePlacementCode, and the telemetry tests on both sides are all good.
Residual risk: live acceptance was manual (15/15 per the description) and e2e/live_test.go has no Azure case. Since the region is now built in, consider running make live-e2e-fs with an Azure profile as release acceptance.
|
Pushed
Behavior after the change:
Verified on this branch:
(The same policy was opened as #77 against |
srstack
left a comment
There was a problem hiding this comment.
4b7faa3 addresses the placement-validation concern from my earlier review:
azure-centralusis now a Filesystem-only placement:region.Validate(Starter) rejects it,region.ValidateFilesystemaccepts it, andResolveFSuses the Filesystem path.ti db ... --region azure-centralusfails locally with exit code 2 and an explicit Filesystem-only error instead of sendingregions/azure-centralusto the Starter API, whileti fsrouting is unchanged.ParsePlacementCode/CanonicalCodestill accept the placement, so--region,ti configure, profiles, and stored credentials keep working.- Starter-rejection coverage was added in
internal/config/region,internal/api/endpoints, andinternal/cli, and the azure case moved out of the Starter placement table.
CI test is green on the new commit. Two non-blocking follow-ups remain: roll out the ti-telemetry-backend enum change before or with the CLI release, and update the published regions page (separate docs repo/submodule), which currently lists neither GCP nor Azure. gcp-us-east1 was intentionally left unchanged.
|
Pushed
|
Signed-off-by: siddon <siddontang@gmail.com>
Signed-off-by: siddon <siddontang@gmail.com>
`region.supported` is shared by TiDB Cloud Starter and Filesystem resolution, so adding azure-centralus there also let `ti db` commands accept the placement locally and send `regions/azure-centralus` to the Starter API, which the product documentation does not list as a Starter placement. Mark the placement as Filesystem-only instead. `region.Validate` (Starter) now rejects it with `placement "azure-centralus" serves TiDB Cloud Filesystem only, not TiDB Cloud Starter`, while the new `region.ValidateFilesystem` accepts it and `ResolveFS` validates through that function. `ParsePlacementCode` and `CanonicalCode` keep accepting every known placement, so `--region`, `ti configure`, profiles, and stored credentials are unaffected. - README, docs/priciples.md, and AGENTS.md document the rule. - Tests cover the Filesystem-only rejection in internal/config/region, internal/api/endpoints, and internal/cli, and move the azure placement out of the Starter placement table. `gcp-us-east1` keeps its current behavior; applying the same model to it is a one-line addition to `filesystemOnly`.
Empty commit to force a fresh mergeability evaluation after the CLA status turned green and the reviewing account switched to APPROVED. No content change.
`gcp-us-east1` serves TiDB Cloud Filesystem only. The v0.2.5 release note and the product documentation already limit TiDB Cloud Starter to the AWS and Alibaba Cloud placements, but the shared placement table still let `ti db` commands accept the placement locally and send `regions/gcp-us-east1` to the Starter API. Add it to `filesystemOnly` so `region.Validate` rejects it with `placement "gcp-us-east1" serves TiDB Cloud Filesystem only, not TiDB Cloud Starter`, while `region.ValidateFilesystem` keeps routing `ti fs` to the built-in endpoint. README, docs/priciples.md, and AGENTS.md now list both Filesystem-only placements, and the Starter rejection tests cover gcp and azure.
718455f to
afa9a89
Compare
ti fs --region azure-centralus ...currently fails during local region validation even though the Azure deployment is available. Add Azure Central US placement and its built-in Filesystem endpoint so the normal CLI workflow can reach the service.azure-centralus.DRIVE9_*isolation, stored credentials under an AWS-default profile, and Azure telemetry acceptance.Validation:
make testpasses; this Mac skips the real WebDAV fixture when the OS denies access.make e2epasses, including both GCP and Azure binary routing cases.go mod tidyleavesgo.modandgo.sumunchanged; formatting and whitespace checks pass.ti fs, including 16 MiB multipart upload, SHA-256 download verification, range reads, recursive copies, and temporary-resource deletion. The repository's live inventory lifecycle also passed.