Add GitHub Actions workflows and provisioning scripts for cloud databases - #5863
Merged
Conversation
johnsimons
force-pushed
the
john/cloud_ci_tests
branch
from
September 8, 2026 23:56
2feaa22 to
3ebda7a
Compare
johnsimons
removed this pull request from stack #5864
September 9, 2026 01:34
johnsimons
added this pull request to stack #5866
September 9, 2026 03:12
johnsimons
force-pushed
the
john/cloud_ci_tests
branch
2 times, most recently
from
September 10, 2026 00:15
554681c to
99a4a3a
Compare
johnsimons
force-pushed
the
john/cloud_ci_tests
branch
from
September 11, 2026 04:24
72488f6 to
d8dd71f
Compare
rbev
approved these changes
Sep 11, 2026
| run: | ||
| shell: pwsh | ||
| concurrency: | ||
| # Deliberately not cancel-in-progress: cancelling a run mid-flight risks skipping the teardown |
Contributor
There was a problem hiding this comment.
There are absolutely still cases where this can and will happen such as an agent failing and timing out the run.
Generally you would just make sure the resources are somewhere predictable with a DeleteAfter tag on them so a daily cleanup can clear them out.
Member
Author
There was a problem hiding this comment.
Yes, you are right, and we are dealing with that situation too.
But in a successful run we want to clean-up even if another commit is pushed
…ase tests Introduces two workflows and supporting PowerShell scripts to run the persistence and acceptance test suites against managed cloud databases (Azure SQL, Azure Database for PostgreSQL, Aurora PostgreSQL, RDS SQL Server). The test workflow provisions a fresh database per run, executes the relevant test projects against it, and tears it down regardless of outcome. A nightly cleanup workflow sweeps any resources tagged `sc-cloud-test` that are older than four hours.
…ub Actions composite action The PowerShell scripts required an explicit teardown step in every workflow that used them, which could be skipped or forgotten if a job failed early. Replacing them with a JavaScript action (node24) allows declaring a post step, which GitHub runs automatically at job end regardless of outcome, making teardown impossible to omit. The action also folds in the stale security group sweep that was previously handled by a nightly cleanup workflow, so that workflow is removed. Each run cleans up what earlier runs left behind, keeping the account self-healing without a scheduled job. Other notable changes: Aurora PostgreSQL moves from db.t4g.medium to db.r6g.large and Azure PostgreSQL from a burstable B1ms to a General Purpose D4ds_v5 to avoid throttling mid-run; the run name now includes the attempt number so a retried run does not collide with its predecessor's still-deleting resources; and a verify-postgresql.cs script is added alongside the existing verify-sqlserver.cs to wait for PostgreSQL servers to become reachable before handing off to the test run.
…e PostgreSQL CLI flag Three independent fixes to the cloud database action: - AWS: instead of failing when a region has no default VPC, create one. RDS needs the default VPC's subnet group for publicly accessible instances, and the account may not have one in every region. Concurrent jobs racing to create it are handled by re-reading after the call. An IAM policy document is added alongside as a reference for the required permissions. - SQL Server verify script: only the initial TCP connection is retried. The previous loop retried database creation and Full-Text Search checks too, turning a real server-side failure into a silent ten-minute wait. The post-connection work is moved outside the retry loop, and `sys.databases` replaces `DB_ID` to handle Azure SQL's logical master database correctly. - Azure PostgreSQL: remove `--high-availability Disabled` because the CLI version on the runner does not accept the flag.
…g a fixed AWS region RDS requires a subnet group spanning at least two availability zones, but regions do not reliably have a default one even when a default VPC exists. This adds a `dbSubnetGroupName` helper that creates and reuses a named subnet group (`servicecontrol-cloud-tests`) as shared account infrastructure, matching the same concurrent-creation pattern already used for the default VPC. The workflow also switches RDS targets to a fixed region (`us-east-2`) rather than the shared `AWS_REGION` secret, which is at its VPC ceiling and cannot host publicly accessible RDS instances. A `tryRun` helper is added to `common.mts` to capture CLI failure output without throwing, allowing callers to distinguish a race-condition loss from a genuine error.
…r run concurrently Both AWS targets run at the same time using the same run name, so their security groups and RDS identifiers collide. Aurora PostgreSQL now appends `-aurora` to its resource names and SQL Server appends `-mssql`, making them distinct within the same run. Also fixes the Azure PostgreSQL database creation command, which requires `--name` rather than `--database-name` for the `flexible-server db create` subcommand.
General Purpose stores its transaction log on remote storage, capping log throughput at roughly one-tenth of what Business Critical provides on local SSD. The test suites create, index, and drop a schema per test around 550 times, so log throughput is the binding constraint—not CPU. A run on General Purpose measured 9 to 21 times slower than a local container and timed out. Business Critical keeps data and log on local SSD and resolves the bottleneck.
…ure-sqlserver All four managed database targets are upgraded to larger instance classes (db.r6g.2xlarge for Aurora PostgreSQL, db.m5.2xlarge for RDS SQL Server, Standard_D8ds_v5 with 512 GB storage for Azure PostgreSQL, BC_Gen5_8 for Azure SQL Server) to avoid CPU and IO throttling mid-run. The test suites are IO bound, and premium SSD IOPS on Azure PostgreSQL scale with disk size, so the storage is intentionally oversized. The azure-sql target is renamed to azure-sqlserver throughout—action inputs, TypeScript types, the workflow dispatch options, and the provisioner file itself—to match the naming convention of the other SQL Server target (rds-sqlserver).
RDS instances take minutes to finish deleting and hold their security group until then. Rather than failing, the deletion attempt now treats DependencyViolation as an expected transient condition and leaves the group for removeStaleSecurityGroups to clean up on a later run. Any other error is still re-thrown.
… as CLI arguments Passwords containing characters like `-` or `+` could be interpreted as CLI flags when passed as a separate argument. Switching to the `--flag=value` form prevents the CLI from misinterpreting the password value. Also removes `-` and `+` from the generated password's symbol set to avoid a leading `-` after shuffling, and ensures the first character is always a letter to satisfy provider constraints that require passwords to begin with an alphabetic character.
Converts cloud-database-tests.yml to a reusable workflow (workflow_call) and calls it from release.yml alongside the existing jobs, so cloud database tests run automatically on every release rather than requiring a manual trigger or branch/tag push.
johnsimons
force-pushed
the
john/cloud_ci_tests
branch
from
September 11, 2026 04:32
d8dd71f to
a5a1da1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces two workflows and supporting PowerShell scripts to run the persistence and acceptance test suites against managed cloud databases (Azure SQL, Azure Database for PostgreSQL, Aurora PostgreSQL, RDS SQL Server).
The test workflow provisions a fresh database per run, executes the relevant test projects against it, and tears it down regardless of outcome.
A nightly cleanup workflow sweeps any resources tagged
sc-cloud-testthat are older than four hours.