[ep-nag]

What this does

Adds community/modules/database/cloudsql, a general-purpose Cloud SQL module (MySQL or PostgreSQL) usable by any deployment. It creates an instance, database and user reachable only over private networking, and delivers the generated password through a Secret Manager secret. The password is never stored in Terraform state, plan files, outputs or startup scripts.

slurm-cloudsql-federation is unchanged. Moving Slurm accounting onto this module is left to a later PR, once this module has run in production.

• Connectivity: Private Service Access (default) or Private Service Connect (use_psc_connection). There is never a public IP (ipv4_enabled = false unconditionally). network_id is required only for a created instance over Private Service Access.
• Credentials: the password is generated as an ephemeral value and written only to the secret and the database user, through write-only arguments. The user's password is read back from the secret, so the two always match, including after an interrupted apply or a replaced secret. password_secret_accessors, or use of modules/project/service-account (the service_account_iam_email input), grants roles/secretmanager.secretAccessor on that one secret only, never project-wide. A user-supplied sql_password is supported but documented as landing in the deployment folder and the saved plan.
• Rotation: increase password_version. The module replaces the secret version and updates the user in the same apply. The old version is disabled rather than deleted, so a failed apply can be recovered by re-enabling it. The README explains how to destroy disabled versions, which are still billed.
• Adoption: with existing_instance_name the module creates only the database, user and secret on an existing instance. The instance must have a private IP (a plan-time postcondition), and adoption cannot be combined with PSC. To adopt an instance created in the same apply, pass the creating module's instance_name output. The adopted instance's database is abandoned on destroy, so its data stays.
• CMEK: encryption_key_name encrypts the instance, and user_managed_replication[].kms_key_name encrypts the secret's replica. Both take keys from the existing kms-key and kms-key-iam modules: use: [kms_key_iam] fills encryption_key_name and orders the instance after the IAM grant to the Cloud SQL service agent. The module contains no key-management resources.
• Other inputs: tier, edition, disk sizing, database flags, query insights, maintenance window, data cache.

Production defaults (created instance)

• deletion_protection = true and api_deletion_protection = true. Both checks run only when the instance itself is deleted, by which point Terraform has already removed the database, user, secret and PSC endpoint. So while either is on, a terraform_data destroy guard that depends on all of them fails the destroy before anything is removed. The database and user also default to ABANDON while protected. Teardown: set both to false, apply, then destroy.
• enable_backups = true and point_in_time_recovery = true (binary logging on MySQL), with retain_backups_on_delete = true. backup_start_time, retained_backups (at least 7, matching the 7-day transaction log retention) and final_backup_retention_days tune them.
• ssl_mode = "ENCRYPTED_ONLY". TRUSTED_CLIENT_CERTIFICATE_REQUIRED is accepted; the module does not issue client certificates.
• set_root_password = true: the built-in root/postgres user gets a random password that is never stored. It is set only at creation, so turning the flag off later cannot blank it.
• availability_type = "ZONAL". REGIONAL MySQL is rejected at plan time unless backups and binary logging are on, as Cloud SQL requires.
• PostgreSQL users default to ABANDON, because a user that owns objects cannot be dropped.

Files

• Module (new): community/modules/database/cloudsql/ (main.tf, variables.tf, outputs.tf, versions.tf, metadata.yaml, README.md). Outputs: instance_name, connection_name, self_link, private_ip_address, database_name, username, password_secret_id, password_secret_version.
• Example (new): community/examples/cloudsql-database.yaml, Private Service Access with a workload service account granted read on the secret through use.
• Test configs (new):
  • cloudsql-psc.yaml: PSC with the production settings (backups, retention, final backup, both protections, REGIONAL, ssl_mode, consumer projects, maintenance window).
  • cloudsql-postgres.yaml: PostgreSQL with database flags and query insights, plus an adopting module.
  • cloudsql-cmek.yaml: CMEK on the instance and the secret replica through kms-key and kms-key-iam.
• Daily test (new, scheduled): cloudsql-database (build, test definition, blueprint, test-validation/test-cloudsql.yml) and its Kueue lock entries. It is not in TO_SKIP, so list_tests.py is unchanged from develop.
• Registration and docs: modules/README.md and examples/README.md list the module and example. tools/validate_configs/test_configs/README.md describes the three configs. The kms-key-iam output description and README name cloudsql as a consumer of encryption_key_name (documentation only).

Daily test coverage

One run deploys:

• a MySQL instance over Private Service Access;
• a second module adopting it in the same apply;
• a PostgreSQL 15 instance over PSC;
• a client VM whose service account holds only per-secret read access.

It checks:

• no public IP, and the module reports the right private IP, including for the adopting module;
• the PostgreSQL instance is PSC-only;
• each password is read by the VM, and login works over TLS for all three users;
• unencrypted logins are refused (MySQL and PostgreSQL), and root cannot log in without a password;
• none of the three passwords is in Terraform state;
• rotation: re-apply with a new password_version, then version 1 is DISABLED, version 2 is ENABLED, the new password works and the old one is refused;
• protected destroy: with deletion protection on, a destroy is refused by the guard, the database is still there and login still works. Then protection is turned off and the deployment is torn down.

The MySQL instance name gets a per-attempt suffix, because a Kueue requeue reuses the build ID and Cloud SQL reserves a deleted instance's name for up to a week.

How the minimal Secret Manager role was derived

I deployed a real instance and impersonated a consumer service account that held nothing but a binding on that one secret:

• With roles/secretmanager.secretAccessor, the consumer could read the password.
• With the binding removed, the read was denied, and the API cited exactly secretmanager.versions.access.

So that role is both sufficient and necessary; no project-level or admin role is needed. Every live test since has used a VM whose only access is this per-secret binding.

The identity running Terraform also needs secretmanager.secrets.setIamPolicy on the secret, plus secretmanager.versions.access, because the module reads the version back on every plan. The first could not be verified independently, because the test identity holds roles/owner.

What's been tested

Local (current head):

• make tests passes: Go unit tests, golden copies, and validate_configs across 99 blueprints, including the four cloudsql blueprints.
• pre-commit passes on every changed file: terraform fmt and validate, tflint, terraform-docs, module-label-check, addlicense, yamllint, codespell.
• Plan-time checks were tested both ways:
  • PSC without a subnetwork; PSA without network_id;
  • a created instance without tier; PSC with adoption;
  • an empty PSC consumer list;
  • REGIONAL MySQL without backups or binary logging;
  • retained_backups below 7;
  • MySQL usernames over 32 characters (16 on MYSQL_5_6).
• Provider floor: google >= 7.13.0 (the first release with root_password_wo); validation fails on 7.12.0. random >= 3.7.1 (3.7.0 was never published).

Live (project ellenp-personal-project; each environment torn down afterwards, then the whole project swept):

• Daily test, current head (2026-10-13): the full test above, run in the CI test-runner image: exit 0, failed=0. Earlier versions of the test also passed on 2026-10-12 and twice on 2026-10-13.
• Production settings at the google 7.13.0 floor:
  • The instance is ZONAL with ENCRYPTED_ONLY, has backups with binary logging and a 03:00 start time, and has the module labels.
  • TLS login works, and the same credentials are refused unencrypted.
  • Destroy is refused while protected. Once unprotected, the final and retained automated backups survive deletion.
• Root password:
  • On the previous revision, turning set_root_password off applied an innocent-looking update that left root@% with no password.
  • On the current code, the same change plans no changes and root is refused.
• Protected destroy:
  • On a previous revision, a destroy with default protections dropped the database (including a test row), the user and the secret before the instance refused.
  • With the guard, the same destroy removed nothing: instance, database, user, enabled secret version, IAM grant, PSC rule and address all remained.
  • Turning protection off and applying then let a destroy remove all 13 resources.
• Rotation: the old version is disabled (not deleted), the new password logs in, and the old one is refused.
• REGIONAL MySQL without binary logging: Cloud SQL rejects it at apply (Error 400 ... binary logging enabled); the module now rejects it at plan time.
• API deletion protection: blocks destroy (The instance is protected) until set to false and applied.
• PostgreSQL over PSC: the endpoint is ACCEPTED, TLS login works through it, and an unencrypted connection is rejected (pg_hba.conf ... no encryption).
• Earlier revisions:
  • Same-apply adoption of MySQL and PostgreSQL (2026-10-12).
  • PostgreSQL user owning objects: dropping the user directly fails, while terraform destroy succeeds through ABANDON (2026-10-12).
  • Secret replacement: switching replication replaces the secret with no 409, and the user follows the new password (2026-10-12).
  • Unreadable version: recovery by re-enabling the version or rotating (2026-10-12).
  • CMEK on the secret replica: confirmed encrypted with the kms-key key, and rotation works in that setup (2026-10-07).
  • Instance CMEK through kms-key and kms-key-iam (2026-10-06).
  • Teardown with deletion_policy: ABANDON on private-service-access (repeated).

Gaps, maintainer decisions and future work

Gaps

• The daily test has not yet run under a Cloud Build trigger. It was run with the same playbook in the CI test-runner image.
• CMEK and REGIONAL are not in the daily test. CMEK was last verified live on an earlier revision; REGIONAL has only been tested as a rejection.
• The daily test checks that unencrypted logins are refused, but it does not verify the server's certificate (mysql --ssl).
• The PostgreSQL username length limit is not enforced (behaviour untested).

Maintainer decisions

• Placement: community/modules/database/cloudsql/ (community, experimental), alongside slurm-cloudsql-federation, rather than core modules/database/.
• Daily test is scheduled. Each run deploys two db-f1-micro instances and an e2-small VM for about 45 minutes.
• Destroy guard uses a destroy-time local-exec, so the machine running Terraform needs sh (true for gcluster and the Open Front End). There is precedent, for example in community/modules/scripts/gcloud.
• deletion_policy: ABANDON rather than REMOVE_PEERING on private-service-access in the example and tests. REMOVE_PEERING needs google >= 8.1.0, which no repo module requires. ABANDON is the existing convention and was verified to tear down cleanly.
• Required APIs: metadata.yaml lists sqladmin, servicenetworking, secretmanager and compute. The test_apis_enabled validator requires all four, even for PSC-only or adoption-only deployments; this is documented in the README.
• Transaction log retention is fixed at 7 days, the ENTERPRISE maximum, so switching edition keeps working.
• sql_password is not an ephemeral variable. gcluster writes blueprint values into the deployment folder, so marking it ephemeral would not keep it off disk. The README says to leave it unset.

Deferred work

None of the following is implemented in this PR, and nothing in this PR depends on them.

1. Move Slurm accounting onto this module. Re-point slurm-cloudsql-federation at it, with an unchanged interface and a zero-diff plan for existing deployments, once this module has run in production.
2. Move the Open Front End's application database onto this module. This was the original driver for this work. OFE would add this module to its deployment, grant its web server's service account read access to the password secret, and have Django read connection details from the outputs and the password from Secret Manager at startup, instead of using SQLite.
3. Check in OFE's Django migrations. This is a prerequisite for item 2: commit reviewed migrations and remove the boot-time makemigrations.
4. Cloud SQL IAM database authentication. Users of type CLOUD_IAM_SERVICE_ACCOUNT, the cloudsql.iam_authentication flag, and roles/cloudsql.instanceUser (plus roles/cloudsql.client through the Auth Proxy). Whether to add this is a maintainer call.
5. Run OFE as a least-privilege identity with the proposed service-account-profiles module (Improve functionality and documentation of Packer resource #228). It can read this module's secret through password_secret_accessors or its own secret_bindings; neither PR depends on the other.
6. Split adoption into a pre-existing-cloudsql module, following the toolkit's pre-existing-* convention.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a robust, production-ready Cloud SQL module designed for secure database deployments within the HPC Toolkit. By leveraging Secret Manager for ephemeral credential handling and enforcing private networking, the module minimizes security risks. It supports both new instance creation and the adoption of existing instances, with extensive documentation and automated daily testing to ensure reliability across various configurations.

Highlights

• New Cloud SQL Module: Introduced a general-purpose Cloud SQL module supporting both MySQL and PostgreSQL, enabling private networking and secure credential management.
• Security and Credentials: Implemented ephemeral password generation using Secret Manager, ensuring that database credentials are never stored in Terraform state or plain text.
• Connectivity Options: Added support for both Private Service Access and Private Service Connect, with mandatory private networking configurations.
• Testing and Validation: Added comprehensive daily integration tests and multiple test configurations (CMEK, PSC, PostgreSQL) to ensure module stability.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new community module cloudsql for creating private Cloud SQL instances (MySQL or PostgreSQL) with credentials managed securely via Secret Manager, along with corresponding examples, integration tests, and documentation updates. The review feedback highlights two key improvement opportunities: first, removing an overly restrictive validation on disk_size_gb that unnecessarily forces disk_autoresize to false; and second, updating the regex validator for network_id in metadata.yaml to allow empty strings, preventing validation failures when using Private Service Connect or adopting existing instances.

[github-actions]

this PR has been inactive for 2 days and has no unresolved comments. @ishitachail, @vikramvs-gg, please review.