Repository navigation
fix(aws-elasticache): typed faults, control-plane lock, atomic creates - #1704
Merged
Merged
Conversation
…ts for snapshot and parameter-group ops
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This lays the foundation for the ElastiCache parity work in #1472. It doesn't add new operations. It fixes how errors reach the wire and makes the control plane safe under concurrent use.
Typed faults. Provider errors now carry their AWS fault code in a typed field (
Fault, withAWSFault()). Before, the code was a prefix in the message. The wire layer takes the code from that field, picks the HTTP status from a table that follows the botocore model, and writes the message with no code prefix. This replaces the old substring matching, which had two bugs:CacheClusterAlreadyExists; it is nowCacheParameterGroupAlreadyExists;CacheSubnetGroupNotFoundFaultcame back as 404; it is now 400, as in AWS.Control-plane lock.
Mockgets a mutex,mu, that guards the cluster, replication group, subnet group, parameter group and snapshot stores, and the tag map. The separate tag mutex is gone: every tag access already ran undermu. The lock order is documented on the struct: mu, then a settle set's lock. Settle sets have no callbacks. No lock is held across a call to monitoring, to the VPC subnet resolver or to a real cache engine.<group>-001is refused during that window. Creating a group whose member id is already used by a cluster returnsCacheClusterAlreadyExists.SnapshotandRestorerun undermu.Id reservation around real engines. A create reserves the id while the real engine is being provisioned. A delete keeps the id reserved until
Deprovisionreturns, so the same engine slot is never provisioned twice.creatingordeleting.InvalidCacheClusterStateorInvalidReplicationGroupState, and a create of the same id returns AlreadyExists.This also locks the remaining parameter-group operations and the snapshot operations, which covers #1690 and #1697. Each has its own race test. Tagging snapshot ARNs is still not supported and stays not-found (#1695). It adds
docs/coverage/nongoals/cache.md, which also notes two behaviours: a persisted snapshot leaves out resources that are stillcreatingordeleting, and a deleting group's member clusters drop out of DescribeCacheClusters first.Tests:
-racetests for duplicate creates of clusters, replication groups, subnet groups and parameter groups.TestModifyVsResetParameterGroupRace(AWS ElastiCache: lock the remaining parameter-group ops #1690) andTestSnapshotVsDeleteReplicationGroupRace, which runs snapshot create, copy and delete against deleting the group (AWS ElastiCache: lock the snapshot ops #1697).TestCreateCacheCollidingWithReservedGroupMember.TestWriteErrStripsCodeFromMessage,TestWriteErrStatusFromTableand an SDK test that checks the typed fault types, the status codes and prefix-free messages.Verified:
go build ./...passes.-race -count=3passes on the ElastiCache provider and server packages, plusTestWiringParity,persistandTestAWSErrorMessageShapeCompat.developmentbaseline.go generate ./...is clean after the commit.cloudemu serve --async-settle: create-cache-cluster and create-replication-group reportcreating; duplicates returnCacheClusterAlreadyExistsandReplicationGroupAlreadyExistswith messages that carry no prefix; a missing subnet group returns HTTP 400; delete then recreate works.aws_elasticache_parameter_group,aws_elasticache_clusterandaws_elasticache_replication_group: apply, plan shows no changes, update (parameter value and tags), plan shows no changes, destroy.Part of #1472