Repository navigation
v2.15.0 - #1717
Merged
Merged
v2.15.0#1717
Conversation
…KMS keys (#1655) Add DeleteRetentionPolicy, ListLogGroups, Associate/DisassociateKmsKey and kmsKeyId/logGroupClass on CreateLogGroup. PutLogEvents now drops events older than 14 days, past retention or more than 2h ahead and reports them in rejectedLogEventsInfo. Expired events are purged so GetLogEvents, FilterLogEvents and storedBytes agree. Part of #1476.
…ment and PutTargets input exclusivity (#1657) Validate Source, DetailType and Detail per entry, enforce the 1 MB request limit (256 KB in the regions without the increase), accept entries for a missing bus the way AWS does, keep result entries aligned with the request, and reject targets that combine Input, InputPath and InputTransformer.
…alue extraction (#1658) * fix(aws-cloudwatchlogs): space-delimited filter patterns and $field metric values Parse [field, ..., name=value] patterns for FilterLogEvents, metric filters, subscription filters and TestMetricFilter, resolve $name/$N/$.path metric values from the matched event, and stop accepting a literal space in %regex% bodies. Part of #1476. * fix(aws-cloudwatchlogs): allow a literal space in %regex% filter patterns The AWS syntax page's own example %^starting|^initializing|^shutting down% contains one.
…its and DescribeMetricFilters filters (#1660) * fix(aws-cloudwatchlogs): metric filter dimensions, default value, unit and validation Publish one datum per dimension set with the filter's unit, emit defaultValue for batches with no match, validate metric transformations and the 100-filter quota like PutMetricFilter, and serve DescribeMetricFilters account-wide with metric name/namespace filters and paging. Part of #1476. * fix(aws-cloudwatchlogs): publish metric filter matches as samples, default value once per period Each matching event is its own sample (StatisticValues per dimension set), defaultValue is published at most once per one-minute period and never for a period with a match, and $N metric values past a space-delimited pattern's fields are rejected.
* feat(eventbridge): archives and replay Add CreateArchive, DescribeArchive, UpdateArchive, DeleteArchive and ListArchives, with archives capturing matching events on PutEvents and honoring retention. Add StartReplay, DescribeReplay, CancelReplay and ListReplays: a replay re-delivers archived events in its time window to the listed rules (or every rule on the bus) with a replay-name field, and replayed events are never archived again. Archives and replays are included in snapshots. * fix(eventbridge): match replay FilterArns on the full rule ARN
…1661) * feat(aws-cloudwatchlogs): Logs Insights queries Add StartQuery, GetQueryResults, StopQuery, DescribeQueries, GetLogRecord and GetLogGroupFields with a query engine for fields, filter, sort, limit and display over system and JSON fields. Queries run on their first poll, StopQuery cancels a Running query, and malformed queries return MalformedQueryException with the error location. Part of #1476. * fix(aws-cloudwatchlogs): guard substr float args and make query admission atomic
…ets Manager (#1662) * feat(eventbridge): connections and API destinations Add CreateConnection, DescribeConnection, UpdateConnection, DeauthorizeConnection, DeleteConnection and ListConnections for BASIC, API_KEY and OAUTH_CLIENT_CREDENTIALS, each backed by a real Secrets Manager secret that is updated with the credentials and deleted with the connection. Add CreateApiDestination, DescribeApiDestination, UpdateApiDestination, DeleteApiDestination and ListApiDestinations, whose state follows the connection's authorization. Both are included in snapshots. * fix(eventbridge): keep connection credentials out of validation errors
…to fix/networkfirewall-parity-1455
…sh, per-subnet zones and IP type, subnet and tag validation)
* feat(eventbridge): deliver events to API destination targets Rule targets that are API destinations now call the endpoint over HTTP: path parameters, query and header parameters (connection values win), Basic, API key or OAuth client credentials auth with a cached token refreshed on 401/407, the destination's invocation rate limit, a 5-second timeout, retries per the target RetryPolicy on the codes EventBridge retries, and the dead-letter queue on failure. Target HttpParameters now round-trip. * fix(eventbridge): deliver to API destinations in the background * fix(aws-eventbridge): detach API destination delivery from the request context * test(providers): record EventBridge SetHTTPClient as a deliberate non-wiring
* fix(aws-cloudwatchlogs): Logs Insights stats, parse and dedup commands * fix(aws-cloudwatchlogs): show parsed fields in default Insights columns
…etricStatistics (#1703) An invalid or huge Period overflowed the bucket step and crashed the server with a divide by zero. Period now follows the AWS rule and returns InvalidParameterValue, buckets are counted in int64 seconds, and the math alarm lookback check no longer overflows.
…tion (#1704) * fix(aws-elasticache): typed faults, control-plane lock, atomic creates * fix(aws-elasticache): reserved group member ids, drop tagMu, race tests for snapshot and parameter-group ops
…ut limits, stream lifecycle states (#1650) * fix(aws-kinesis): live SubscribeToShard push, throttling metrics, input limits, stream lifecycle states - SubscribeToShard keeps the event stream open and pushes a new SubscribeToShardEvent whenever records land, for up to 5 minutes, until the client disconnects, or until the shard ends (final event carries ChildShards and no ContinuationSequenceNumber). The loop runs on the request goroutine; lifetime and poll interval are injectable for tests. - ContinuationSequenceNumber now captures the read position even when no records were delivered, and the shard's starting sequence number is accepted as AT/AFTER_SEQUENCE_NUMBER, so resuming does not replay. - Publish WriteProvisionedThroughputExceeded (PutRecord/PutRecords), ReadProvisionedThroughputExceeded (GetRecords) and PutRecords.ThrottledRecords per operation with value 0: the emulator enforces no shard throughput limits, so nothing is ever throttled. - PartitionKey longer than 256 characters is rejected with ValidationException on PutRecord and on PutRecords (whole request). - GetRecords Limit outside 1..10000 returns InvalidArgumentException instead of being clamped. - With --async-settle, CreateStream reports CREATING and DeleteStream reports DELETING for a settle window; only ACTIVE streams can be deleted. Default behaviour is unchanged. Part of #1513 * fix(aws-kinesis): race-free stream reaping, ACTIVE gating, snapshot ARN filter - removeStream now does compare-and-delete through memstore UpdateOrDelete for both the stream entry and its ARN index entry, so a reader purging an expired DELETING stream can no longer remove a stream that a concurrent CreateStream just installed under the same name (only reachable with --async-settle). - UpdateShardCount, Increase/DecreaseStreamRetentionPeriod, UpdateStreamMode, Start/StopStreamEncryption, SplitShard and MergeShards return ResourceInUseException unless the stream is ACTIVE, sharing DeleteStream's check. Without --async-settle every stream is ACTIVE, so nothing changes. - Snapshot serializes only the ARN index entries of streams it captures, so a DELETING stream left out of the snapshot no longer leaves a dangling entry. - Fold the PutRecords count and PartitionKey checks into validatePutRecordsRequest, bringing PutRecords back under the gocyclo limit. - Export driver.MaxGetRecordsLimit and use it in both the provider and the wire handler's Limit 0 message, so the bound lives in one place. * fix(aws-kinesis): SubscribeToShard idle heartbeats and prompt end on shutdown - An idle subscription now pushes an empty SubscribeToShardEvent (unchanged continuation) every 5s, as real enhanced fan-out does, so KCL 2.x no longer cancels and resubscribes on a quiet shard. - cloudemu serve stamps a shutdown signal (wire.ShutdownSignal) on every request's base context and closes it when http.Server.Shutdown begins. Open subscriptions watch it and end promptly instead of holding a graceful shutdown open until its timeout. Request contexts are not canceled, so ordinary in-flight requests are unaffected. - pushSubscription's wait and idle checks move into small helpers to keep it under the gocyclo limit.
…n/sort/filters, DescribeSecret region and access fields (#1651) * fix(aws-secretsmanager): replication ops, ListSecrets planned-deletion/sort/filters, DescribeSecret region and access fields - ListSecrets honors IncludePlannedDeletion (with DeletedDate), SortBy and SortOrder, real primary-region/owning-service prefix filters and '!' negation; unknown filter keys, SortBy/SortOrder values and out-of-range MaxResults return ValidationException, and NextToken is a cursor (InvalidNextTokenException when malformed). - Add ReplicateSecretToRegions, RemoveRegionsFromReplication and StopReplicationToReplica, and honor CreateSecret AddReplicaRegions, as replica metadata on the primary; a replicated primary can't be deleted. - DescribeSecret reports PrimaryRegion, ReplicationStatus and LastAccessedDate (day resolution, stamped by GetSecretValue). - Use the real AWS messages for operations on a secret scheduled for deletion and for CreateSecret over such a name. Part of #1516 * docs(coverage): Secrets Manager replication and ListSecrets non-goals Part of #1516 * refactor(aws-secretsmanager): bring ListSecrets and BatchGetSecretValue handlers under the gocyclo limit Move BatchGetSecretValue's request checks into validateBatchRequest and ListSecrets' per-entry enrichment (rotation info, DeletedDate) into listEntry. Both handlers were at cyclomatic complexity 11. * fix(aws-secretsmanager): validate replica Regions, close the delete/replicate race, stamp LastChangedDate - Reject a replica Region that doesn't match the API model's RegionType pattern (^([a-z]+-)+\d+$) with InvalidParameterException, on both CreateSecret AddReplicaRegions and ReplicateSecretToRegions, so a typo can't leave a phantom replica that blocks DeleteSecret. - DeleteSecret now checks for replicas under the same write lock that schedules (or force-marks) the deletion, so a concurrent ReplicateSecretToRegions can't leave a secret both deleted and replicated. - ReplicateSecretToRegions and RemoveRegionsFromReplication update LastChangedDate, which AWS defines as the last time the secret was modified in any way. * docs(coverage): document ListSecrets negated-value semantics and replica Region checks Values within one filter are OR'd, negated ones included; excluding several prefixes takes one filter per prefix. Pin that in a test.
…missing-keyspace error, GetTable omissions (#1645) * fix(aws-keyspaces): tag existence and restrictions, name validation, missing-keyspace error, GetTable omissions - TagResource/UntagResource/ListTagsForResource validate the ARN shape (ValidationException) and require an existing keyspace or table (ResourceNotFoundException); the trailing "/" is optional. - Tags follow the documented restrictions on every path that accepts them: key 1-128 and value 1-256 characters, allowed character set, reserved "aws:" prefix, and at most 60 tags per request. - Keyspace/table names enforce the documented 1-48 char pattern [a-zA-Z0-9][a-zA-Z0-9_]{0,47} (a leading digit is allowed); UDT names get the documented 48-char limit. - CreateTable/CreateType/RestoreTable on a missing parent keyspace return ResourceNotFoundException instead of ValidationException. - GetTable omits readCapacityUnits/writeCapacityUnits for PAY_PER_REQUEST and an unset cdcSpecification.viewType. - Seed the documented fourth system keyspace, system_schema_mcs, which likewise cannot be deleted. Part of #1512 * fix(aws): skip resources deleted mid-walk in tagged RGT discovery A resource deleted between a service's list call and its tag read makes the tag read return NotFound. project returned that error, which aborted DiscoverResources for every service. Keyspaces hit it once ListTagsForResource started rejecting unknown ARNs. Skip the vanished resource instead, as the resourcediscovery walkers already do; other errors still propagate. * fix(aws-keyspaces): copy tags under the lock in Snapshot, canonicalise tag ARNs on restore Snapshot handed json.Marshal the live tag maps after releasing the lock, so a concurrent TagResource raced it. Deep-copy them under the lock. Restore now keys tags by the canonical ARN of an existing keyspace or table. Tags an older build stored under the slash-less ARN form are merged in (the canonical entry wins on a clash), and entries for ARNs that name no resource are dropped, so they can't leak onto a resource created later under that ARN. * fix(aws-keyspaces): enforce the 60-item TagList bound in the provider TagResource, UntagResource, CreateKeyspace, CreateTable and RestoreTable now reject more than 60 tags on the library path too, matching the wire handler. Scope the omitEmptyEnum comment to cdcSpecification.viewType, and rename the parity test files to feature names (names_test.go, tags_test.go, tables_test.go).
…gs, grant principal validation (#1647) * fix(aws-kms): add DeriveSharedSecret and custom key stores, reject aws: tags and malformed grant principals - DeriveSharedSecret (ECDH): CreateKey accepts KeyUsage=KEY_AGREEMENT for ECC_NIST_P256/P384/P521 and advertises KeyAgreementAlgorithms=[ECDH] in KeyMetadata and GetPublicKey. The shared secret is computed with crypto/ecdh from the key's private half and the peer's DER SubjectPublicKeyInfo, so it equals what the peer derives locally. - Custom key stores: Create/Describe/Update/Delete/Connect/Disconnect CustomKeyStore for AWS_CLOUDHSM and EXTERNAL_KEY_STORE stores, with the documented name/cluster/proxy-URI uniqueness errors and connection-state rules. CreateKey takes CustomKeyStoreId/XksKeyId; keys in a disconnected store fail crypto with KMSInvalidStateException, rotation is unsupported for them, and a store holding keys can't be deleted. Stores persist in snapshots. - TagResource, CreateKey and ReplicateKey reject tag keys in the reserved aws: namespace (case-insensitive) with TagException. - CreateGrant rejects a GranteePrincipal/RetiringPrincipal that is not an IAM/STS principal ARN, service principal or account ID with InvalidArnException, and values outside the modeled pattern with ValidationException. Part of #1523 * docs(coverage): regenerate for KMS DeriveSharedSecret and custom key stores, add KMS non-goals Part of #1523 * fix(aws-kms): report Unavailable for keys in a disconnected store, tighten grant principals - Keys in a custom key store that is not CONNECTED now report KeyState Unavailable (Enabled false) on DescribeKey/ListKeys, except PendingDeletion keys. The state is derived on read, so EnableKey/DisableKey on an Unavailable key take effect once the store reconnects and snapshots keep the stored state. Crypto checks it first and fails with KMSInvalidStateException "<key ARN> is unavailable", also for a Disabled key. - CreateGrant principals must be iam root/user/role or sts assumed-role/federated-user ARNs with no Region and a 12-digit account. ValidationException messages now include the value, in the AWS format. - GenerateRandom honours CustomKeyStoreId: CustomKeyStoreNotFoundException, UnsupportedOperationException for an external key store, and CustomKeyStoreInvalidStateException for a store that is not connected. - CreateKey into a store names the single-Region symmetric rule in its UnsupportedOperationException message. - Drop the unused ExInvalidMarker and ExUnsupportedOperationForCustomStore constants; add driver.WithMessage for sentinel errors with a specific message. - Rename the issue-named test files by feature and add tests for the XKS connectivity switch and the custom key store list order.
…G_ID default (#1649) * fix(aws-efs): protection API, async lifecycle, scheduled metrics, LONG_ID default Close the open real-cloud parity gaps in #1519: - Add UpdateFileSystemProtection (PUT /2015-02-01/file-systems/{id}/protection) with the ENABLED | DISABLED | REPLICATING enum, BadRequest for other values, IncorrectFileSystemLifeCycleState when the file system is not available, and ReplicationAlreadyExists for a replication source or destination. DescribeFileSystems reflects the new setting. - Model the creating/deleting lifecycle of file systems and mount targets under AsyncSettle: create returns "creating", delete returns while "deleting", and the resource is gone (FileSystemNotFound / MountTargetNotFound) once the window elapses. Operations the EFS model marks with IncorrectFileSystemLifeCycleState are rejected until the file system is available. The default (AsyncSettle off) stays synchronous. The windows are persisted in snapshots. - Publish AWS/EFS metrics from the shared serve scheduler (EFS is now a config.Tickable, ticked before CloudWatch): StorageBytes on FileSystemId + StorageClass every 15 minutes, and PermittedThroughput once a minute for provisioned file systems. Metrics whose values cloudemu does not model are not published. - DescribeAccountPreferences reports LONG_ID for FILE_SYSTEM and MOUNT_TARGET when no preference was set, and PutAccountPreferences rejects SHORT_ID and unknown values with BadRequest. Part of #1519 * fix(aws-efs): copy state under lock in snapshots, atomic access-point create, report reaps from Tick Address the review on #1649: - Snapshot now copies each file system's mutable state while holding its lock: the mount-target and access-point maps and values, the mtReadyAt / mtDeleteAt lifecycle windows, and the file-system and access-point tag maps. It also copies the IP counters. Before, they were marshaled after the lock was released while requests and Tick kept writing them, which races and can crash with a concurrent map read and write. - CreateAccessPoint checks the lifecycle state under the same lock that stores the access point, so a concurrent DeleteFileSystem either sees the access point (FileSystemInUse) or has already marked the file system deleting. The immediate (default) delete also marks the removed record as deleting, so a create waiting on its lock is rejected instead of storing a child on a file system that is gone. - UpdateFileSystemProtection rejects REPLICATING with BadRequest. EFS sets that state on a replication destination; callers choose ENABLED or DISABLED. - Tick reports reaping a deleted file system or mount target as a state change, so serve persistence saves it, and it reaps even with no CloudWatch backend wired. - Restore turns a missing mount-target or access-point map (dropped by omitempty) into an empty one, so a file system restored with neither can take new ones instead of panicking on a nil map. Part of #1519
…enderId (#1539) * fix(aws-sts): report the role's RoleId in AssumedRoleId AssumeRole, AssumeRoleWithWebIdentity and AssumeRoleWithSAML returned the fixed AssumedRoleId prefix AROACLOUDEMU0000000000 for every role. Real STS returns "<role id>:<session name>", where the role id is the role's unique RoleId. Use the IAM role's RoleId when the role is known and keep the synthetic prefix only when no IAM driver knows it. GetCallerIdentity and the SQS SenderId derive from this id, so two roles with the same session name no longer collapse onto one identity. Part of #1514 * fix(aws-sqs): publish queue gauges every minute and resolve SenderId Publish the AWS/SQS queue-state gauges (ApproximateNumberOfMessagesVisible, ApproximateNumberOfMessagesNotVisible, ApproximateNumberOfMessagesDelayed, ApproximateAgeOfOldestMessage and, for FIFO queues, ApproximateNumberOfGroupsWithInflightMessages) from a Tick on the shared serve scheduler: one sample per active queue per minute of provider-clock time, where a queue is active while it holds messages or was accessed in the last six hours. A queue whose consumers stopped keeps reporting a growing backlog and age, and each minute holds one sample, so Sum/Average/SampleCount match real SQS. The counters stay on the request paths and every metric is published after the queue lock is released. The access and sample times are part of the SQS snapshot. GetQueueAttributes ApproximateNumberOfMessagesNotVisible now counts in-flight messages only, not delayed ones, matching the API reference and the gauge. SenderId is the caller's IAM unique id (user id, or role id:session for an assumed role) resolved by the shared awsidentity.Resolver, matching GetCallerIdentity's UserId; library and service-to-service sends keep the account id. Part of #1514
…pe check and job state-change events (#1648) * fix(aws-batch): add job lifecycle, scheduling policies, job definition type check and job state-change events Add SubmitJob, DescribeJobs, ListJobs, CancelJob and TerminateJob. Jobs move SUBMITTED -> PENDING -> RUNNABLE -> STARTING -> RUNNING -> SUCCEEDED or FAILED on the provider clock, honour dependsOn and array jobs, and follow the documented cancel vs terminate rules. SubmitJob validates the queue (exists, ENABLED), the job definition revision, dependencies, share identifier, array size, priority, timeout and tags. ListJobs supports the documented filters and pages with a stable cursor. Add scheduling-policy CRUD and validate schedulingPolicyArn on CreateJobQueue/UpdateJobQueue. Enforce the RegisterJobDefinition type enum (container, multinode) and the properties each type requires. Jobs and scheduling policies are taggable, and tags follow the TagResource limits and the reserved aws: prefix. Publish "Batch Job State Change" events to the EventBridge default bus from the shared serve scheduler (config.Tickable) and on job API calls. New state is included in snapshots. Part of #1496 * docs(coverage): regenerate for the Batch job and scheduling-policy operations Part of #1496 * fix(aws-batch): bound array child index to int32 when parsing job IDs Parse the array child index with strconv.ParseInt(..., 10, 32) so the value converted into ArrayProperties.Index is bounded at parse time (CodeQL go/incorrect-integer-conversion). lookup already rejected any index at or above the array size, so no out-of-range value reached the conversion; this makes the bound explicit. * fix(aws-batch): apply property overrides and address review findings - SubmitJob merges ecsPropertiesOverride, eksPropertiesOverride and nodeOverrides into the job detail instead of dropping them, and rejects an override the job definition can't take (wrong properties kind, unknown container name, numNodes without an open-ended range, targetNodes covering no range). - SubmitJob refuses an explicit INACTIVE revision (name:revision or ARN). - Replayed intermediate "Batch Job State Change" events no longer carry the final container exitCode. - With async settle a job stays RUNNING for 5s (DefaultBatchJobRunDwell) so a real client can terminate a running job; the dwell is stored on the job record and survives snapshots. - ListJobs with arrayJobId applies the JOB_NAME and other filters to the children. - Scheduling-policy binding (CreateJobQueue, UpdateJobQueue) and DeleteSchedulingPolicy are serialized so a queue can't end up bound to a deleted policy. - RegisterJobDefinition stores platformCapabilities as sent instead of echoing a default EC2, which made Terraform multinode definitions drift.
* fix(aws-batch): report multi-node parallel job nodes
DescribeJobs now resolves node job IDs ("<jobId>#<n>") and ListJobs with
multiNodeJobId lists the job's nodes instead of an empty list. Each node
carries nodeDetails (nodeIndex, isMainNode) and the container, ECS or EKS
properties of the node range that covers it; ListJobs summaries carry
nodeProperties (isMainNode, numNodes, nodeIndex).
Nodes are derived from the job record, the same way array children are,
so there is no new stored state. Every node follows the job's lifecycle
and, like array children, sends no state-change events of its own.
* fix(aws-batch): reject multi-node work on Fargate job queues
SubmitJob now returns a ClientException when nodeOverrides is sent to a
job queue whose compute environments are FARGATE or FARGATE_SPOT, or when
a multi-node job definition is submitted to one. The SubmitJob reference
says nodeOverrides "isn't applicable to jobs that are running on Fargate
resources; use containerOverrides instead", and the RegisterJobDefinition
reference says multinode isn't supported for jobs run on Fargate.
RegisterJobDefinition also rejects nodeProperties together with the
FARGATE platform capability, per the same reference.
The override tests that submit multi-node jobs now use an EC2 queue, so
they keep testing the override rules rather than the platform rule.
…oker subnets, managed KMS key (#1654) * fix(aws-kafka): async lifecycle, scheduled AWS/Kafka metrics, real broker subnets, managed KMS key Close the open real-cloud parity gaps in #1517: - ListNodes reports each broker's clientSubnet from the cluster's own BrokerNodeGroupInfo.ClientSubnets, spreading brokers evenly across them (round-robin by broker ID), instead of a fabricated subnet-<brokerId>. The field is omitted when the cluster records no subnets. - Under AsyncSettle a new cluster reports CREATING (create response, Describe, List) and settles to ACTIVE; a deleted cluster reports DELETING and is gone (NotFoundException) once its window ends, keeping its name until then. Update operations on a non-ACTIVE cluster are rejected with BadRequestException. A new VPC connection reports CREATING, then AVAILABLE. The default (AsyncSettle off) stays synchronous. The windows are per-record deadlines persisted in snapshots. - Publish DEFAULT-level AWS/Kafka metrics from the shared serve scheduler (Kafka is now a config.Tickable, ticked before CloudWatch), once a minute per ACTIVE provisioned cluster: ActiveControllerCount, GlobalTopicCount, GlobalPartitionCount and OfflinePartitionsCount on "Cluster Name", and UnderReplicatedPartitions and UnderMinIsrPartitionCount on "Cluster Name" + "Broker ID". Host and traffic metrics cloudemu does not model are not published. Kafka.SetMonitoring is wired to CloudWatch. - The default encryption-at-rest key is the account+region's AWS managed aws/kafka key ARN (one stable UUID key id per account and region), completed by the provider at create time and reported by both v1 and v2 describes, instead of a hard-coded key/msk-default in us-east-1. Part of #1517 * fix(aws-kafka): address review on async lifecycle, metrics tick and encryption defaults - Snapshot deep-copies each cluster, configuration, VPC connection and replicator under its own lock (topics, operations, SCRAM secrets, tags, raw options), so encoding no longer races a concurrent CreateTopic. The per-store loops move into helpers, which also brings Snapshot back under the gocyclo limit. - withDefaultEncryption fills every EncryptionInfo field the request leaves out with its documented default (managed at-rest key, ClientBroker TLS, InCluster true) and keeps explicit values, including InCluster false. - Tick judges cluster state at its own now argument instead of the options clock, and removes clusters whose DELETING window has ended, reporting the change so persist saves it even when nothing reads the cluster. - Topic create/update/delete, SCRAM secret associate/disassociate and cluster policy put/delete are rejected with BadRequestException while a cluster is CREATING or DELETING (only reachable under --async-settle). - GetCompatibleKafkaVersions resolves the cluster through getCluster, so a cluster past its DELETING window is NotFound. Part of #1517
…c-v2-cbor route (#1680) * fix(aws-keyspaces): report ttl and clientSideTimestamps only once enabled GetTable always returned ttl and clientSideTimestamps as {status: DISABLED}, even for a table that never set them. Both model enums (TimeToLiveStatus, ClientSideTimestampsStatus) hold only ENABLED, so real Keyspaces never reports DISABLED and omits the members on such a table. hashicorp/aws reads a present block into state, so a table that sets neither planned a forced replacement on every run. The provider now records "" for a setting that was never enabled (DISABLED, the documented default, maps to "" too) and GetTable emits the members only when set. An UpdateTable DISABLED no longer turns an enabled setting off, matching the docs ("can't disable it"). Restore clears the DISABLED that older builds stored on every table. * fix(aws-keyspaces): serve the rpc-v2-cbor protocol aws-sdk-go-v2/service/keyspaces v1.37.0 added Smithy RPC v2 CBOR beside AWS JSON 1.0 and prefers it, so hashicorp/aws 6.69+ sends every Keyspaces call to POST /service/KeyspacesService/operation/<Operation> with Smithy-Protocol: rpc-v2-cbor. CloudWatch claimed every rpc-v2-cbor request under /service/ and answered UnknownOperationException, so that provider could not even create a keyspace. keyspaces.CBORHandler matches only the KeyspacesService path and registers before CloudWatch. It only converts the wire format: the CBOR body is re-encoded as JSON and run through the existing JSON handler, and the answer is re-encoded as CBOR, with integral numbers as CBOR integers and the four model Timestamp members as tag 1, which the SDK requires. Errors keep their status and code as a CBOR __type/message map. Under --enforce-auth the handler is a Resolver naming cassandra:<Operation> from the path, and a deny is a CBOR AccessDeniedException. * docs(aws-keyspaces): note the rpc-v2-cbor route in the SDK server docs
…er (#1681) * fix(aws): skip resources deleted mid-walk in the RGT walkers that bypass project The Kafka, Kinesis, GuardDuty and Glue walkers and the OpenSearch walker read tags in a loop of their own instead of going through project, so a resource deleted between the list call and its tag read failed the whole GetResources call for every service. Paginated walkers now collect every page with listAll and then go through project, which skips a NotFound tag read and propagates any other error. - Kafka, Kinesis and OpenSearch tag reads return NotFound for a deleted resource, so project skips them as is. - GetDetector reports a missing detector as BadRequestException, as real GuardDuty does, so the GuardDuty row re-lists detectors when GetDetector fails and reports NotFound only when the detector is gone. - Glue GetTags never fails for a deleted job, so the walk never aborted, but it emitted a row for the deleted job. The Glue row now checks the job with GetJob first. * fix(aws): skip resources deleted mid-walk in SageMaker and Redshift discovery sagemakerDiscovery and the Redshift part of rdsDiscovery list resources and then read each one's tags in a separate call. Both tag reads return NotFound for a resource that is gone, which failed the whole discovery walk (Resource Groups Tagging API GetResources and Resource Explorer) when a delete landed in between. Skip that resource, the same as project does. Any other error still propagates. * test(aws): cover deletes racing every discovery walker that reads tags separately - TestDiscoverySurvivesConcurrentDelete runs each walker 2000 times next to a create/delete loop for Kafka, Kinesis, OpenSearch, GuardDuty, Glue, SageMaker and Redshift. Before the fix every case except Glue failed within the first 512 walks. - TestTaggedWalkersSkipResourceDeletedAfterList deletes a resource after listing and projects the stale list through each walker's row function. - TestSageMakerDiscoverySkipsVanishedResource and TestRedshiftDiscoverySkipsVanishedCluster check that NotFound is skipped and any other tag-read error propagates.
…isabledException (#1682) * fix(aws-kms): return InvalidMarkerException for an unknown Marker on list operations A Marker the server never issued now gives InvalidMarkerException on ListKeys, ListAliases, ListGrants, ListRetirableGrants, ListKeyRotations, ListResourceTags and DescribeCustomKeyStores, which all model it. ListKeyPolicies does not model it, so it keeps ValidationException. A Marker longer than 1024 characters or with a character outside U+0020-U+00FF fails the modeled MarkerType constraint and gives ValidationException on every operation. * fix(aws-kms): name the key in the DisabledException message Real KMS answers an operation on a Disabled key with "<key ARN> is disabled." instead of a generic "key is disabled". The error is built with driver.WithMessage, so errors.Is still matches driver.ErrKeyDisabled for Secrets Manager and other callers. Every operation that returns DisabledException goes through requireUsable, so they all get the same message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release Notes — v2.15.0
The zero-dependency in-memory default is unchanged.
✨ Features
Amazon CloudWatch Logs
StartQuery,GetQueryResults,StopQuery,DescribeQueries,GetLogRecordandGetLogGroupFields, with a query engine forfields,filter,sort,limit,display,stats,parseanddedupover system and JSON fields.FilterLogEvents, metric filters, subscription filters andTestMetricFilter, validated the way AWS does, with the per-group quotas.defaultValueis sent once per period with no match, and$name,$Nand$.pathmetric values are read from the matched event.CreateExportTaskwrites log events to a real S3 bucket, with the task lifecycle and the describe and cancel calls.DeleteRetentionPolicy,ListLogGroups, KMS key association and log group classes.PutLogEventsdrops events older than 14 days, past retention or more than 2 hours ahead, as AWS does.Amazon EventBridge
PutEventsand honor retention, and replays send them back to the bus with their rules, filtered on the full rule ARN.PutEventsroutes events through theEndpointIdit is given.AWS Batch jobs
SubmitJob,DescribeJobs,ListJobs,CancelJobandTerminateJob. Jobs move through the real states on the provider clock, honordependsOn, array jobs and multi-node parallel jobs with their nodes, and apply container, ECS, EKS and node overrides. Scheduling policies have full CRUD, and "Batch Job State Change" events go to the EventBridge default bus. Multi-node work on Fargate is rejected, as the API reference says.AWS KMS
KEY_AGREEMENTkeys for the NIST P-256, P-384 and P-521 curves, with a real ECDH shared secret that matches what the peer derives.Unavailableand fail crypto calls.AWS Secrets Manager replication
ReplicateSecretToRegions,RemoveRegionsFromReplication,StopReplicationToReplicaand the replica fields onDescribeSecret.ListSecretssupports planned deletion, sorting and the full filter set.AWS Network Firewall
Real update tokens with
InvalidTokenException, change protection, TLS inspection configurations, VPC endpoint associations, resource policies, Availability Zone associations, flow operations and analysis reports. Subnets get their real zones from the VPC service, deletes of protected or in-use resources are refused, and every list supports its filters and paging.Amazon Kinesis enhanced fan-out
SubscribeToShardkeeps its event stream open, pushes new records as they arrive, sends idle heartbeats and ends promptly when the server shuts down.🔧 Enhancements
Real-cloud parity: AWS
EventBridge:
PutEventsvalidates each entry, enforces the 1 MB request limit and keeps result entries aligned with the request.PutTargetsrejects conflicting input fields.CloudWatch:
GetMetricDataandGetMetricStatisticsvalidatePeriod, so a bad value returnsInvalidParameterValueinstead of crashing the server.Step Functions:
StartSyncExecutionrejects STANDARD state machines.ElastiCache: returns the real typed faults, serializes control-plane changes and reserves ids so concurrent creates can't collide.
Kinesis: throttling metrics, partition key and
GetRecordslimits, and CREATING and DELETING stream states under--async-settle.Kafka:
aws/kafkamanaged key.EFS:
UpdateFileSystemProtection, creating and deleting states for file systems and mount targets, AWS/EFS storage and throughput metrics, and theLONG_IDdefault.SQS:
ApproximateNumberOfMessagesNotVisiblecounts in-flight messages only;SenderIdis the caller's IAM unique id.STS
AssumedRoleIdnow uses the role's realRoleId.Keyspaces:
hashicorp/aws6.69 and later can create keyspaces;GetTableleaves out the settings a table never enabled, so Terraform plans stay clean.KMS: an unknown
Markeron list calls returnsInvalidMarkerException,DisabledExceptionnames the key ARN,aws:tag keys are rejected and grant principals are validated.Resource discovery: the Resource Groups Tagging API and Resource Explorer skip a resource deleted mid-walk instead of failing the whole call.
👥 Contributors
Thanks to everyone who shipped code in this release.