Skip to content

fix(aws-codeartifact): close real-cloud parity gaps in #1456 - #1718

Open
thzgajendra wants to merge 2 commits into
stackshy:developmentfrom
thzgajendra:fix/codeartifact-parity-1456
Open

thzgajendra wants to merge 2 commits into
stackshy:developmentfrom
thzgajendra:fix/codeartifact-parity-1456

Conversation

@thzgajendra

@thzgajendra thzgajendra commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • CodeArtifact in cloudemu served 15 of the API's 48 operations and accepted input real AWS rejects, so aws codeartifact login, publishing a package, permission policies and package groups did not work, and invalid names, a foreign domain owner, a dangling upstream and a second external connection were silently accepted.
  • This change closes every item of AWS CodeArtifact: open real-cloud parity gaps #1456 in one PR: all 48 operations of aws-sdk-go-v2/service/codeartifact v1.45.0, the validation, ownership, upstream and external-connection fixes, package version events on EventBridge, CloudTrail recording, tagging and discovery.
  • Package assets are stored as real bytes (emulator cap 128 MiB per asset), so publish, list, download, copy, dispose and delete round-trip.

Changes

The problem (in plain words)

A developer pointing their tools at cloudemu hits a wall quickly. aws codeartifact login fails (GetAuthorizationToken answers 501 no handler registered), get-repository-endpoint answers unsupported path, and there is no way to publish or read a package, set a permissions policy or create a package group. Worse, the calls that did work were too forgiving: create-repository --repository "bad name" succeeded, --domain-owner 999999999999 was ignored, a repository could list a repository that does not exist as its upstream, and a second associate-external-connection silently replaced the first. A test suite written against cloudemu therefore passed on code that real AWS would reject.

How this fixes it

The base driver.CodeArtifact interface is unchanged. The new operations arrive as small optional interfaces (Access, Policies, Packages, PackageVersions, PackageGroups, plus RepositoryAdministratorLister) that the wire handler finds by type assertion, as the ECS and Bedrock services already do. All validation, defaults and state rules live in the provider, so the Go library and cloudemu serve behave the same.

  • State and locking. One lock guards every change that looks at another entity (create with a quota, delete with dependents, publish, copy, policies, groups). Reads return deep copies. Events are collected under the lock and published after it is released, because EventBridge delivery can call back into the mock.
  • Packages. Only generic packages can be published through the API (as in AWS). A version moves Unfinished -> Published <-> Unlisted <-> Archived, and Disposed or Deleted for good. Asset bytes are kept and checked against the SHA-256 the caller sends. A domain's assetSizeBytes counts identical content once, as AWS stores it once per domain.
  • Origin controls and package groups. Package-level and group-level controls follow the user guide: the group pattern grammar ($, ~, *), the most specific match, strong and weak (case or separator variant) matches, inherited modes and allowed-repository lists. A blocked publish is refused.
  • Wire. A route table holds all 48 verb and path pairs and also gives the CloudTrail event name. Errors carry the exact exception, HTTP status (400, 402, 404, 409), resourceId, resourceType and reason. A per-operation IAM resolver (codeartifact:<Operation>, plus sts:GetServiceBearerToken for the token) makes --enforce-auth precise.
  • Cross-cutting. CodeArtifact Package Version State Change events (documented detail, per-package sequenceNumber), CloudTrail recording, Resource Groups Tagging discovery and tagging of domains, repositories and package groups, snapshots and --persist (bytes included unless --persist-metadata-only), and a Terraform fixture.

Not emulated, and written down in docs/coverage/nongoals/codeartifact.md: the registry protocols (npm, pip, Maven, ... are not served), ingestion from public registries, request-rate throttling, evaluating resource policies, and the repository state-change event (AWS publishes no payload for it; its reference names it "Goshawk Repository State Change"). A few error choices the AWS reference does not state are chosen conservatively and listed there as unconfirmed.

Diagram

flowchart LR
    C[aws CLI / SDK / Terraform] -->|REST-JSON /v1/...| R[route table: 48 verb + path]
    R --> H[wire handlers: decode, map errors]
    H -->|optional capability| P[provider mock: validation, state rules, one lock]
    P --> S[(domains, repositories, packages, versions, assets, groups, policies)]
    P -. after unlock .-> E[EventBridge: Package Version State Change]
    R -. method + path .-> T[CloudTrail event name]
    P --> D[snapshot / persist]
Loading

Provider Coverage

  • AWS (CodeArtifact is an AWS-only service; Azure, GCP and OCI are not applicable)

Checklist

  • All tests pass (go test ./...)
  • Linter passes (golangci-lint run --timeout=9m ./...) - 0 issues on the changed CodeArtifact packages; the repo-wide run reports only issues that exist on development
  • Every provider the change applies to implements the same behavior - CodeArtifact is an AWS-only service in cloudemu
  • Unit tests added to provider test files; the real-SDK round trips sit beside the service in server/aws/codeartifact, as in the recent single-service PRs, so nothing was added to cloudemu_test.go
  • Regenerated docs (go generate ./... for coverage; go run ./internal/compatgen for the compat matrix) and committed the result

Test Plan

How I tested it

User environment: macOS (arm64), Go 1.25.0; the repo Dockerfile built and run with default flags; go install ./cmd/cloudemu with an empty $HOME; AWS CLI v2.30.0 with an explicit --endpoint-url; boto3 1.43.118 in a fresh venv; OpenTofu v1.13.1 with the real hashicorp/aws ~> 5.0 provider and an empty provider "aws" {} block; npm; a Go program in a fresh module. Every scenario ran against a build of stackshy/development (baseline) and this branch. The branch server was built with -race; the race detector reported nothing over the whole session.

  • aws codeartifact login --tool npm writes the npm config (baseline: 501); --dry-run for pip, twine and swift prints the commands.
  • Generic package lifecycle through the CLI: publish with --unfinished, publish the last asset, list, download (cmp shows identical bytes), archive (download then ConflictException), restore, copy twice (ALREADY_EXISTS), dispose, delete versions, delete package. Baseline: 501 for all of it.
  • A real OpenTofu project: init, apply (domain, two repositories with an upstream and an external connection, repository policy), re-plan No changes, update in place, re-plan No changes, import of repository and domain by ARN, destroy. On the baseline the same project fails at PutRepositoryPermissionsPolicy with 404 ResourceNotFoundException. The repo's own contrib/terraform fixture codeartifact passes too.
  • 45 operations through boto3, each response compared with the botocore output shape: no missing required member, no member outside the model.
  • Each item of AWS CodeArtifact: open real-cloud parity gaps #1456 side by side with the baseline. Baseline: invalid name accepted, foreign --domain-owner ignored, dangling upstream accepted, second external connection replaced the first, deleting an upstream accepted, max-results=0 accepted. Branch: ValidationException, ResourceNotFoundException, ResourceNotFoundException, ConflictException (first connection kept), ConflictException, ValidationException.
  • All 33 added operations sent raw at the model's verb and path: none was served on the baseline (14 405 and 12 404 from the S3 fall-through, 7 unsupported path); on the branch every one is answered by CodeArtifact.
  • Limits through boto3: 1000 repositories per domain (1001st ServiceQuotaExceededException 402, listing walks 10 pages of 100), 10 upstreams (11th refused), 200 tags (201st refused, overwrite at the limit fine), 350 assets per version, an asset of exactly 128 MiB accepted and 128 MiB + 1 byte refused, name boundaries (2 and 50 characters for domains, 2 and 100 for repositories).
  • 60 parallel create-domain of one name: exactly one succeeds. 40 parallel creates against the 10-domain quota: 10 domains. 40 parallel publishes, status updates racing delete-package, and publishes racing delete-repository: nothing resurrected, 0 orphan bytes.
  • --persist restart: revision, asset bytes, policies, groups and tags survive; state written by the baseline binary loads in this binary; --persist-metadata-only keeps structure but not bytes; snapshot, rewind and fork restore bytes.
  • --enforce-auth with seeded IAM users: a read-only user reads and downloads but every write answers AccessDeniedException; a user allowed only DescribeRepository on one repository sees only that repository; get-authorization-token needs the sts action as well. S3 and SQS keep working. A copy needs codeartifact:ReadFromRepository on the source repository as well as write access to the destination: a user with only the destination permission is denied and nothing is copied.
  • EventBridge rule on aws.codeartifact to an SQS queue: publish, archive and delete deliver three events with the documented envelope and detail. CloudTrail lookup-events lists the calls from codeartifact.amazonaws.com.
  • Resource Groups Tagging lists domain, repository and package group with their tags and can tag and untag them.

New tests (all in providers/aws/codeartifact and server/aws/codeartifact unless noted), red on development and green here for the regression rows: name and owner patterns, ownership on lookups, upstream rules (dangling, self, duplicate, cycle, 10 maximum, XOR with an external connection), the ten external connection names, delete guards, tag limit at 200/201, policy revision lock, publish and status state machine, copy, dispose, origin controls, group grammar (every pattern in the AWS user guide), hierarchy and strong/weak matching, events, snapshot round trip, pagination of every list; race tests for the domain, repository, tag, upstream, asset and group invariants, policy revision compare-and-set, create-vs-delete orphans and aliasing; real-SDK round trips; a 48-operation route and model check (testdata/operations.json from the botocore model) and an error-matrix test that drives every operation into a validation error and a not-found error and checks each exception is one the model lists for that operation; IAM resolver tests; TestRecordsCodeArtifactEvents (CloudTrail), TestCodeArtifactPackageVersionEvents (EventBridge to SQS), TestCodeArtifactDiscoveryAndTagging and TestCodeArtifactWinsBeforeS3AndLeavesSiblingsAlone (routing seam).

Library API. driver.CodeArtifact is unchanged. Added: driver.Access, Policies, Packages, PackageVersions, PackageGroups, RepositoryAdministratorLister, APIError fields ResourceID, ResourceType, Reason, ExServiceQuota, and resource type and format constants. Existing methods now return typed errors for input they used to accept (invalid names, a foreign domain owner, dangling upstreams), which is the behaviour change #1456 asks for.

Coverage

providers/aws/codeartifact 81.6% -> 94.4%; server/aws/codeartifact 69.4% -> 95.1%; every function added is exercised by a test. The local CI gate (build, vet, go test ./..., -race on touched packages, go mod tidy -diff, golangci-lint on new lines, go generate and compatgen drift, Structure, contrib modules) passes.

Related Issues

Closes #1456

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AWS CodeArtifact: open real-cloud parity gaps

1 participant