Skip to content

distribute: s3, azure and testflight backends - #35

Closed
Interlap01 wants to merge 9 commits into
mainfrom
distribute-backends
Closed

Interlap01 wants to merge 9 commits into
mainfrom
distribute-backends

Conversation

@Interlap01

@Interlap01 Interlap01 commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

What

builder ios distribute and ios build --distribute take --backend (else distribute.backend in builder.json, else github):

  • s3: Amazon S3 and S3-compatible stores (R2, MinIO, GCS with HMAC keys) through distribute.endpoint. The IPA and a short m.plist go under <prefix>ios-builder/<id>/ with marker metadata. They are linked by SigV4 presigned GET URLs, written in Go without the AWS SDK. --ttl runs from 2m to 168h (default 1h). Credentials come from AWS_*, else AWS_PROFILE in ~/.aws/credentials.
  • azure: the same bucket backend on Azure Blob. Every request carries a service SAS signed with AZURE_STORAGE_KEY (or the connection string).
  • testflight: needs an App Store signed IPA (the inverse of the OTA check). It uploads, waits for processing and adds the build to an internal group from --group/distribute.group, creating the group if it is missing. An external group is refused before the upload. ios build --distribute --backend testflight runs ios release to that one internal group, so it also picks the next build number.

Inspect and CheckDistribution now take the backend. The whole target (client, bucket, credentials, group) is resolved before a build is pushed.

Why

The GitHub draft-release + gist backend needs the gist scope, and its links last only five minutes. Teams with a bucket get links that last up to a week (--once --ttl 168h). App Store builds, which iOS cannot install over the air, now have a route to devices through internal TestFlight.

QR size

The bucket link is longer, so the code is bigger. Measured on representative links and pinned in TestBucketQRSizes:

backend modules
github 41
azure 57
s3 69 (73 columns with the border, fits 80)
R2 73
s3 with a session token ~105

At ~105 modules the code no longer fits 80 columns, and the session prints a note saying so. The Azure SAS escapes only the signature's +, which keeps that code a version smaller.

Cleanup

  • Ending a session deletes both objects. --once leaves them.
  • --cleanup deletes only objects under <prefix>ios-builder/ that carry the marker metadata. S3 checks the marker with HEAD, Azure through include=metadata.
  • testflight has nothing to clean up.

How tested

  • SigV4 against AWS's published S3 vectors: the presigned URL plus the GET Object, PUT Object, lifecycle and list examples.
  • The fake S3 server re-signs every request as it arrived on the wire. It covers upload, mint, automatic refresh, a "device" install that fetches the manifest and then the IPA (from both the old and the new link), cleanup, delete-failure leftovers, upload errors with the S3 code, paginated cleanup sweeps, TTL limits and credentials loading.
  • The Azure SAS is checked against the documented 16-field string-to-sign. The fake Azure server runs a session with refresh, an install, paginated cleanup and leftovers.
  • The testflight flow runs on the existing fake App Store Connect: upload and wait, add to the group, create a missing group internal, refuse an external group before the upload or before any change.
  • Command-level tests cover the backend-aware validation and preflight.
  • go test -race ./..., go vet, gofmt and golangci-lint v2.12.2 are clean.

No live calls were made to any cloud.

Left out

  • SSO and credential_process AWS profiles are not read. The error says to export static or temporary keys instead.
  • Multipart upload is not implemented. A single PUT covers IPAs up to 5 GB.

SubmitTestFlight gains Internal, which refuses an external group before
anything changes and creates missing names internal; ToInternalGroup
checks the group, uploads with wait and adds the build. release.Options
passes Internal through as InternalGroups.
distribute.backend in builder.json (with the bucket, account and group
settings) and BackendName pick github, s3, azure or testflight. Inspect
and CheckDistribution take the backend: the over-the-air ones keep
refusing App Store builds, testflight refuses everything else.
The IPA and a short m.plist go under <prefix>ios-builder/<id>/ with
marker metadata and are linked by SigV4 presigned GET URLs (query
presign written here, checked against AWS's published vectors), valid
for --ttl up to seven days. Each mint presigns the IPA again and
rewrites the manifest in place. Cleanup lists the upload folder and
deletes only objects whose HEAD shows the marker. Credentials come from
AWS_* or the shared credentials file; R2, MinIO and GCS go through
distribute.endpoint, path-style.

A presigned manifest URL makes a version 13 code (69 modules) against
the gist's version 6; temporary credentials push it to ~105, and the
session says so when a code is wider than 80 columns.
The same bucket backend on Azure Blob Storage: every request, Builder's
own included, carries a service SAS signed with the account key, so one
signer covers upload, delete, list and the install links. The SAS keeps
its colons, slashes and padding unescaped, which leaves a version 10
code (57 modules). TestBucketQRSizes pins each backend's size.
…uild

--backend (else distribute.backend, else github) picks the store; the
target is resolved before a build is pushed, so a missing bucket,
credential or group fails first. testflight uploads an App Store IPA
and adds it to the internal group --group; ios build --distribute
--backend testflight runs ios release to that one internal group, which
also picks the next build number. --ttl sets the s3/azure link
lifetime; the backend flags on ios build need --distribute.
ios distribute without a GitHub login reported the missing token
instead of the missing IPA, as it did before backends existed.
@Interlap01

Copy link
Copy Markdown
Collaborator Author

Dropped: the GitHub draft release + gist backend stays the only distribute backend.

@Interlap01 Interlap01 closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant