Skip to content

Add support for MacOS deployments, parallel S3 downloads, and fix DNS resolution in EUSC & ISO regions - #431

Merged
gmiliaras merged 13 commits into
mainfrom
2.1.2
Oct 14, 2026
Merged

gmiliaras merged 13 commits into
mainfrom
2.1.2

Conversation

@gmiliaras

Copy link
Copy Markdown
Contributor

Issue #, : #430

Description of changes:

Agent release 2.1.2.

New Features

  • macOS support: EC2 Mac instances on Apple silicon (aarch64) with macOS 15, 26 and 27. AppSpec permissions that set acls or context are rejected on macOS; self-update is not available on macOS yet.
  • Parallel S3 downloads (opt-in): when s3_download_max_concurrency is set above 1, bundles are downloaded as parallel range requests pinned to the object's ETag/version. New config keys are s3_download_max_concurrency (default 1, max 64) and s3_download_chunk_size (default 8MiB, min 64KiB). The default behaviour is unchanged: one GetObject.

Fixes

  • Endpoint DNS suffixes: the command endpoint now uses the correct DNS suffix for the EUSC and ISO partitions.
  • Deployment log: fixes Agent v2 logs behavior mismatch with v1 #430; script output is written to the deployment log line by line, matching the v1.x agent.

Other

  • Test fixes for flaky timing bounds and an ETXTBSY race in the PATH isolation test.
  • CHANGELOG entries for 2.0.1, 2.1.0 and 2.1.2.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

When tests run in parallel, a child forked by another test thread can
briefly inherit the just-written script's write descriptor, so exec
fails with "Text file busy". Retry a few times with a short sleep.
The command endpoint was built with amazonaws.com for every region
outside the China partition, so hosts in the European Sovereign Cloud
and the ISO partitions resolved an endpoint that does not exist. Map
each partition's region prefix to its DNS suffix:

- eusc-*    (aws-eusc)  -> amazonaws.eu
- us-iso-*  (aws-iso)   -> c2s.ic.gov
- us-isob-* (aws-iso-b) -> sc2s.sgov.gov
- eu-isoe-* (aws-iso-e) -> cloud.adc-e.uk
- us-isof-* (aws-iso-f) -> csp.hci.ic.gov

Bump codedeploy-commands to 2.1.1.
Build, package and run the agent on macOS 15 and later (EC2 Mac,
aarch64 and x86_64):

- Gate the ACL (setfacl) and SELinux (semanage/restorecon) installer
  commands on Linux. On other Unix hosts, AppSpec permissions that set
  `acls` or `context` are rejected before any install command runs;
  `owner`, `group` and `mode` still apply.
- Strip the dyld loader variables (DYLD_INSERT_LIBRARIES and friends)
  on macOS where LD_* is stripped on Linux, since dyld ignores LD_*.
- Refuse self-update on macOS before any network use: the install
  script has no pkg package type yet.
- Add a launchd LaunchDaemon that mirrors the systemd unit, and .pkg
  build, install and uninstall scripts under macos/pkg.
- Gate the ACL and SELinux security tests on Linux.
- Document supported platforms in README.md and the macOS build in
  DEVELOPMENT.md.
Raise the elapsed-time upper bounds that flaked on loaded runners, and
test the throttle jitter bound directly instead.
On Linux the worker uses PR_SET_PDEATHSIG to die with its master. macOS
has no parent-death signal, and launchd considers the job stopped once
the master exits, so a restart would start a second master and worker
next to the orphaned worker. Watch the master's pid with kqueue
(EVFILT_PROC / NOTE_EXIT) on a background thread and exit when it
fires.
Write each scripts.log line to the deployment log when it is produced,
as "[<ts>] [<deployment_id>]<message stripped>". Previously the hook
command copied the 2048-byte diagnostic tail into the log after each
lifecycle event, which dropped the deployment ID, doubled the
timestamp, added a blank line per record, and lost every line older
than the tail.

All deployments now share one deployment-log handle, so concurrent
deployments no longer rotate the file independently. The
failure-summary line is removed; the log holds only script log lines.

Blank script output lines are now logged as a bare "[stdout]" record in
both files; they were previously dropped. The deployment log gets the
same ANSI-stripped line as scripts.log.
A bundle is downloaded with one GetObject over one connection, so large
bundles are limited by single-connection throughput. Setting
s3_download_max_concurrency above 1 splits the download into parallel
range requests. The default stays 1, a single unranged GetObject.

- The first request fetches the first range and reads the object size
  from Content-Range, so an object of one chunk or less still takes one
  request.
- Later ranges send If-Match with the first ETag (and the versionId, if
  pinned), so an object replaced mid-download fails the deployment
  instead of producing a mixed bundle. Each response must carry the
  Content-Range that was asked for, and at most that many bytes are
  written.
- Each range retries up to 3 times when its body fails mid-stream, which
  the SDK does not retry. Failed requests are left to the SDK's own
  retries, and local write errors fail at once.
- Ranged requests turn off the SDK's response checksum check, since a
  full-object checksum cannot be checked against one range.
- Ranges are written to their own offsets of one file with the new
  write_all_at (pwrite on Unix, seek_write on Windows).
- The runtime is multi-threaded only when concurrency is above 1, with
  min(cores, concurrency) workers, capped at 8. The worker's agent-update
  client only fetches the install script, so it stays single-threaded.
- The download falls back to one unranged GetObject when the endpoint
  refuses ranges (501) or the object is empty (416), judged by HTTP
  status when the error has no body; when there is no strong ETag or
  version to pin the object; or when a response without Content-Range
  may be truncated.

New config keys, clamped rather than rejected: s3_download_max_concurrency
(default 1, max 64) and s3_download_chunk_size (default 8MiB, minimum
64KiB).

tests/s3_parallel_download.rs runs against a fake S3 endpoint and checks
parallelism, the concurrency cap, If-Match, the fallbacks, retries, a
replaced object, a later range answered with the whole object, and a
checksum header on a range response.
Add CHANGELOG entries for 2.0.1, 2.1.0 and 2.1.2.

@123anvarov-create 123anvarov-create left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gmiliaras
gmiliaras merged commit 421768c into main Oct 14, 2026
11 checks passed
@gmiliaras
gmiliaras deleted the 2.1.2 branch October 14, 2026 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent v2 logs behavior mismatch with v1

4 participants