[cavanaug]

Summary

Part 2 of the Firehose S3 delivery series tracked in #2708, following merged #2715.

• Encode each immediate-delivery batch as one real GZIP stream using the existing flate2 dependency. Preserve binary record bytes and apply AppendDelimiterToRecord before compression.
• Use .gz by default, honor FileExtension overrides, and persist Content-Encoding: gzip together with the encoded body and its MD5 ETag.
• Add a metadata-aware shared S3 writer without changing existing callers. Delegate to upstream's versioned delivery commit so bucket versioning, default Object Lock retention, ownership, and durable failure handling remain intact.
• Cover official Firehose/S3 SDK calls, raw CloudWatch Logs payloads through the internal adapter, and exact compressed bytes and metadata surviving restart, including bucket-default KMS, replication, and S3 notifications.
• Update service documentation for this increment only. Delivery remains immediate; Snappy and size/time buffering follow separately.
• Follow the owner's Firehose: staged contributions for S3 output fidelity, GZIP/Snappy, and size/time buffering #2708 admission convention: modeled configurations still round-trip, but requesting an unimplemented delivery capability fails public record admission with InvalidArgumentException naming the setting. Preserve CFN settings so those streams get the same checks rather than silently writing substitute bytes.
• Derive the output codec from the same parser admission uses (Compression::from_format), so admission and encoding cannot disagree and no request path relies on an unreachable!.

Rebase compatibility

Rebased onto upstream main at 774c772d6. Retains #2712's SES publishing, #2713's source admission, #2714's compression validation, #2720's raw GZIP Logs records, #2725's ordinary S3 versioning/Object Lock semantics, #2730's default CRC64NVME full-object checksums, and #2737's shared write-side-effect path. Checksums and ETags describe the compressed S3 payload, not decoded records or at-rest KMS ciphertext. Bucket-default encryption, replication, and notifications remain on the shared commit path.

Failed persistent writes do not publish memory-only objects, matching current upstream. Records are still acknowledged without retries in this immediate-delivery increment; the buffering increment adds retention/retry of failed batches.

Test plan

• Verified the final tree (head dfd71471c): 1,721 focused unit tests across CloudFormation (635), Firehose (70), Logs (361), and S3 (655); 82 rebuilt-server SDK tests covering Firehose/GZIP, compression/admission validation, SES events, checksums, persistence, SSE-KMS, and write side effects; one Athena cross-service regression; 107 Firehose/SES/S3 conformance tests.
• Workspace all-target Clippy with warnings denied, formatting, documentation counts, and diff checks passed on that tree.
• cargo test --workspace was run without exclusions on 055708266 (before the final single-parser refactor and docs wording) and stopped on fakecloud-apigateway::data_plane::tests::vpc_link_proxy_resolves_target_nlb_and_attempts_connection: expected 502, got 504 (128/129 API Gateway tests passed). The exact failure was independently reproduced on untouched upstream 153774026; API Gateway's test and implementation are unchanged by this PR or the later upstream commits. Closed port 127.0.0.1:54321 times out in this WSL environment instead of refusing connections. This is a failing local workspace result, not a passing gate; no test skip/exclusion was added. The focused gates above passed on the final revision.
• A pre-publication code/design and STRIDE security review of the whole stack found no GZIP-specific vulnerabilities; its codec-parsing and documentation findings are addressed here.

Maintainer alignment

Follows the owner's #2708 guidance: exact byte concatenation, modeled-enum acceptance on Create/Update/CFN, and named InvalidArgumentException on unsupported record admission. The interval-range correction and Firehose-owned flusher/reset/shutdown lifecycle remain in the later buffering increment, not this codec PR.

Series context

Refs #2708; does not close the tracking issue. The next increments are stacked on this PR: Xerial Snappy in #2746, then the complete buffered-delivery lifecycle in #2747.

[vieiralucas]

Thanks a lot for this, @cavanaug. Real GZIP output with the right Content-Encoding, ETag and checksum over the stored bytes is a big fidelity step for anyone reading Firehose output from S3, and the admission checks follow the #2708 plan exactly. I pushed a few follow-ups before merging: a destination KMSEncryptionConfig is now implemented (delivered objects are written with SSE-KMS under AWSKMSKeyARN) instead of rejected at admission, with unit and e2e coverage; the two S3 delivery writers are folded into one put_object_via_store_with_options; CloudFormation now coerces string-typed nested booleans and numbers in the destination blocks so DescribeDeliveryStream echoes model types; and the processor docs wording is updated. Merged. Really appreciate the contribution.