[cavanaug]

Summary

Adds opt-in Direct PUT throughput quotas to Firehose. The functional revisions
listed in the prior PR description and Cubic summary are implemented:

• Registers the three regional Firehose throughput quotas. Enforcement requires
  both an explicit account/region applied value and an enabled enforcement switch;
  AWS defaults alone never throttle (auto_scaling_default in introspection).
• Public PutRecord and PutRecordBatch share per-stream-instance request,
  record, and decoded-byte accounting, resolved through one batched quota lookup
  before taking Firehose locks. Internal Logs/SES/decoded handoffs stay unmetered.
• Positive record/byte allowances admit a boundary-crossing record and carry
  excess forward into later one-second windows, without idle credits. Debt is
  calculated from its last charge, so rejected retries cannot change fractional
  recovery timing; the review-found decimal-rate bug has a RED/GREEN regression.
• Request-rate exhaustion rejects the whole call with HTTP 500
  ServiceUnavailableException, message Slow down.. Record/byte exhaustion
  returns ordered HTTP 200 batch-entry failures and FailedPutCount; PutRecord
  uses the same whole-call error for its rejected record.
• Malformed base64 rejects the whole request with HTTP 400
  SerializationException, before quota consumption or buffering.
• Accepted records retain the existing S3 buffering, encryption capture, retry,
  retention, and shutdown behavior. Applied values/account overrides persist;
  usage and burst debt do not.
• Revises deterministic engine tests, official AWS SDK tests, and the canonical
  Firehose/Service Quotas documentation. SDK tests cover partial acceptance,
  explicit failed-entry resubmission, default-limit bypass, decoded-byte charging,
  SDK retries, and independently enforced same-name streams in two accounts.

The one-second accounting, burst carry-over, and dimension-to-error mapping are
local testing policies, not claims about AWS's unpublished throttling algorithm.
Skipping default limits is a local convenience, not a simulation of AWS scaling
latency. No live AWS load experiment was performed.

Post-review follow-up — 71d42171b

Addresses the latest completeness, accuracy, and engineering review in one
follow-up commit, without rewriting the existing branch history:

• Both public PUT APIs now share record-data decoding and size validation.
  Missing/null Data returns HTTP 400 InvalidArgumentException; wrong-type
  data or malformed base64 returns HTTP 400 SerializationException.
• A RED/GREEN regression covers both operations, including a valid leading
  batch entry. Rejected requests admit nothing and leave request, record, and
  decoded-byte capacity available to a subsequent valid request. The existing
  record/batch size limits and empty-record behavior are preserved.
• Restores the stream_delivery invariant comment and shares throttle error
  code/message constants between whole-call and batch-entry responses.
• Renames misleading E2E tests and documents the API-reference HTTP 500 versus
  vendored-model HTTP 503 discrepancy. Throttling remains HTTP 500; closed
  admission during local shutdown remains HTTP 503 with its distinct message.
• Updates the parity matrix, services index, canonical Firehose/Service Quotas
  pages, and agent guidance for explicit auto-scaling quota values and regional
  defaults. Documents the malformed-batch compatibility change and cross-service
  lazy template timing; provider behavior itself is unchanged.

Follow-up verification

• Server build passed.
• Targeted unit suites: core 444, Firehose 190, and Service Quotas 76
  passed (710 total). These are fresh follow-up results; the earlier SDK and
  conformance results below are historical and were not rerun in this follow-up.
• Firehose E2E suites passed: firehose 14 and firehose_throughput 11
  (25 total), against the rebuilt server. Official SDKs exercise supported
  calls; malformed inputs are exercised through the raw JSON endpoint.
• cargo clippy --workspace --all-targets -- -D warnings, formatting,
  documentation-count checks, and whitespace checks passed. Cargo still reports
  the existing proc-macro-error2 future-incompatibility notice.
• An independent read-only re-review of the follow-up diff reported no critical,
  important, or minor findings.
• A fresh cargo test --workspace is not green: it stopped at the same
  unchanged API Gateway test already documented below,
  data_plane::tests::vpc_link_proxy_resolves_target_nlb_and_attempts_connection.
  It received HTTP 504 instead of 502 after approximately 29 seconds.
  Because this run was fail-fast, the remaining workspace suites did not run.
  No unrelated assertions were relaxed and no tests were skipped to hide this.

This follow-up is not a claim that the complete workspace suite or required
CI/Cubic checks are green, and no merge was performed.

Verification before follow-up (4d4014bea)

• Targeted unit suites: core 444, Firehose 189, Rust SDK 4, and Service
  Quotas 76 passed (713 total, also passing with normal test concurrency),
  including deterministic burst repayment,
  concurrent admission, account/region/stream isolation, batched provider locking,
  default bypass, reset/restore, internal handoffs and validation precedence.

• Workspace Clippy with all targets and -D warnings, formatting and whitespace
  checks passed. The dependency proc-macro-error2 emits an existing Cargo
  future-incompatibility notice, not a Clippy failure.

• Conformance harness unit tests: 107 passed; official SDK Firehose conformance
  integration tests: 6 passed.

• TypeScript type-check and 13 Service Quotas client tests passed; Go vet passed.

• Website: Zola 0.22.1 built 185 pages and 10 sections; documentation-count checks
  passed.

• A fresh final cargo build --bin fakecloud passed, followed by Firehose Smithy
  conformance against that rebuilt server: 416/416 variants, all 12 operations,
  zero failures and crashes.

• Independent review identified the fractional-retry bug described above; its
  re-review confirms the fix and reports no remaining blocking findings.

• Official AWS SDK E2E against the rebuilt final server: 87 passed across
  firehose, firehose_throughput (11), servicequotas, firehose_buffering,
  firehose_persistence, firehose_gzip, firehose_snappy,
  firehose_compression_validation, and ses_firehose_events.

• Complete workspace unit/doc-test run in CI scope with --no-fail-fast:
  12,138 passed, 1 failed, 0 ignored, across 237 test/doc-test groups. The
  single failure is the unchanged API Gateway test detailed below. This is not a
  passing workspace suite.

Commands used for the major gates:

cargo build --bin fakecloud
cargo test -p fakecloud-core -p fakecloud-firehose -p fakecloud-servicequotas -p fakecloud-sdk --lib
cargo test -p fakecloud-e2e --test firehose_throughput --test firehose --test servicequotas --test firehose_buffering --test firehose_persistence --test firehose_gzip --test firehose_snappy --test firehose_compression_validation --test ses_firehose_events
cargo test -p fakecloud-conformance --test firehose --lib
target/debug/fakecloud-conformance run --services firehose --format json
cargo test --workspace --exclude fakecloud-e2e --exclude fakecloud-conformance --exclude fakecloud-tfacc --exclude fakecloud-parity --no-fail-fast
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
git diff --check

Workspace validation limitation

The workspace unit command (CI scope, excluding fakecloud-e2e,
fakecloud-conformance, fakecloud-tfacc, and fakecloud-parity) is not green.
The unchanged API Gateway test
data_plane::tests::vpc_link_proxy_resolves_target_nlb_and_attempts_connection
expects HTTP 502 for a connection refusal on localhost port 54321, but receives
HTTP 504 after its 29-second timeout in this environment. It reproduces alone,
including with proxy variables removed. No listener exists on that port, yet a
direct local connection also times out rather than refusing. This is outside the
Firehose change; no assertions were relaxed, tests skipped, or unrelated code
changed to manufacture a passing workspace result. Do not treat this PR as
merge-ready until required CI and Cubic checks are green.

Automated review status

Cubic's check for signed commit 4d4014bea5a9c0e760ec9a5847bc96ca1dd6a57b
completed neutral, not an approval: the team's monthly review line limit has
been reached, so Cubic did not perform a fresh review. The independent code
review and re-review described above completed locally; they do not replace the
required Cubic check for merging.

[cavanaug]

Ok, this is now ready for merge after owner approval.