artifact-cas: verify downloads against the requested digest via disk staging before sending to the client #3460

Problem

#3365 made the Artifact CAS verify uploads server side: the client stream is spilled to an unlinked file in the staging directory, hashed in the same pass, and only handed to the storage backend once the hash matches the declared digest.

Downloads were not updated and the two paths behave differently:

• gRPC ByteStream.Read streams chunks to the client as they arrive from the backend and only compares the hash after the last byte was sent. A corrupted or tampered blob in the backend is fully delivered before the server reports a mismatch. Callers that do not re-hash (e.g. the control plane) depend on the trailing gRPC error being propagated.
• HTTP GET /download/{digest} verifies before writing the response, but does so by buffering the whole artifact in memory, so CAS memory grows with artifact size.

Proposed approach

Reuse the staging directory introduced in #3365 for downloads:

1. Add a shared spill-and-verify helper in the CAS service layer: stream a producer into an unlinked temp file under staging_dir while hashing, compare against the requested digest, rewind, and return the file.
2. Refactor the upload path onto the helper (no behaviour change).
3. gRPC Read: stage the backend egress, verify, then stream the file to the client. On mismatch nothing is sent.
4. HTTP download: stage, verify, set Content-Length from the verified size, copy the file to the response. Drop the in-memory buffer.
5. Report a download mismatch as gRPC DataLoss / HTTP 500 with an explicit content-integrity message instead of the current Unauthorized / 401.
6. Update the staging_dir docs and Helm chart comments to cover downloads, since ephemeral-storage sizing now has to account for concurrent downloads as well.

Client-side verification in the CLI stays as defence in depth.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri