You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Store original document bytes in RustFS, content-addressed, via fsspec/s3fs.
Build
put(data: bytes) -> str writes to {sha256[0:2]}/{sha256} and returns the key. Idempotent /
conditional (write-only-if-absent): a second put of an already-stored key leaves the existing
object untouched, required for A12 and for safe retries after a crashed write.
Read back and re-hash before acking the write. Content-addressing makes integrity
verifiable for free, and this catches a write failure while a user can still retry. This is a
write-time check, not durability — it gives zero protection against loss after a successful
ack (bit rot, a RustFS bug, disk failure). See "Why read-back matters more here" below.
get(key) -> bytes.
Backend configured as a URI so MinIO/S3 are drop-in.
Why the hash and not a UUID
Content hash is already the document identity. Keying objects by UUID would mean the same
document arriving by two paths stores twice, breaking dedupe and making A5/A12 meaningless.
Why read-back matters more here than usual
RustFS is Beta — its own README marks distributed mode, lifecycle management and KMS as
"Under Testing". Directory-ingested files keep a second copy in the watched directory. Uploaded files do not. For those, RustFS holds the only copy. Read-back-and-re-hash catches a
failed write at ack time; it does nothing for loss after a successful ack. That window is an
accepted risk for this slice, not one this check closes.
Because storage goes through fsspec, swapping to MinIO or S3 is an endpoint change if Beta bites.
Definition of done
Round-trip test against the compose RustFS: put → get → bytes identical, key matches the hash,
and a corrupted-write simulation is detected rather than acked.
Activity
[-]RustFS object storage: content-addressed originals[/-][+]NU-003 RustFS object storage: content-addressed originals[/+][-]NU-003 RustFS object storage: content-addressed originals[/-][+]NU-004 RustFS object storage: content-addressed originals[/+]