Skip to content

feat(preprod): Gate snapshot objectstore writes behind an option - #124210

Merged
NicoHinderling merged 4 commits into
masterfrom
feat/snapshots-objectstore-usecase-option
Sep 15, 2026
Merged

NicoHinderling merged 4 commits into
masterfrom
feat/snapshots-objectstore-usecase-option

Conversation

@NicoHinderling

@NicoHinderling NicoHinderling commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Adds the default-off boolean option preprod.snapshots.objectstore.snapshots-usecase.enabled to select the destination for new snapshot content, using the shared storage reader introduced by #124194, now on master.

With the option off, writes remain on preprod. When enabled, server-side writes (manifests, precomputed head images, comparison results, diff masks, and ZIP archives) use preprod_snapshots, and reads prefer that namespace before falling back to preprod. Upload-options requests with usecase=auto receive a token for the selected usecase and its name in the response; getsentry/sentry-cli#3408 and getsentry/cli#1587 use that contract for direct screenshot uploads. Requests without usecase=auto continue receiving preprod tokens even when the option is enabled.

The Objectstore policy change in getsentry/ops#23486 is merged; confirm it is deployed and deploy the backend stack fully before enabling the option through sentry-options-automator. Because deduplication is scoped by usecase, upgraded clients re-upload images once after the switch; later uploads deduplicate normally.

Outstanding review blocker: comparison outputs currently reuse artifact-pair keys. Recomputing a comparison across a usecase switch and then rolling back can read an older manifest or mask while the database describes the newer result. Before rollout, comparison generations need authoritative, generation-qualified storage references with rolling-deploy compatibility. This is a broader follow-up than cleanup; do not enable getsentry/sentry-options-automator#9713 until it is addressed.

@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 11, 2026
@sentry

sentry Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Sentry Snapshot Testing

Name Added Removed Changed Renamed Unchanged Skipped Status
sentry-frontend
sentry-frontend
- - - - - - ⏳ Processing

⚙️ sentry-frontend Snapshot Settings

@NicoHinderling
NicoHinderling marked this pull request as ready for review September 11, 2026 21:06
@NicoHinderling
NicoHinderling requested review from a team as code owners September 11, 2026 21:06
Comment thread src/sentry/preprod/snapshots/storage.py Outdated
@NicoHinderling
NicoHinderling force-pushed the feat/snapshots-objectstore-usecase-option branch from def5081 to 20a9161 Compare September 11, 2026 21:13
@NicoHinderling NicoHinderling changed the title feat(preprod): Switch snapshot objectstore writes via option feat(preprod): Gate snapshot objectstore writes behind an option Sep 11, 2026
@NicoHinderling
NicoHinderling force-pushed the feat/snapshots-objectstore-usecase-option branch from 20a9161 to 03d1484 Compare September 11, 2026 21:19
@github-actions github-actions Bot removed the Scope: Backend Automatically applied to PRs that change backend components label Sep 11, 2026
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 11, 2026
@NicoHinderling

Copy link
Copy Markdown
Contributor Author

Closing: we're moving size-analysis content to its own objectstore usecase instead of moving snapshots, so no CLI change is needed.

@NicoHinderling
NicoHinderling force-pushed the feat/snapshots-objectstore-usecase-option branch 3 times, most recently from c2b90bf to 30fd559 Compare September 14, 2026 18:46
NicoHinderling added a commit that referenced this pull request Sep 14, 2026
…124194)

Prepares snapshot storage to move from the shared `preprod` Objectstore
usecase into `preprod_snapshots`. This PR registers the new usecase and
adds cross-usecase reads, HEAD checks, and deletes through
`SnapshotStorage`. Writes remain on `preprod`, so this deploy does not
move stored content or change the write destination. The fallback metric
records when reads use the secondary usecase.

The upload-options endpoint also accepts `usecase=auto` and echoes the
chosen usecase in its response. Requests without the parameter keep
receiving `preprod` tokens for compatibility with existing CLI releases.
getsentry/sentry-cli#3408 and getsentry/cli#1587 opt into this contract;
#124210 adds the default-off option that selects `preprod_snapshots` for
new writes.

Register `preprod_snapshots` in the Objectstore service configuration
**before deploying this prerequisite**: fallback reads may access it
whenever an object is missing from `preprod`. Deploy the backend stack
fully before enabling the write-selection option. No content-copy
migration is required; existing uploads remain readable through the
fallback.
Base automatically changed from feat/snapshots-objectstore-usecase to master September 14, 2026 22:33
@NicoHinderling
NicoHinderling requested a review from a team as a code owner September 14, 2026 22:33
Add preprod.snapshots.snapshots-usecase.enabled (default false). When
on, server writes and usecase=auto tokens target the snapshots usecase
instead of preprod. Reads keep falling back across both.
@NicoHinderling
NicoHinderling force-pushed the feat/snapshots-objectstore-usecase-option branch from 30fd559 to e3cc176 Compare September 14, 2026 22:36
NicoHinderling added a commit that referenced this pull request Sep 14, 2026
Adds `UsecaseId.PREPROD_SIZE` (`preprod_size`) and explicit `image_type`
handling in the shared image endpoint. Uses the shared snapshot reader
introduced by #124194, now on `master`.

- `preprod_size_app_icon`: read `preprod_size`, then `preprod` if
missing.
- `preprod_snapshots`: delegate to `get_snapshot_storage`, preserving
its two-namespace reader.
- No `image_type`: use the shared snapshot reader for legacy requests.
IDs remain opaque; no prefix selects a reader.
- Any other value: reject before storage access.

The independent #124210 controls the snapshot reader's primary/write
usecase. The reader defaults to `preprod` first; when that option is
enabled, it tries `preprod_snapshots` first and falls back to `preprod`.
This PR neither enables that option nor resolves its outstanding rollout
blocker.

Both size-icon fallback reads and snapshot GET/HEAD fallback reads emit
`preprod.objectstore.fallback` logs with the reader's `image_type`,
`operation`, and `found` result for the Preprod Health dashboard. The
logs count completed secondary reads, not primary hits or storage
errors; the snapshot secondary namespace depends on the write option.
Marks `UsecaseId.PREPROD` deprecated for new code while retaining
migration compatibility. Cleanup is tracked in EME-1313, and both
January 1, 2027 TODOs remain.

Companion frontend: #124330. Companion writer: getsentry/launchpad#684.
Confirm the Objectstore expiration safeguards from getsentry/ops#23486
are deployed, deploy the shared reader and this backend change, then
deploy the frontend before switching Launchpad writes. Cached clients
that omit `image_type` can still miss icons stored only in
`preprod_size` until refreshed; the default reader does not infer icon
type from IDs.

@jamieQ jamieQ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes generally make sense. The (presumed) bot comment seems like it probably needs a solution:

Outstanding review blocker: comparison outputs currently reuse artifact-pair keys. Recomputing a comparison across a usecase switch and then rolling back can read an older manifest or mask while the database describes the newer result. Before rollout, comparison generations need authoritative, generation-qualified storage references with rolling-deploy compatibility.

Comment thread src/sentry/options/defaults.py Outdated
)

register(
"preprod.snapshots.snapshots-usecase.enabled",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 'snapshots-usecase.enabled' seems slightly vague since it doesn't imply that it has something to do specifically with objectstore/storage.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the name

Rename preprod.snapshots.snapshots-usecase.enabled to
preprod.snapshots.objectstore.snapshots-usecase.enabled so the option
name makes clear it selects the objectstore usecase for snapshot writes.
@NicoHinderling

Copy link
Copy Markdown
Contributor Author

Changes generally make sense. The (presumed) bot comment seems like it probably needs a solution:

Outstanding review blocker: comparison outputs currently reuse artifact-pair keys. Recomputing a comparison across a usecase switch and then rolling back can read an older manifest or mask while the database describes the newer result. Before rollout, comparison generations need authoritative, generation-qualified storage references with rolling-deploy compatibility.

i thought for a while about this and:

The bad sequence needs all three of: the option turns on, a staff user hits the recompare endpoint for a pair that was already compared before the flip, and then the option turns off again. Ordinary comparison runs never recompute a successful row, so without a staff recompare a rollback is also safe.

in order to "patch" this concern, we'd have to add some sort of versioning.. and honestly I feel like this change should go through pretty smoothly so I'm biased to just ship as is since we don't do the recompare oftens anyways and this should work out

@NicoHinderling
NicoHinderling enabled auto-merge (squash) September 15, 2026 16:12
@NicoHinderling
NicoHinderling merged commit e69862e into master Sep 15, 2026
90 checks passed
@NicoHinderling
NicoHinderling deleted the feat/snapshots-objectstore-usecase-option branch September 15, 2026 16:17
NicoHinderling added a commit to getsentry/sentry-cli that referenced this pull request Sep 15, 2026
Snapshot uploads now request `usecase=auto` and build their Objectstore
session from the usecase returned by the server rather than hardcoding
`preprod`. Both batched HEAD checks and image uploads use that session,
keeping their paths aligned with the minted token. This lets the backend
move new snapshot content into `preprod_snapshots` without another CLI
change.

Servers that omit the response field default to `preprod`, so existing
Sentry versions retain their current behavior. The integration coverage
exercises actual upload-options requests, authenticated Objectstore
batches, and snapshot creation for `preprod_snapshots`, explicit
`preprod`, and an omitted usecase. Object keys stay unchanged; the first
upload after a usecase switch re-uploads images once because
deduplication is scoped by usecase.

Pairs with getsentry/sentry#124194 (cross-usecase reads and the
upload-options contract) and getsentry/sentry#124210 (default-off write
selection). getsentry/cli#1587 implements the same contract in the
replacement CLI; both clients are being updated. Register
`preprod_snapshots` in Objectstore before deploying the backend
prerequisite and enable the option only after the backend rollout. Older
CLI releases keep writing to `preprod` and remain readable through
fallback reads.
NicoHinderling added a commit to getsentry/cli that referenced this pull request Sep 15, 2026
Snapshot uploads now request `usecase=auto` from the upload-options
endpoint and use the server's returned usecase for both HEAD
deduplication checks and PUT uploads, rather than hardcoding `preprod`.
This keeps Objectstore paths aligned with the minted token and lets the
backend move new snapshot content into `preprod_snapshots` without
another CLI change.

When older servers omit `usecase`, response parsing defaults it to
`preprod`, preserving the existing upload behavior. Object keys remain
unchanged; because deduplication is scoped by usecase, the first upload
after a server-side switch re-uploads images once.

Pairs with getsentry/sentry#124194 (cross-usecase reads and the
upload-options contract) and getsentry/sentry#124210 (server-controlled
write selection). getsentry/sentry-cli#3408 implements the same contract
in the legacy CLI; both clients are being updated. The backend option
remains off by default: register `preprod_snapshots` in Objectstore
before deploying the backend prerequisite, then enable the option only
after the backend rollout. Older CLI releases keep writing to `preprod`
and remain readable through fallback reads.
BYK pushed a commit to getsentry/sentry-mcp that referenced this pull request Sep 18, 2026
Snapshot uploads now request `usecase=auto` from the upload-options
endpoint and use the server's returned usecase for both HEAD
deduplication checks and PUT uploads, rather than hardcoding `preprod`.
This keeps Objectstore paths aligned with the minted token and lets the
backend move new snapshot content into `preprod_snapshots` without
another CLI change.

When older servers omit `usecase`, response parsing defaults it to
`preprod`, preserving the existing upload behavior. Object keys remain
unchanged; because deduplication is scoped by usecase, the first upload
after a server-side switch re-uploads images once.

Pairs with getsentry/sentry#124194 (cross-usecase reads and the
upload-options contract) and getsentry/sentry#124210 (server-controlled
write selection). getsentry/sentry-cli#3408 implements the same contract
in the legacy CLI; both clients are being updated. The backend option
remains off by default: register `preprod_snapshots` in Objectstore
before deploying the backend prerequisite, then enable the option only
after the backend rollout. Older CLI releases keep writing to `preprod`
and remain readable through fallback reads.
BYK pushed a commit to getsentry/sentry-mcp that referenced this pull request Sep 23, 2026
Snapshot uploads now request `usecase=auto` from the upload-options
endpoint and use the server's returned usecase for both HEAD
deduplication checks and PUT uploads, rather than hardcoding `preprod`.
This keeps Objectstore paths aligned with the minted token and lets the
backend move new snapshot content into `preprod_snapshots` without
another CLI change.

When older servers omit `usecase`, response parsing defaults it to
`preprod`, preserving the existing upload behavior. Object keys remain
unchanged; because deduplication is scoped by usecase, the first upload
after a server-side switch re-uploads images once.

Pairs with getsentry/sentry#124194 (cross-usecase reads and the
upload-options contract) and getsentry/sentry#124210 (server-controlled
write selection). getsentry/sentry-cli#3408 implements the same contract
in the legacy CLI; both clients are being updated. The backend option
remains off by default: register `preprod_snapshots` in Objectstore
before deploying the backend prerequisite, then enable the option only
after the backend rollout. Older CLI releases keep writing to `preprod`
and remain readable through fallback reads.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants