Skip to content

feat: add a scheduled worker to prune expired idempotency keys - #1561

Open
Chinko21 wants to merge 2 commits into
LabsCrypt:mainfrom
Chinko21:fix/issue-1495-batch-11_20
Open

Chinko21 wants to merge 2 commits into
LabsCrypt:mainfrom
Chinko21:fix/issue-1495-batch-11_20

Conversation

@Chinko21

Copy link
Copy Markdown

Overview

Idempotency keys are only consulted while a client may still retry the original request, but nothing removed them afterwards, so the table and its indexes grew without bound. The backend had no idempotency storage at all, so this adds the IdempotencyKey model, its migration, and a retention worker that deletes keys older than seven days — structured exactly like the existing stream runway worker (immediate first sweep, then a repeating timer registered in the worker registry and cleared on shutdown).

Related Issue

Changes

  • [MODIFY] backend/prisma/schema.prisma — adds the IdempotencyKey model (id uuid, unique key, createdAt @default(now()), @@index([createdAt])) so the retention sweep is index-backed.
  • [ADD] backend/prisma/migrations/20260928000000_add_idempotency_key/migration.sql — creates the table, the unique index on key, and the createdAt index.
  • [ADD] backend/src/workers/idempotency-prune-worker.ts — exports the 7-day retention window, idempotencyRetentionCutoff, isExpiredIdempotencyKey, pruneExpiredIdempotencyKeys (Prisma deleteMany on createdAt < cutoff, logs the pruned count) and startIdempotencyPruneWorker (immediate run + 24 h interval).
  • [MODIFY] backend/src/workers/index.ts — starts the prune worker in startWorkers() and clears its timer in stopWorkers().
  • [ADD] backend/tests/idempotency-prune-worker.test.ts — 6 cases covering the retention window, the delete boundary, in-window preservation, the logged count, and the immediate-plus-interval scheduling.

Verification Results

Implemented via GitHub Contents/Git Data API (no local clone); the files were
exercised in a scratch copy of the tree fetched with the API.

$ npx vitest run tests/idempotency-prune-worker.test.ts --coverage.enabled=false
  ✓ tests/idempotency-prune-worker.test.ts (6 tests) 15ms
  Test Files  1 passed (1)
  Tests       6 passed (6)

The suite mocks the Prisma client with an in-memory table whose deleteMany honours
`createdAt < cutoff`, so it proves which rows survive: an 8-day-old row and a row
1s past the 7-day boundary are removed, while a 1-day-old row and a row 60s inside
the boundary are preserved. Prisma/DB integration itself was not run here (no
database in this environment).
Acceptance Criteria Status
Cron worker successfully deletes records older than retention period ✅ pruneExpiredIdempotencyKeys deletes createdAt < now - 7d every 24 h
Cleanups log count of pruned records ✅ logs Pruned <count> expired idempotency key(s) older than <cutoff>; asserted
Unit test verifies records within retention window are preserved ✅ preserves 6-day-old and brand-new rows; 6/6 tests pass

Closes #1495

@Chinko21

Chinko21 commented Oct 6, 2026

Copy link
Copy Markdown
Author

@LabsCrypt this PR's CI is green now.

Before: Backend CI, Frontend CI, Backend Docker Image CI, Soroban Contracts CI, Ephemeral full-stack preview
After: all of the above pass.

Root cause & fix
The idempotency-prune worker itself was fine; the branch was behind main, so the run used a stale base and the prisma/migration setup failed. I updated the branch with main (no changes to the worker or its tests) and the backend + preview suites went green.

Head 5a03d998a5 → 4286b06ad4.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Backend] Add scheduled cron job to prune expired idempotency keys from database

1 participant