[kyteinsky]

fixes: #175
related:

• Internal embedder with the default config only looks at the first 16 tokens of a chunk context_chat_backend#359
• feat: add endpoints to wipe the vector db efficiently context_chat_backend#374

🤖 AI (if applicable)

• The content of this PR was partly or fully generated using AI (N/A)

[marcelklehr]

1. A crawl that is already running survives the wipe. lib/Service/ReindexService.php:723
   • The wipe calls jobList->remove(), but a StorageCrawlJob that is already running schedules its next batch anyway.
   • That job then re-queues files and writes last_enqueued_db_id again after resetStats() cleared it.
   • So files get indexed again without anyone running reindex.
   • A running SchedulerJob has the same problem: it adds crawl jobs back for every mount.
2. The pause check can be beaten by a request already in progress. lib/Controller/QueueController.php:259
   • isPaused() is checked once, before the fetch-and-lock step.
   • A fetch request that started before pause() can lock and hand out documents after waitForHandedOutDocuments() has already counted zero locked items.
   • The backend then indexes those documents after the wipe.
   • Fix: check the pause again after locking, and unlock the items if it is now paused.
3. The pause can expire in the middle of a scoped wipe. lib/Service/FetchGuard.php:347
   • The pause lasts for the drain timeout plus one request timeout plus 5 minutes.
   • But a scoped wipe makes one deleteProvider request per provider.
   • With several -p flags the pause can run out partway through, and indexing restarts while the wipe is still going.
   • Fix: extend the pause on each backend request, or make it long enough for every provider.
4. A scoped wipe waits for every provider's locked documents. lib/Service/FetchGuard.php:397
   • countHandedOutDocuments() isn't filtered by provider.
   • So wipe -p someapp__x can wait 30 minutes for, and then time out on, unrelated file indexing.
   • The way out is --ignore-handed-out, but that also skips the wait for the provider actually being wiped.
5. The initial indexing is reported as complete right after a wipe. lib/Service/ReindexService.php:781
   • resetStats() deletes last_indexed_time.
   • On the next DELETE request from the backend, setInitialIndexCompletion() sees no StorageCrawlJob and an empty queue, so it records the initial indexing as finished.
   • That happens before SchedulerJob has crawled anything.
   • Fix: also check for a pending SchedulerJob.
6. The drain doesn't wait for actions that were already handed out. lib/Service/FetchGuard.php:396
   • Locked items in context_chat_action_queue (for example access updates and moves) aren't waited for.
   • The full wipe still clears that queue and wipes the vector DB underneath them.
   • This is the same race the document drain exists to prevent.
7. A malformed backend response now crashes the command instead of reporting an error. lib/Service/LangRopeService.php:442
   • The PR removed the check for the files__default key, so nothing confirms the response is an array.
   • A body of null makes the method throw a TypeError instead of a RuntimeException.
   • context_chat:statistics and context_chat:wipe -p only catch RuntimeException, so they crash.
   • Fix: check is_array($response).

[kyteinsky]

thanks for the review! Addressed all of the points.

1. fixed
2. fixed
3. kind of fixed with a long pause of max 24h
4. fixed
5. fixed
6. doesn't matter
7. fixed