Repository navigation
fix: grant no locks once the application starts shutting down - #41
Merged
Merged
Conversation
beforeApplicationShutdown() released a snapshot of the held locks, and each release woke the acquire() calls waiting for its key. A woken acquire() took the lock at once and added it to the held set after the snapshot, so nothing released it: it kept renewing after onApplicationShutdown() had closed the store's connection, logged "Could not renew the lock", and was then reported lost. An acquire() whose store call was still in flight when the hook began had the same problem. From the start of the hook, acquire() resolves null without a store call, a grant that lands after that goes back to the store, and the callers waiting for a key stop waiting at once. withLock() then rejects with a LockNotAcquiredError whose new `reason` is 'shutdown', with a message that says so. Closes #40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
beforeApplicationShutdown()releases a snapshot of the held locks, and each release wakes theacquire()calls waiting for that key. A wokenacquire()gets the lock straight away, and that lock is added after the snapshot, so nothing releases it. It keeps renewing afteronApplicationShutdown()has closed the store's connection: each renewal fails, and the lock is later reported lost. Anacquire()whose store call was in flight when the hook began ends up the same way.Issue Number: #40
What is the new behavior?
From the start of
beforeApplicationShutdown(),Locksgrants no new locks:acquire()(and the internaltryAcquire()) resolvesnullwithout a store call.null.acquire()returning, resolvesnullinstead of a releasedLock.withLock()rejects withLockNotAcquiredErrorwithreason: 'shutdown'and the messageCould not acquire the lock "k": the application is shutting down.reasonis a new optional constructor argument that defaults to'taken', so the existing messages are unchanged.Thanks to @johnnyhuirilef for the report, the reproduction and the
closingflag, which this PR builds on. On top of it, this PR wakes the sleeping waiters, returnsnullfor a grant whose lock the hook already released, and adds thereasononLockNotAcquiredError.Does this PR introduce a breaking change?
Other information
A grant that only lands after
onApplicationShutdown()has closed the store can't be given back, so that key stays taken for onettl. Elections and ticks already behave this way at shutdown, and the hook doesn't wait for such grants, so a stalled store doesn't hold up the deploy.The docs (
content/reliability/locks.md, theacquire()/withLock()paragraph, the shutdown paragraph and the errors table) need a follow-up in docs.nestjs.com.