You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add concurrency control to LocalFileDescriptorStore #686
Currently, LocalFileDescriptorStore in server/app/backend/local_file.py has the same concurrency problems that #591 fixes for LocalFileIdentifiableStore (see #554):
Multiple instances (e.g. one per uWSGI worker) can operate on the same directory. Each instance keeps its own _object_cache, which is not invalidated when another instance modifies the files.
add() and commit() check for the file's existence before writing it, without holding a lock. A concurrent add() or discard() can create or delete the file in between, so writes are silently overwritten or a discarded descriptor is written back.
add() and commit() write directly into the target file with open(..., "w"). A crash during the write leaves a truncated file, and a concurrent read can see a half-written one. LocalFileIdentifiableStore already avoids this by writing to a temp file and swapping it in with os.replace().
LocalFileDescriptorStore should get the same mechanism as LocalFileIdentifiableStore after #591:
Acquire a DirectoryLock on the storage directory, including the disable_strict_locking opt-out for systems without fcntl
Perform file and cache operations under a single _write_lock
Write atomically via a temp file and os.replace()
Add a close() method that releases the directory lock
Ideally, both stores share this logic, so that it does not have to be maintained twice.
Activity