Skip to content

fix(local-disk): remove folders a delete or move leaves empty - #35

Merged
kamilmysliwiec merged 1 commit into
masterfrom
fix/local-disk-prune-empty-dirs
Oct 23, 2026
Merged

kamilmysliwiec merged 1 commit into
masterfrom
fix/local-disk-prune-empty-dirs

Conversation

@kamilmysliwiec

Copy link
Copy Markdown
Member

PR Checklist

Please check if your PR fulfills the following requirements:

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Other... Please describe:

What is the current behavior?

LocalDisk.delete() unlinks a file and its sidecar but keeps the folder it was in. A later put(), copy() or move() onto that folder's name then fails with StorageKeyConflictError, although exists() and list() show no key there. InMemoryDisk and S3Disk accept the same steps. move() leaves the source's folder behind the same way.

Issue Number: #33

What is the new behavior?

Thanks to @johnnyhuirilef for the report, the repro and both ideas for the fix. This PR combines them:

  • Prune on delete and move (idea 1). After a delete or move, each folder it left empty is removed, deepest first, up to the root (never the root itself, never .nest-storage). It uses non-recursive rmdir and stops at the first folder that still holds something or can't be removed. This keeps the tree from filling with empty folders that list() would still walk.
  • Write-side fallback (idea 2). When the final rename hits a folder at the key, the write tries rmdir on it. If the folder was empty, the rename is retried; if not, it is still a StorageKeyConflictError. This covers roots that already hold empty folders from earlier versions, a crash between unlink and rmdir, or another tool.
  • Race handling. If a concurrent delete prunes a write's folder after ensureParent(), the rename fails with ENOENT. The write then creates the folder again and retries, up to 3 attempts. If a file was put at that folder's name in between, the write gets StorageKeyConflictError, so the documented rule that a and a/b can't both exist still holds.

Sidecars are flat files under .nest-storage/meta/, so they leave no folders behind. exists() and list() are unchanged.

Tests: one test in the shared disk suite runs on every disk (it failed only on LocalDisk before). LocalDisk-only tests cover pruning, an empty folder left by another tool, a folder that still holds something (a conflict), and symbolic links. Two deterministic race tests stub rename.

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

  • Behaviour change: any folder under the root that a delete or move leaves empty is removed, including folders created by hand (for example a pre-made uploads/). The next write that needs it creates it again.
  • The write-side fallback removes one level only. An already-present chain of empty folders such as a/b/ still blocks put('a') until a delete through the disk prunes it.
  • Windows: EPERM (rename onto a folder) and EPERM/EBUSY (rmdir) are handled, but untested, since CI runs on Linux only.

Deleting the last file under a folder left the folder on disk, so a
later put, copy or move onto the folder's name failed with
StorageKeyConflictError although no key was there. A move left the
source's folder behind the same way.

A delete or move now removes each folder it leaves empty, up to the
root. On a write, an empty folder at the key is removed before the
rename, which fixes roots that already hold such folders. If a
concurrent delete removes the write's folder before the rename, the
write creates it again and retries.

Closes #33
@kamilmysliwiec
kamilmysliwiec merged commit 0ff4458 into master Oct 23, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant