Skip to content

feat: allow deletion while a scan is running - #433

Closed
Kadrian wants to merge 1 commit into
Byron:mainfrom
Kadrian:delete-while-scanning
Closed

Kadrian wants to merge 1 commit into
Byron:mainfrom
Kadrian:delete-while-scanning

Conversation

@Kadrian

@Kadrian Kadrian commented Nov 16, 2026 •

Copy link
Copy Markdown

Re #273. On large trees (home dir, millions of files) the scan takes very long, and since 73f27e8 marked items can't be deleted until it ends – only Traversal already running.

73f27e8 blocked this as queued entries of deleted directories crashed with InvalidIndex. This fixes that cause instead:

  • on removal, the running traversal forgets the removed nodes by walker identity (directory id / root / preexisting path), as tree slots get reused and a stale index can't be probed
  • entries below forgotten directories are dropped
  • removing the node a traversal integrates into ends that traversal

Limits, kept deliberately small:

  • scan pauses while deletion runs (event loop unchanged, walker blocks on its bounded channel), resumes after
  • still blocked during dua clean discovery (staged candidates) and with pending --export (snapshot should match the scan)

Tests: entries_below_nodes_removed_during_traversal_are_dropped fails without the fix (entries land in reused slots); the former blocked-journey test now covers deletion during a scan plus the snapshot block. Also tried manually: deleted a directory ~0.5M entries into a scan of /, scan kept running past 3M entries without issue.

Long scans made deletion unusable until they finished (Byron#273).
The block existed as queued entries of deleted directories hit
removed or reused tree slots; drop those entries instead.

Scan pauses while deletion runs. Still blocked for `dua clean`
discovery and pending snapshot exports.
@Kadrian

Kadrian commented Nov 16, 2026 •

Copy link
Copy Markdown
Author

Hey @Byron , thank you so much for building this tool! Here's a suggestion, since I'm typically running "dua i" on my home dir and was unable to delete with the scan still ongoing (the full scan would take hours, can't wait so long). Hope you might find the fix valuable!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Deletion notifications are dropped for children whose scan events are still queued, allowing nonexistent files and their sizes to reappear in the tree after a partial deletion.

1 open issue
What changed in this PR

Enables deletion during ordinary interactive scans, addressing #273 by filtering queued entries for removed nodes rather than blocking deletion until scanning finishes.

Changes:

  • Tracks removed nodes by walker identity to prevent queued entries from using stale tree indices.
  • Allows deletion during ordinary scans while retaining blocks for cleanup discovery and pending snapshot exports.
  • Adds regression coverage for queued descendants and deletion during scans.
File Description
src/​traverse.rs Tracks removed identities, filters queued descendants, and tests traversal behavior.
src/​interactive/​app/​tests/​journeys_with_writes.rs Tests deletion during scans and blocking while snapshot export is pending.
src/​interactive/​app/​deletion_progress.rs Relaxes deletion blocking and notifies active traversal before removing tree nodes.

Review effort: Balanced

@Byron Byron self-assigned this Nov 16, 2026
@Byron

Byron commented Nov 16, 2026

Copy link
Copy Markdown
Owner

Thanks! While I like the features, I don't like the complexity that comes with it once AI is happy.
So I am changing it to instead be a queuing of a deletion, so it happens once the scan is done. At least I am investigating what it would mean to pause the traversal, and then resume it after deletion.
In any case, I will take it from here.

Lastly, today you can queue the items to delete, and then quit. This will print the paths to delete which can be deleted traditionally (albeit less fast).

@Kadrian

Kadrian commented Nov 16, 2026

Copy link
Copy Markdown
Author

today you can queue the items to delete, and then quit. This will print the paths to delete which can be deleted traditionally (albeit less fast).

Yes, I noticed that too, but slightly more effort to then manually take that list and remove.

I am investigating what it would mean to pause the traversal, and then resume it after deletion.

Not sure if it helps but from my perspective it would be totally fine if any affected sums are stale / become incorrect after deletion of some folders, as long as I can still delete as I go.

In any case, I will take it from here.

Thank you!

@Byron

Byron commented Nov 16, 2026

Copy link
Copy Markdown
Owner

I am closing this PR in favor of https://github.com/Byron/dua-cli/pulls/435 which will take a while until I get to it, and #434 which has landed already to simply queue deletion as intermediate patch.

While I understand it's not what you want just yet, there should be no issue in using your own build in the meantime.

@Byron Byron closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants