Skip to content

Watch folders: wait for a file's data, and retry a file that failed - #1127

Merged
KnifeLemon merged 2 commits into
mainfrom
fix/watch-folder-burst
Nov 4, 2026
Merged

KnifeLemon merged 2 commits into
mainfrom
fix/watch-folder-burst

Conversation

@KnifeLemon

Copy link
Copy Markdown
Owner

Refs #767 (the reporter still owes a log, so this doesn't close it)

What I measured

improper image header ... ReadPNGImage comes from ImageMagick. It means the first bytes of the file aren't a signature ImageMagick knows. With the real engine:

File named .png holding Result
a PNG, a JPEG or a BMP converts (the engine goes by content)
zeros, or zeros at the start improper image header
text, or junk before a PNG improper image header
half a PNG unexpected end-of-file (another message)

A burst alone doesn't cause it. Real filee watch, files dropped with the Shell's MoveHere/CopyHere (the engine Explorer uses):

  • 150 PNGs of 1.9 MB, moved: 150 done, all in originals. Copied: the same.
  • 2500 small PNGs, moved, Keep mode: 2500 done. InternalBufferSize and the Error rescan lose nothing here, so "only 70" isn't a missed event as far as I can reproduce.

This doesn't reproduce the report. It closes what is really wrong in that path.

What was wrong

  1. A file with no data yet counted as ready. A program can reserve a file's size, let go of it and write later. For those seconds the file keeps its size and time and opens without sharing. Reproduced end to end on main: failed: improper image header, then done. Now it waits for data at its start, for up to 30 s: some formats do start with zeros (ISO), and a damaged file must still get its error. It reads 4 KB, so DICOM's 128-byte preamble doesn't wait.
  2. A failed file was never looked at again before it changed. _handled is filled before the conversion runs. It now gets one more turn after 10 s, per version of the file, so a broken file reports twice and then stays quiet. The cost: a broken file makes a second failed job in the toast, and a long video that fails at the end runs twice.
  3. A batch that could not start was forgotten without a word (_convert returned null or threw). It gets the same one retry.

The change to FolderWatcher is small and additive, as a private branch edits other parts of that file.

Not changed

An overflow rescan in Keep mode also notices files that were already there and converts them again. Rare (it takes thousands of events), and unrelated to this report.

Tests

FolderWatcherTests: 7 new cases fail against main's FolderWatcher.cs. FolderWatcherTests, WatchRuleOptionsTests, WatchResultsTests, FolderCopyTests: 87 passed, 2 skipped. CLI watch tests: 8 passed. dotnet format --verify-no-changes clean on the changed files.

🤖 Generated with Claude Code

KnifeLemon and others added 2 commits November 4, 2026 23:14
A file that opens without sharing and keeps its size and time for 2 seconds can still hold nothing: a program can
reserve the size, let go, and write later. ImageMagick then says "improper image header" for a file that is fine a
moment later. A watch folder now waits for data at the start of the file (up to 30 seconds, as some formats do start
with zeros and a damaged file must get its error).

A file that failed was never looked at again before it changed, because it counted as handled before its conversion
ran; a batch that could not start at all was forgotten without a word. Both now get one more turn after 10 seconds.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test rewrote a file with the same size right after writing it and expected the settle wait to start over. Within
the file system's clock tick the time stays the same, so nothing shows that the file changed: on CI it converted at
once. That was the file with its data in it, so the watcher did nothing wrong. A real writer comes later, so the test
now gives the rewrite a later time.

Refs #767

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@KnifeLemon
KnifeLemon merged commit 102d921 into main Nov 4, 2026
51 of 55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant