Skip to content

fix(deps): ship zsmooth per CPU baseline, so pre-AVX2 machines work (#82) - #88

Merged
StuartCameronCode merged 4 commits into
mainfrom
fix/zsmooth-non-avx2-cpus
Aug 31, 2026
Merged

fix(deps): ship zsmooth per CPU baseline, so pre-AVX2 machines work (#82)#88
StuartCameronCode merged 4 commits into
mainfrom
fix/zsmooth-non-avx2-cpus

Conversation

@StuartCameronCode

@StuartCameronCode StuartCameronCode commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Fixes #82.

The bug

Upstream builds zsmooth for an AVX2 baseline with no runtime dispatch (only haswell and znver4 assets exist for x86), so the bundled binary loads fine and then dies with an illegal instruction the instant a filter runs on any pre-2013 x86 CPU.

Confirmed by the reporter on a Celeron J4105 and a Core i7 870, neither of which has AVX at all. The symptom was a bare vspipe exited with exit code -10737417950xC000001D STATUS_ILLEGAL_INSTRUCTION, distinct from the 0xC0000005 access violation of the CTMF crash in #83. vspipe prints nothing on a native crash, so nothing anywhere named zsmooth.

Affected everything reaching core.zsmooth.*: CCD, Cnr4, SpotLess → RemoveDirt, Noise Reduction → mClean and TemporalDegrain2, and hybrid_mv. QTGMC was not affected — havsfunc uses rgvs. linux-x64 took the same haswell asset and had the same bug.

The fix

x86 bundles now ship both builds in vapoursynth/zsmooth/, outside the autoload directory, and the worker loads exactly one by path.

  • They cannot share a directory — each registers the namespace zsmooth, so whichever autoloads second is rejected. VAPOURSYNTH_EXTRA_PLUGIN_PATH takes only one directory (verified: a ;-separated pair silently loads just the first), and on macOS/Linux vapoursynth/plugins is autoloaded implicitly by R78 anyway.
  • Loading by path, not a forcens alias, so the namespace stays zsmooth and no call site changed. An alias would have needed a _zs() indirection through mclean.py, removedirt.py, temporaldegrain2.py and hybrid_mv.py — and left AVX2 users on the slow build in all four.
  • No selected build means no LoadPlugin, so a newer worker still works against a deps ≤ 1.9.0 bundle that autoloads a single zsmooth. The app can be upgraded before the deps download finishes, so that window is real.

Why not just ship one portable build

Measured (Zig 0.15.2, same source, 720x576, fps, best of 3), as a fraction of haswell speed:

CCD r0 CCD r1 Cnr4 CCD 16-bit RemoveGrain Repair Median
x86_64 0.50 0.33 0.33 0.39 0.68 0.66 0.73
x86_64_v2 0.72 0.63 0.71 0.69 0.69 0.65 0.73

That is 2–3x on the two filters the Chroma Denoise pass is made of, charged to every modern machine to serve the rare old one. Ratios hold at 1080p and multithreaded. A locally built -Dcpu=haswell matched the shipped binary within ±3%, so this is like-for-like and not a toolchain artefact.

x86_64_v2 is the fallback rather than plain x86_64: 1.4–2.1x faster on the filters that matter, 0.6 MB smaller, and it covers everything from Nehalem (2009) on — including both CPUs in the report.

The two builds produce identical chroma, so falling back costs throughput and nothing else.

Side effects

  • The portable build has no upstream asset, so Windows and Linux compile it with a pinned Zig toolchain. This is Windows' first from-source plugin — Zig brings its own libc and needs no MSVC.
  • macOS x64 gets faster. It builds from source for the Monterey 12.7.6 vspipe-bin Symbol not found #39 minos floor and had always used Zig's default SSE2 baseline — the 0.50/0.33 column. It now builds both, so Intel Macs get the fast path for the first time.
  • NTSTATUS decoding in format_exit_status: 0xC000001D now reads "illegal instruction: a plugin used a CPU feature this machine does not have" instead of -1073741795.

Verification

  • CCD end-to-end (encode and preview) on the haswell build, on the v2 build with haswell hidden, and against a simulated old-layout bundle.
  • The real Windows deps section run in isolation — it caught a bug: 2>&1 on git clone under $ErrorActionPreference = "Stop" turns ordinary progress into a terminating error in PowerShell 5.1. That would have been a red deps build.
  • The script's v2 output is byte-identical to the binary benchmarked above, so the measurements apply to what ships.
  • Rust 565 passed; Dart push gate passed.

Guards: test_154/test_155 (both scripts, both bundle layouts), zsmooth_never_offers_a_build_this_cpu_cannot_run (runs on every platform whatever hardware CI draws), and a Dart test that loads the chosen build and renders a frame — the fault is in the kernel, so constructing the node proves nothing.

This cannot be verified in CI. Every hosted runner has AVX2, so no CI job can exercise the fallback path. Locally: hide the haswell build and re-run. Intel SDE (sde -nhm --) is the only way to prove a build runs on a CPU you do not have.

Merge order

Needs deps 1.10.0 published before merge — deps-version.json already points at it.

🤖 Generated with Claude Code


Also in this PR: FFmpeg pinned to 9.0 everywhere

Not planned — found by running the deps build. Two blockers surfaced, and both had to be fixed here because deps 1.10.0 cannot be built without them.

FFmpeg had silently diverged across the bundles

FFmpeg, measured 2026-08-31 how
Windows master N-125978 (post-9.0) BtbN master-latest, unpinned
macOS x64 9.0.1 evermeet getrelease, unpinned
macOS arm64 9.0.1 martin-riedl latest, unpinned
Linux 7.1, two majors behind BtbN n7.1, pinned

FFmpeg is what interprets everything in pipeline_executor.rs — the accumulated -vf chain, the colour metadata flags, -ss/-frames:v trimming, the hardware-encoder options. A per-OS FFmpeg is a per-OS encoder, so the same job could encode differently depending on where it ran. All four now pin 9.0, which is where three of them already were.

The Linux pin is also what broke the build: BtbN publish every series to one rolling latest tag and drop old ones as they age, so n7.1 became a 404. It surfaced as tar: Error is not recoverable because curl had no -f and wrote the nine-byte Not Found body into the tarball.

A version pin against a rolling tag is not a pin — n7.1 was always going to 404, it was only a question of when. The fix pins the series (n9.0-latest), which is how all three upstreams actually publish.

Two guards, both needed:

  • assert_ffmpeg_series in each script runs ffmpeg -version on what was actually installed and fails if it isn't the pinned series. Pinning alone can't stop an upstream moving a URL.
  • app/test/ffmpeg_version_pin_test.dart (push gate) fails if the three pins disagree, if one stops being a bare major.minor, or if a script stops asserting. Without it the pins are just comments.

The parser rejects a master build outright, so reverting any platform to an unpinned URL is a red build rather than a silent regression.

macOS x64 haswell zsmooth didn't compile

The build added in the first commit failed inside zsmooth's Zig FFTW port:

kalloc.c:91:10: error: call to undeclared library function 'memalign'

The port sets HAVE_MEMALIGN on every non-Windows target, but macOS has no memalign() — it's declared in <malloc.h>, which the same file already knows macOS lacks (HAVE_MALLOC_H is gated on !is_mac). kalloc.c only reaches that branch at MIN_ALIGNMENT 32, i.e. with AVX enabled — so it's invisible at every SSE-level baseline and killed only the haswell build. HAVE_POSIX_MEMALIGN is already true, so clearing it falls through to posix_memalign.

Patched by cloning the FFTW fork at the ref build.zig.zon names and repointing it as a path dependency — path deps take no hash, so it survives a cache wipe, unlike editing Zig's global package cache. Pre- and post-checks hard-fail, so an upstream fix surfaces as an error telling you to drop the patch rather than silently doing nothing.

Verified by cross-compiling from Windows (-Dtarget=x86_64-macos.12.0 -Dcpu=haswell): reproduces the failure exactly, and builds clean with the patch. The target doesn't have to be the host — much cheaper than a macOS CI round trip.

StuartCameronCode and others added 4 commits August 28, 2026 22:14
)

Upstream builds zsmooth for an AVX2 baseline with no runtime dispatch, so
the bundled binary loaded fine and then died with an illegal instruction
(0xC000001D) the instant a filter ran on any pre-2013 x86 CPU. Confirmed by
the reporter on a Celeron J4105 and a Core i7 870, neither of which has AVX
at all. It hit CCD, Cnr4, SpotLess/RemoveDirt, mClean and TemporalDegrain2;
QTGMC was unaffected, since havsfunc uses rgvs.

x86 bundles now carry both a haswell and an x86_64_v2 build in
vapoursynth/zsmooth/, outside the autoload directory, and the worker loads
exactly one by path. They cannot share a directory — each registers the
namespace `zsmooth`, so whichever autoloads second is rejected — and
VAPOURSYNTH_EXTRA_PLUGIN_PATH takes only one directory, verified. Loading by
path rather than under a forcens alias keeps the namespace intact, so no call
site changes and the vendored modules stay on the fast build too.

Shipping one portable build instead was measured and rejected: x86_64 runs
CCD at 0.50x and Cnr4 at 0.33x of haswell (x86_64_v2 0.72x / 0.71x), i.e.
2-3x on the two filters the Chroma Denoise pass is made of, charged to every
modern machine. A locally built -Dcpu=haswell matched the shipped binary
within 3%, so the comparison is like-for-like. The two builds produce
identical chroma, so the fallback costs throughput and nothing else.

The portable build has no upstream asset, so Windows and Linux compile it
with a pinned Zig toolchain — Windows' first from-source plugin, which needs
no MSVC. macOS x64 builds both, which also makes Intel Macs fast for the
first time: that arch had always used Zig's default SSE2 baseline.

No selected build means no LoadPlugin, so a newer worker still works against
a deps bundle from before the split (verified end to end both ways).

Also decode Windows NTSTATUS crash codes in the reported error. A bare
"exit code -1073741795" was all this report had to go on, and 0xC0000005
(issue #83's CTMF crash) reads almost identically.

Guards: test_154/test_155 cover both scripts and both bundle layouts;
zsmooth_never_offers_a_build_this_cpu_cannot_run runs on every platform
whatever hardware CI draws; the Dart side loads the chosen build and renders
a frame, since the fault is in the kernel. deps-expected-plugins.json entries
may now be bundle-relative paths, understood by all three packaging guards.
Needs deps 1.10.0 published before merge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… zsmooth

Two blockers for the deps 1.10.0 build, found by running it.

FFmpeg had silently diverged across the bundles, and had been that way for
months:

  Windows     master N-125978 (post-9.0)  BtbN master-latest, UNPINNED
  macOS x64   9.0.1                       evermeet getrelease, UNPINNED
  macOS arm64 9.0.1                       martin-riedl latest, UNPINNED
  Linux       7.1, two majors behind      BtbN n7.1, pinned

FFmpeg is what interprets everything in pipeline_executor.rs — the accumulated
-vf chain, the colour flags, -ss/-frames:v trimming, the hardware-encoder
options — so a per-OS FFmpeg is a per-OS encoder. All four now pin 9.0, which
is where three of them already were.

The Linux pin is also what broke: BtbN publish every series to one rolling
`latest` tag and drop old ones as they age, so n7.1 became a 404 and the deps
build died at `tar: Error is not recoverable`. It read as a tar bug because
curl had no -f and wrote the nine-byte "Not Found" body into the tarball. Every
FFmpeg fetch now uses -f, and each script runs `ffmpeg -version` on what it
actually installed and fails if it is not the pinned series — pinning alone
cannot stop an upstream moving a URL. ffmpeg_version_pin_test.dart fails the
push gate if the three pins disagree, if one stops being a bare major.minor, or
if a script stops asserting; without it the pins are only comments. The parser
rejects a `master` build outright, so reverting to an unpinned URL is red.

Separately, the macOS x64 haswell zsmooth added in the previous commit did not
compile. zsmooth's Zig fftw port sets HAVE_MEMALIGN on every non-Windows
target, but macOS has no memalign() — it is declared in <malloc.h>, which the
same file already knows macOS lacks (HAVE_MALLOC_H is gated on !is_mac).
kalloc.c only reaches that branch at MIN_ALIGNMENT 32, i.e. with AVX enabled,
so it is invisible at every SSE-level baseline and killed only the haswell
build. HAVE_POSIX_MEMALIGN is already true, so clearing it falls through to
posix_memalign. Patched by cloning the fftw fork at the ref build.zig.zon names
and repointing it as a path dependency, which takes no hash and so survives a
cache wipe; pre- and post-checks hard-fail so an upstream fix surfaces as an
error telling you to drop the patch.

Verified by cross-compiling from Windows (-Dtarget=x86_64-macos.12.0
-Dcpu=haswell): reproduces the failure exactly, and builds clean with the
patch. The target does not have to be the host, which is far cheaper than a
macOS CI round trip.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The interpolated path "$FullTargetDir\ffmpeg\ffmpeg.exe" reached the file with
its \f collapsed into a literal form feed (0x0C), so the check ran against
"$FullTargetDir<FF>fmpeg<FF>fmpeg.exe" and the deps build died immediately
after a successful download.

Join-Path instead: no backslashes to escape, so the whole class of corruption
is gone rather than fixed once.

Verified by running the real section against a scratch target — it fetches
n9.0, installs, and reports "FFmpeg 9.0 installed (matches the pinned 9.0
series)" for n9.0.1-11-ge47273f4d9. That harness is also what caught the git
2>&1 bug earlier; PowerShell sections in this script are worth running before
pushing, because a deps build is a slow way to find a typo.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sed backreference in assert_ffmpeg_series arrived in the file as a literal
0x01, so the check read an empty version and failed every Linux and macOS deps
build with "FFmpeg is  but this bundle pins 9.0" — after a download that had
worked. Same class as the form feed in the Windows path check: escapes mangled
in transit rather than a logic error.

Rewritten with grep -oE and a [.] bracket expression, so the parser contains no
backslashes at all and there is nothing left to mangle.

Exercised against real binaries rather than reasoned about: a BtbN n9.0.1 build
passes, the unpinned master build in deps/ is rejected (it reports
N-125978-..., which is exactly the drift this is meant to catch), and a missing
binary is rejected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode merged commit 2f6256d into main Aug 31, 2026
8 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problems with VHS preset on Windows version

1 participant