Version 3 Rewriting by neon-nyan · Pull Request #11 · CollapseLauncher/SharpHDiffPatch.Core

neon-nyan · 2026-08-23T17:46:02Z

TODO: Description

Benchmark Result

Test: Single file, LZMA2 compressed

Method	Mean	Error	StdDev	Ratio	RatioSD	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
v2	19.46 ms	18.01 ms	0.987 ms	1.00	0.06	-	-	-	60.43 MB	1.00
v2+OldFastBuffer	57.55 ms	49.20 ms	2.697 ms	2.96	0.17	-	-	-	34.4 MB	0.57
v3	16.92 ms	18.52 ms	1.015 ms	0.87	0.06	468.7500	468.7500	468.7500	4.58 MB	0.08

Test: Directory, LZMA2 compressed

Method	Mean	Error	StdDev	Ratio	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
v2	7.028 s	0.3049 s	0.0167 s	1.00	-	-	-	937.31 MB	1.00
v2+OldFastBuffer	7.176 s	0.2930 s	0.0161 s	1.02	-	-	-	692.19 MB	0.74
v3	4.639 s	0.3283 s	0.0180 s	0.66	3000.0000	2000.0000	2000.0000	136.46 MB	0.15

Test: Single file, LZMA2 compressed - Native (Result: 25 ms)

old : "G:\UnityPlayer-SEA.dll"
diff: "G:\UnityPlayer.lzma2.diff"
out : "G:\UnityPlayer-CN-native.dll"
  input oldDataSize: 30144448
       diffDataSize: 39523
       diffDataType: HDiff
  saved oldDataSize: 30144448
  saved newDataSize: 30144408
       compressType: "lzma2" (need decompress 4)

  patch progress: [==============================]  100.0%
  patch ok!

hpatchz time: 0.025 s

Test: Directory, LZMA compressed - Native (Result: 4.906 s)

old  dir: "G:\Hi3SEA\"
diffFile: "G:\Hi3SEAtoCNExecOnly.lzma2.diff"
out  dir: "G:\Hi3CN-native\"
         diffDataSize: 107925175
         diffDataType: DirHDiff
         checksumType: "fadler64"
         oldPathIsDir: true
         newPathIsDir: true
       new path count: 196 (fileCount:178)
  copy from old count: 122 (copySize: 86831653)
   ref old file count: 56 (oldRefSize: 799499793)
   ref new file count: 56 (newRefSize: 927363046)
           oldSumSize: 886331446
           newSumSize: 1014194699
       diffDataType: HDiff (in DirHDiff)
  saved oldDataSize: 799499793
  saved newDataSize: 927363046
       compressType: "lzma2" (need decompress 4)

hpatchz run with checksum plugin: "fadler64" (checksumSets: new copy)

  patch progress: [==============================]  100.0%
  patch ok!

hpatchz dir patch time: 4.906 s

SharpHDiffPatch -> SharpHPatchZ

Attempt to implement a thread-safe Merged Stream Wrapper for Read/Write operations. Warning: Most of the Code for the reader are vibe-coded. So, need to check it further.

+ Struct type adjustment inside DirectoryPatchMetadata + Use UnmanagedArray<T> for some input-output ref lists instead of a direct array pointer + Fix CreateUnmanagedInt64List and CreateUnmanagedInt64As32List reading on backed numbers + Add extern functions: - shpz_get_last_errorA - shpz_get_last_errorW - shpz_patch_from_filepath - shpz_patch_from_FILE + Make some string arguments in extern functions auto-detect the encoding of the string - shpz_read_header_signature_string - shpz_init_from_filepath - shpz_patch_from_filepath + Make the ExceptionHelper throw functions returns the Exception instead of throwing within + Add implicit cast for Utf16UnmanagedString and NativeStringW + Remove unnecessary type size check on UnmanagedArray<T>.GetSpan() + Use numbered error code on ExceptionHelper instead of flags + Add callback allocation check on ProgressCallback to avoit ExecutionEngineException due to attempt on calling null callback.

Use .NET Framework 4.7.2 for testing the lowest version of .NET to be supported

Sign in to view

+#if NET6_0_OR_GREATER
+                    copyOverTcs.SetCanceled(token);
+#else
+                    copyOverTcs.SetCanceled();
+#endif


+ Reenable PrefetchedReadStream for Zstd

socket-security · 2026-08-24T23:07:17Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	microsoft.netframework.referenceassemblies@1.0.3

View full report

socket-security · 2026-08-24T23:07:18Z

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report

For compatibility testing. As per Microsoft documentation, .NET Standard should support down to .NET Framework 4.6.1 https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0

+ HPatch.TryGetHDiff13PatchMetadata (Extern: shpz_util_get_hdiff13_patch_metadata) + HPatch.TryGetHDiff19DirectoryPatchMetadata (Extern: shpz_util_get_hdiff19_patch_metadata)

sentry · 2026-08-28T19:32:05Z

+                catch (OperationCanceledException) when (token.IsCancellationRequested)
+                {
+#if NET6_0_OR_GREATER
+                    copyOverTcs.SetCanceled(token);


Bug: The corePatcherThread cancellation handler signals the wrong TaskCompletionSource, copyOverTcs instead of corePatcherTcs, causing a hang or an unhandled InvalidOperationException during cancellation.
_{Severity: HIGH}

Suggested Fix

In the catch (OperationCanceledException) block for corePatcherThread in HDiff13DerivedPatcher.cs, change the calls from copyOverTcs.SetCanceled() to corePatcherTcs.SetCanceled(). This ensures the correct task is marked as canceled.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: SharpHPatchZ/Patch/HDiff13DerivedPatcher.cs#L80 Potential issue: In the `corePatcherThread`, the `catch` block for `OperationCanceledException` incorrectly signals `copyOverTcs.SetCanceled()` instead of `corePatcherTcs.SetCanceled()`. This leaves the `corePatcherTcs` task incomplete. As a result, the main thread, waiting on `Task.WhenAll`, will hang. If the `copyOverThread` has already completed its task, attempting to cancel the already-completed `copyOverTcs` will throw an `InvalidOperationException`, leading to an unhandled exception that terminates the process.

Also affects:

SharpHPatchZ/Patch/HDiff13DerivedPatcher.cs:83~83

sentry · 2026-08-28T19:32:05Z

+
+    public void Dispose()
+    {
+        _lifetimeLock.EnterWriteLock();


Bug: The Dispose() method in RandomMergedStreamWrapper is not idempotent. A second call will throw an ObjectDisposedException because it tries to use a lock that was disposed on the first call.
_{Severity: MEDIUM}

Suggested Fix

Move the idempotency check if (Interlocked.Exchange(ref _disposed, 1) == 1) return; to be the first statement in the Dispose() method, before the _lifetimeLock.EnterWriteLock() call. This will ensure subsequent calls exit immediately without causing an exception.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: SharpHPatchZ/IO/Reader/RandomMergedStreamWrapper.cs#L141 Potential issue: The `Dispose()` method in `RandomMergedStreamWrapper` is not idempotent, violating the `IDisposable` contract. On the first call, it acquires a `ReaderWriterLockSlim` and disposes of it in the `finally` block. If `Dispose()` is called a second time on the same instance, it attempts to acquire the lock again via `_lifetimeLock.EnterWriteLock()`. Since the lock has already been disposed, this call throws an `ObjectDisposedException`. The idempotency check is placed after the lock acquisition, so it is never reached on the second call.

- shpz_util_get_hdiff13_patch_metadata -> shpz_util_get_patch_metadata - shpz_util_get_hdiff19_patch_metadata -> shpz_util_get_directory_patch_metadata

WARNING: Fully Vibe-coded. Further test is required

Btw, 6 🤷 7

sentry · 2026-08-29T13:09:55Z

+
+            Parallel.For(0, count, new ParallelOptions
+            {
+                MaxDegreeOfParallelism = (int)options.ParallelThreads


Bug: CopySimilarFilesContext.RunCopy() passes options.ParallelThreads directly to Parallel.For, which crashes if the value is 0. This occurs when PatchOptions is default-initialized.
_{Severity: HIGH}

Suggested Fix

In CopySimilarFilesContext.RunCopy(), validate options.ParallelThreads before passing it to Parallel.For. Reuse the logic from CorePatcher.GetWorkerCount() to safely convert a value of 0 to a valid thread count, like Environment.ProcessorCount.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not valid. Location: SharpHPatchZ/Patch/HDiff13DerivedPatcher.CopySimilarFilesContext.cs#L34 Potential issue: When a directory patch is initiated via the unmanaged API with `null` options, a default `PatchOptions` object is created where `ParallelThreads` is 0. This value is passed directly to `Parallel.For`'s `MaxDegreeOfParallelism` in `CopySimilarFilesContext.RunCopy()`, causing an `ArgumentOutOfRangeException`. Unlike `CorePatcher`, this code path lacks a guard to handle the zero value by converting it to a valid number of threads, such as `Environment.ProcessorCount`. This leads to a crash when patching directories that contain similar files.

neon-nyan added 27 commits August 9, 2026 00:55

Implement Header Reader API

17789bd

Rename project

2621f4d

SharpHDiffPatch -> SharpHPatchZ

Rename namespaces

187ce28

Move and rename extern prefix

9b7e5c2

Push local changes

cf55385

Implement RandomMergedStreamReader

c99778f

Attempt to implement a thread-safe Merged Stream Wrapper for Read/Write operations. Warning: Most of the Code for the reader are vibe-coded. So, need to check it further.

Merge branch 'main' into v3

d31b7e0

Split HDiff13DerivedPatcher codes

afe4586

Fix InvalidTypeWithPointersNotSupported error on older .NET

c26407a

Implement BigArrayPool

9aeb271

Fix LZMA chunk decompression

4fb74ed

Update Test

2a13db9

Use .NET Framework 4.7.2 for testing the lowest version of .NET to be supported

Fix duplicate NuGet reference

b762580

Implement RleCoverInfo read

84f1b87

Update HDiff13DerivedPatcher.CopySimilarFilesContext.cs

64b558c

Implement core patching

662369d

Add PatchOptions presets

28e0177

Fix LZMA2 decoding

fa039d7

Use native memory for buffer allocation

4f15f44

Use native memory and parallel decompress in LZMA/LZMA2

73b95ce

Add reading support for Kuro Games HDiff format

47707ba

Add input size sanity test for Kuro HDiff format

ebd84ed

Update ExceptionHelper.cs

b7112dc

Inline input file size check for Kuro Games format

15271be

Remove old project and full switch to SharpHPatchZ

522b43c

Move assets

d3c9186

sentry Bot reviewed Aug 23, 2026

View reviewed changes

Comment thread SharpHPatchZ/Patch/HDiff13DerivedPatcher.cs

Comment on lines +71 to +75

#if NET6_0_OR_GREATER

copyOverTcs.SetCanceled(token);

#else

copyOverTcs.SetCanceled();

#endif

This comment was marked as outdated.

Sign in to view

neon-nyan added 2 commits August 24, 2026 04:56

Fix Zlib support

2fbd7d4

Allow prefetched stream on any compressed streams

4917b77

neon-nyan added 5 commits August 24, 2026 21:52

Avoid buffer copy routine in RLE Adder

865334e

Use preloaded buffer to create string list

95bc90c

Optimize PatchOptions.BigBuffer

55e493d

Fix throw from PatchResult not returning error code

f08d862

Code cleanup

ff41482

sentry Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread SharpHPatchZ/Extension/StreamExtension.cs

Comment thread SharpHPatchZ/Patch/HDiff13DerivedPatcher.CorePatcher.cs

Fix double disposal on compression stream

87bbcb7

+ Reenable PrefetchedReadStream for Zstd

sentry Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread SharpHPatchZ/IO/Reader/RandomMergedStreamWrapper.cs

Comment thread SharpHPatchZ/IO/Compression/Lzma/LzmaInputStream.cs

Update projects

69cdadf

sentry Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread SharpHPatchZ/Patch/HDiff13DerivedPatcher.cs Outdated

Make SharpHPatchZ.Program compile-able on older .NET Framework 4.7.x

b5e50fb

neon-nyan added 3 commits August 25, 2026 06:10

[Program] Downgrade to .NET Framework 4.6.1

1560389

For compatibility testing. As per Microsoft documentation, .NET Standard should support down to .NET Framework 4.6.1 https://learn.microsoft.com/en-us/dotnet/standard/net-standard?tabs=net-standard-2-0

Update ZstdNet usage

cbf5a2e

Update SharpHPatchZ.Program.csproj

d3ff3c9

sentry Bot reviewed Aug 24, 2026

View reviewed changes

Comment thread SharpHPatchZ/IO/Reader/DecompressionTransitionStream.cs

neon-nyan added 2 commits August 25, 2026 17:07

Add utility function

b46bd99

+ HPatch.TryGetHDiff13PatchMetadata (Extern: shpz_util_get_hdiff13_patch_metadata) + HPatch.TryGetHDiff19DirectoryPatchMetadata (Extern: shpz_util_get_hdiff19_patch_metadata)

[Program] Print patch info

7229133

sentry Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread SharpHPatchZ/HPatch.UnmanagedExtern.cs Outdated

Code QA and fix read logic

7f0508a

sentry Bot reviewed Aug 28, 2026

View reviewed changes

neon-nyan added 7 commits August 29, 2026 03:30

[Program] Adjust options

c005bdf

[Program] Split commands for patch and info

4bfc2f4

Rename extern

f1507b0

- shpz_util_get_hdiff13_patch_metadata -> shpz_util_get_patch_metadata - shpz_util_get_hdiff19_patch_metadata -> shpz_util_get_directory_patch_metadata

Add experimental C header for native library support

76818a4

WARNING: Fully Vibe-coded. Further test is required

[Program] Add few property groups for AOT publish

776b31d

Explicitly use Hi3Helper.ZstdNet 1.6.7

71037d5

Btw, 6 🤷 7

Docs

500a599

sentry Bot reviewed Aug 29, 2026

View reviewed changes

Navigation Menu

Uh oh!

Version 3 Rewriting - #11

Version 3 Rewriting#11
neon-nyan wants to merge 64 commits into
mainfrom
v3

neon-nyan commented Aug 23, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

socket-security Bot commented Aug 24, 2026 •

edited

Loading

Uh oh!

socket-security Bot commented Aug 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

sentry Bot Aug 28, 2026

Uh oh!

sentry Bot Aug 28, 2026

Uh oh!

sentry Bot Aug 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

neon-nyan commented Aug 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO: Description

Benchmark Result

Test: Single file, LZMA2 compressed

Test: Directory, LZMA2 compressed

Test: Single file, LZMA2 compressed - Native (Result: 25 ms)

Test: Directory, LZMA compressed - Native (Result: 4.906 s)

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

socket-security Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

socket-security Bot commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sentry Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

sentry Bot Aug 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

neon-nyan commented Aug 23, 2026 •

edited

Loading

socket-security Bot commented Aug 24, 2026 •

edited

Loading

socket-security Bot commented Aug 24, 2026 •

edited

Loading