Repository navigation
SIMD (AVX2 and NEON) optimization of JPEG2000-RCT and MD5 - #491
Merged
Merged
Conversation
DPX RGB 10-bit method A (LE and BE) and RGB 16-bit (LE and BE) are transformed with AVX2 if the CPU supports it, scalar code is used for the remaining pixels. 512-bit vectors are not used, they lower the frequency of the core on some CPUs, slowing down the decoding by the other threads.
Wrappers may decode in 2 steps (CanSplit()): Decode() decodes a frame to a frame given by the caller, the decoder signals the end of the decoding with raw_frame_ready::Ready() (maybe from another thread, maybe not in order). Our FFV1 decoder uses it: frames are output (written or checked) in output threads while the next frames are decoded, in parallel if each frame is in its own file, else in order. The count of frames between the decoding and the end of the output is limited by Depth(). The parsing of decoded frames (conformance and coherency checks) is done in order without the decoded content, which is not needed if the file size is known. Matroska parsing waits for the end of the outputs when an element other than a block is parsed, when a track shares its output with another track (compound file), and at the end. Our FFV1 decoder always decodes in 2 steps, so its wrapper does not implement Process().
MD5_Update_Lanes() computes the MD5 of the parts of a frame in a SIMD lane (AVX2, or AVX-512VL with 256-bit vectors), in parallel with the frames given by other threads at the same time, in a single thread. Lanes are used only if at least 3 frames are computed together, else each frame is computed with MD5_Update() in the thread of its caller (measured: 1 lane is 2x slower than scalar code on AMD Zen 5, 2 lanes are not faster). The output threads accept to wait a limited time for other frames (lower CPU usage), the wait is cancelled when no more frames can be given until the end of pending outputs (MD5_Lanes_Flush()). Without SIMD support or with OpenSSL, MD5_Update() is used in the thread of the caller as before.
The 8 lanes are 2 vectors of 4 lanes with independent dependency chains, so the vector units are used while a step waits for the previous one. F and G are a bit select, H is a 3-way XOR if SHA3 instructions are available, the rotate is a shift then a shift with insert. Measured on Apple M1 with 8 frames hashed at the same time: 3.5x less CPU than scalar code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.