Where ancient silicon refuses to yield, and old-guard CPUs withstand the passage of time, Nitro NG is forged.
Nitro NG is a highly parallel, hardware-aware pre-compression utility engineered natively in C++20 for Linux (POSIX), Windows x64, and Windows XP x86. Its sacred purpose is to cut through opaque game files, locate buried zlib/deflate compressed layers, and inflate them into their raw, expanded state with bit-exact mathematical precision.
By restructuring the layout of your data into expanded streams, Nitro NG conditions game assets so secondary solid compression heavyweights (like LZMA2, ZSTD, or Oodle) can squeeze them down to their absolute physical limits.
Version 1.09 represents the culmination of a complete architectural rewrite. It takes the groundbreaking streaming model introduced in v1.08 and perfects it with polished cross-platform build scripts, fixed console UX, validated real-world benchmarks, and rock-solid memory determinism.
Scanner threads verify streams on-the-fly without storing decompressed data. Memory arenas adapt based on thread counts, utilizing immediate heap reclamation in Low-Thread Mode (≤ 2 threads) and retaining allocation pools in High-Thread Mode (> 2 threads).
Input files are divided into segments with a sliding window for backpressure control, scaling output write chunks dynamically from 2 MB for HDDs up to 16 MB for NVMe drives.
Why guess when the file tells you? The zlib header's FLEVEL byte provides a strong hint about the original compression level. The encoder tries the most likely level first, remembering the last successful level per class, slashing failed verification attempts by 30-40%.
Real-time statistics are sampled via relaxed-order atomics on a dedicated ticker thread (~10 Hz) with zero mutex contention. An Exponential Moving Average (EMA) smooths short bursts, preventing violent ETA oscillations.
Replaced the legacy sequential decoder with an active, task-based worker pool. Workers fetch tasks dynamically via atomic fetch-add, accelerating file restoration by 2x–3x on multi-core hardware while keeping memory strictly controlled via a bounded byte budget.
Eliminated MapViewOfFile entirely. We now use direct overlapped I/O (portable_pread), preventing the Windows memory manager from aggressively caching massive .pre archives. Full cross-compilation support for 32-bit Windows XP (Subsystem 5.01) with Large Address Aware linking.
- Fixed Console UX: Resolved the visual bug where the "Levels" summary would only display the last level. Now correctly shows
Levels: 6 (x693), 7 (x1). - Optimized Build System: Scripts now compile a local, optimized zlib 1.3.2 and link it statically. Replaced slow filesystem scans with instant compiler queries (
$CXX -print-file-name) for toolchain detection. - Validated Metrics: All performance claims are now backed by real-world, bit-exact tested benchmarks..
Summary of chronological cross-platform performance data using a dense stream distribution workload (352.78 MB input, 694 zlib streams, setup_virtuaverse_1.38_(92629).exe):
Used DualCore with Wine 6.0.3
| Engine Version | Environment / Runtime | Peak Process RAM | Encode Time | Pipeline Architecture & Control |
|---|---|---|---|---|
| Nitro NG v1.03 (precomp_zlib) | Windows x64 (Wine) | 1183 MB | 2m 18.754s | First Release / Unbounded Heap Allocation |
| Nitro NG v1.05 | Windows x64 (Wine) | 1647 MB | 2m 33.438s | Concurrent Multi-thread / Unthrottled Allocation (Bare-Metal Snapshot Impact) |
| Nitro NG v1.09 | Windows x64 (Wine) | 89 MB | 1m 36.744s | Adaptive Backpressure & Aggressive Slot Release |
| Nitro NG v1.09 | Linux Native (Raw) | 104 MB | 1m 26.949s | Linear Streaming / Direct Page Cache Write |
| Xtool 0.8.9 (Razor12911) | Windows x64 (Wine) | 489 MB | 2m 18.000s | High-Thread Buffer Aggregation |
| Precomp 0.48* (Christian S.) | Windows x64 (Wine) | 80 MB | 7m 16.821s | Monolithic Sequential / Intense Flags |
*Precomp 0.48 Flags: -cn -intense -t-pzgnfjsmb3 | Recompressed streams: 1859/1885 (Nitro & Xtool match ~696) **Note: Precomp 0.48 uses optimized compilation flags for CMake.
| Tool & Configuration | Environment / Runtime | Peak Process RAM | Decode Time | Multi-Threading & I/O Profile |
|---|---|---|---|---|
| Precomp 0.48 (Classic) | Windows x64 (Wine) | 33 MB | 2m 59.043s | Hardcoded Monolithic Sequential Execution (3x Slower) |
| Xtool 0.8.9 (Modern) | Windows x64 (Wine) | 353 MB | 2m 49.000s | High-Thread Buffer Aggregation (HDD Thrashing) |
| Nitro NG v1.09 | Windows x64 (Wine) | 45 MB | 2m 31.654s | Bounded Queue with 2 MB Core-Aware Disk Flushing |
| Nitro NG v1.09 (Real HDD) | Linux Native (Raw) | 51 MB | 2m 00.947s | Pure Uncached Mechanical Plato I/O (Dropped Caches) |
| Nitro NG v1.09 (Cached) | Linux Native (Raw) | 46 MB | 0m 46.000s | Native Linux Kernel Page-Cache Streaming Mode |
🛡️ A Note of Respect to the Pioneers: Absolute respect to classic tools like Precomp by Christian Schneider and the incredible work of Razor12911 with Xtool; they were the absolute pioneers of this technology. This benchmark does not seek to compete in raw horsepower on modern workstations, but strictly to measure the efficiency of memory management within environments with limited hardware resources.
- The v1.05 Multi-Thread Flaw: Parallel execution without backpressure led to a 1647 MB memory spike due to async buffer dumping.
- The 1.09 Resolution: Hardware-aware profiling reduces encoding peaks to 89 MB on minimal threads while maintaining faster speeds.
- Elastic Deployment: Automatically scales up to 16 MB buffers on high-thread workstations with NVMe arrays.
The following metrics track the chronological maximum memory spikes (monitored and verified under strict heap analysis) using the exact same dense stream distribution workload to measure the impact of the streaming verification model.
Environment: Linux x86_64 Host / Tested under Wine 6.0.3 Runtime (Strict Emulated Subsystem Architecture).
| Engine Version | Peak Process RAM | Encode Time | Pipeline Architecture / Memory Control | Operational Status |
|---|---|---|---|---|
| Nitro NG v1.03 | 1183 MB (1,212,328 KB) |
2m 18s | Legacy Multithread / Unbounded Heap Allocation | Functional (Baseline) |
| Nitro NG v1.05 | 1647 MB (1,686,756 KB) |
2m 33s | Concurrent Streams / No Thread Throttling | Unthrottled Spike |
| Nitro NG v1.09 | 89 MB (91,136 KB) |
1m 36s | Streaming Verification & Segmented Backpressure | Production (Deterministic) |
-
Memory Footprint Optimization: Version 1.09 achieves a 15x reduction in peak RAM usage compared to the v1.05 pipeline by completely eliminating the memory-mapped I/O layer, sharded caches, and large per-thread pre-allocations.
-
Throughput Efficiency: Encoding speed dropped from 2m 33s to just 1m 28s (a 1.7x performance multiplier). This is achieved via a sliding window mechanism (W = num_threads × 2 + 2) that applies backpressure to worker threads, preventing pending queue growth.
-
On-the-Fly Verification: Instead of storing intermediate decompressed data in the heap, the streaming model inflates and immediately re-deflates 64 KB chunks, validating integrity directly via
memcmpagainst the original stream. Memory scales strictly with the segment window, not with input file size or stream density.
Made for the lulz, proven by data.
Nitro NG delivers bit-exact restoration while maintaining a tightly bounded memory footprint. The bounded window and task-based decoder ensure that RAM usage scales with the segment window size, not with the input file size.
# Encode (Pre-compress)
nitro_ng e <input.bin> <output.pre> [-j threads]
# Decode (Restore)
nitro_ng d <input.pre> <restored.bin> [-j threads]Options:
-j N : Number of worker threads. (Default: hardware concurrency)
Examples:
nitro_ng e input.bin output.pre -j 4
nitro_ng d output.pre restored.bin -j 4Nitro NG uses custom build scripts that automatically compile a local, zlib 1.3.2 and link it statically. No external runtime dependencies are required.
./compile_linux.sh./compile_windows.sh./compile_windows_xp.sh(Requires gcc-mingw-w64-i686-posix and g++-mingw-w64-i686-posix on Debian/Ubuntu)
- Integrity: 100% verified. Bit-for-bit restoration matches the original input (MD5 confirmed).
- RAM Footprint: Bounded and deterministic. ~112 MB peak on encode, ~63.9 MB on decode for a 352 MB input. Scales with the segment window size, not with the input file size.
- UX Stability: Clean, single-line live progress with hidden console cursor and artifact-free space padding. Works flawlessly on narrow terminals and Windows/Wine environments.
Nitro NG is licensed under the GNU General Public License v3.0 (GPLv3).
- Fully Auditable: You can inspect the code to verify there is no telemetry, no black-box behavior, and no hidden compromises.
- Community Driven: Want to add support for Oodle, LZ4, or Zstandard? You can fork it, modify it, and improve it without asking for permission.
"The journey from v1.05 to v1.09 represents a complete transformation from a prototype with memory limits to a production-ready tool. v1.08 gave us the streaming architecture; v1.09 made it bulletproof, auditable, and ready for the real world."
BIT-EXACT OR DEATH. THE NORTH REMEMBERS EVERY BYTE.
