perf(audio): optimize cross-platform processing and complete native audio settings

openless-all/app/crates/micyou-dsp/UPSTREAM.md

@@ -5,8 +5,9 @@ Source: [MicYou-Dev/MicYou](https://github.com/MicYou-Dev/MicYou), commit
 
 `src/lib.rs` comes from `tauri-app/crates/micyou-audio/src/dsp.rs`.
 `../micyou-infer/src/{lib,ops}.rs` and the two `src/assets/*.mcy` blobs
-come from the corresponding upstream paths. The inference runtime and model
-blobs are imported without algorithm changes. Original copyright notices are
+come from the corresponding upstream paths. The model blobs are imported unchanged. The inference runtime retains the
+upstream algorithms, with baseline x86-64 SSE and ARM64 NEON row projections using separate
+multiply/add operations and the original accumulation order. Original copyright notices are
 retained in all source files. See `LICENSE` for MicYou's complete license and
 plugin exception; `LICENSE-PureVox.txt` and `LICENSE-AEC7.txt` are the upstream
 MIT model notices. The inference crate also carries MicYou's license.
@@ -24,6 +25,9 @@ OpenLess changes:
   PureVox wet/dry energy ratio. This prevents AGC from undoing suppression
   when a conservative amount of dry signal is mixed back in.
 - Add serde defaults for DSP settings.
+- Make the true FFT spectrum analyzer opt-in and throttle snapshots to 10 Hz.
+  Closing monitoring releases its FFT resources.
+- Use reusable host buffers and cached rational-phase resampling kernels.
 - Define the small AEC failure enum locally instead of depending on MicYou's
   audio host.
 
@@ -32,9 +36,17 @@ resampling, fixed 480-sample hops, model-delay compensation, end-of-stream
 flush and exact output-duration accounting in `openless-core/src/audio.rs`
 and `audio_dsp.rs`. The main microphone path does not install plugin hooks.
 
+Android capture requests 48 kHz directly, and all enhancement algorithms keep
+that processing rate. The original input filter is preserved even at 48 kHz,
+using cached coefficients; ASR output is downsampled to 16 kHz. CPAL retains a device-native fallback if
+48 kHz cannot be opened. Capture queues are not enlarged for playback or
+monitoring.
+
 AEC7 and its aligned-reference API are retained and tested here. OpenLess's
 microphone hosts do not yet provide synchronized speaker-reference capture,
-so the application does not expose or claim active acoustic echo cancellation.
+so the application does not expose AEC7 as an active microphone effect.
+Android may instead use its session-bound platform AcousticEchoCanceler when
+the device supports it; availability and activation are reported separately.
 The optional comb filter follows upstream's simple delay subtraction; it is
 not a learned room dereverberation model.
 
@@ -81,3 +93,68 @@ do not automatically trigger separate PR workflows.
 Most upstream implementation/model fixes can follow this route without manual
 copying. Upstream API/dependency/format changes that break the adapter or tests
 still require review and a deliberate patch/dependency update.
+
+## Shared performance changes (2026-10-22)
+
+All hosts that use `AudioProcessor` (including desktop Tauri and Linux egui)
+reuse mono, frame and resampled output scratch buffers. Rational resampling
+phases are precomputed once per stream, capped at 1024 phases; unusual clocks
+fall back to the original kernel calculation. The output duration and filter
+formula are unchanged, including the original filter at native 48 kHz.
+
+Contiguous matrix and GRU projection rows use separate SIMD multiply and add:
+SSE on baseline x86-64, NEON on ARM64 targets with NEON enabled. Other targets
+keep the scalar fallback. No additional inference runtime or worker pool is
+loaded. This does not change OS-specific capture APIs or require AVX support.
+
+The synthetic full-chain probe defaults to 48 kHz and checks exact output
+length. Run on the target device before drawing battery or latency conclusions:
+
+```sh
+cargo run --release --manifest-path openless-all/app/Cargo.toml -p openless-core --example audio_performance -- rnnoise 48000
+```
+
+## Before/after equivalence validation (2026-10-24)
+
+The frozen baseline is commit `f0744e10`. The same `openless-core` example
+`audio_equivalence` was built against that baseline and the optimized sources,
+using their locked dependencies, in separate release build directories.
+Each build exports 48 cases: 16/44.1/48 kHz, mono/stereo, four denoiser modes,
+and default/combined gain-EQ-comb-VAD settings. Inputs contain silence,
+harmonics, deterministic noise, high-frequency interference, an impulse,
+irregular chunk boundaries, and a partial final frame.
+
+On Linux x86-64, all 48 output PCM files were byte-identical to the baseline:
+maximum difference 0 LSB, with matching durations. The full hashes, errors and
+diagnostic processing times are committed in
+`../../scripts/fixtures/audio-equivalence-2026-10-24.json`. This establishes
+equivalence for those inputs and host, not a listening study or ASR accuracy.
+
+Export both builds to separate directories with `audio_equivalence OUTPUT_DIR`,
+then run `python3 scripts/compare-audio-outputs.py BEFORE AFTER --report REPORT`.
+The comparator fails on any PCM difference or duration/case-set mismatch;
+`python3 scripts/test-compare-audio-outputs.py` verifies a one-LSB mutation
+and a truncated output cannot pass. The probe sources must be identical in
+both builds. Original baseline sources/models must remain unchanged.
+
+Desktop Tauri and Linux egui use the shared processor directly. Harmony uses
+the external PCM recorder, which reuses conversion scratch per session.
+The iOS native recorder now optionally applies the same processor with
+session-frozen settings and a reused input buffer; the stop/drain barrier
+flushes its tail before finishing the archive. Enhancement remains off by
+default. Its native Swift settings page persists the same software DSP
+controls as egui and Tauri, with writes blocked while a capture is active.
+Native iOS conversion uses a separate
+Swift AVAudioConverter worker; its output scratch is now actor-owned and
+reused with exact output capacities and a two-entry bound.
+`ios/scripts/test-audio.sh` compares that worker byte-for-byte against
+the frozen pre-optimization Swift worker using its complete, fixed 100 ms
+input path, over up to 32 format/ownership/packetization combinations and
+end-of-stream flush (older macOS runtimes only exercise legacy ownership).
+The old variable-packet path itself lost 960 samples in the 44.1 kHz CI case.
+The new worker honors the converter's requested input frame count and drains
+available output before accepting another packet. Variable-packet output must
+match the complete fixed-packet baseline, not its truncated old counterpart;
+the duration assertion remains enabled. That Apple SDK test
+was added but cannot be executed on this Linux host. No claim of on-device
+equivalence, battery savings, or real-time capture latency is made here.

openless-all/app/crates/micyou-dsp/src/lib.rs

@@ -939,6 +939,9 @@ pub struct DspProcessor {
     agc_smoothed_gain: f32,
     // VAD fade state (0.0 = muted, 1.0 = full)
     vad_fade: f32,
+    #[cfg(feature = "noise-suppression")]
+    spectrum_analyzer: Option<SpectrumAnalyzer>,
+    spectrum_tick: usize,
     // Spectrum snapshots
     raw_spectrum: Vec<f32>,
     processed_spectrum: Vec<f32>,
@@ -981,6 +984,53 @@ pub fn parse_plugin_chain_node(node: &str) -> Option<&str> {
 
 const RNNOISE_FRAME_SIZE: usize = 480;
 
+#[cfg(feature = "noise-suppression")]
+struct SpectrumAnalyzer {
+    fft: Arc<dyn rustfft::Fft<f32>>,
+    buffer: Vec<Complex<f32>>,
+    scratch: Vec<Complex<f32>>,
+    window: Vec<f32>,
+}
+
+#[cfg(feature = "noise-suppression")]
+impl SpectrumAnalyzer {
+    fn new() -> Self {
+        let fft = rustfft::FftPlanner::new().plan_fft_forward(512);
+        let scratch = vec![Complex::default(); fft.get_inplace_scratch_len()];
+        Self {
+            fft,
+            scratch,
+            buffer: vec![Complex::default(); 512],
+            window: (0..512)
+                .map(|i| 0.5 - 0.5 * (std::f32::consts::TAU * i as f32 / 511.0).cos())
+                .collect(),
+        }
+    }
+
+    fn analyze(&mut self, data: &[f32], target: &mut [f32]) {
+        self.buffer.fill(Complex::default());
+        let count = data.len().min(512);
+        for (i, sample) in data[data.len() - count..].iter().enumerate() {
+            self.buffer[i].re = sample * self.window[i];
+        }
+        self.fft
+            .process_with_scratch(&mut self.buffer, &mut self.scratch);
+        for (band, value) in target.iter_mut().enumerate() {
+            let low = 20.0 * 1200.0_f32.powf(band as f32 / 64.0);
+            let high = 20.0 * 1200.0_f32.powf((band + 1) as f32 / 64.0);
+            let start = ((low * 512.0 / 48_000.0) as usize).min(256);
+            let end = ((high * 512.0 / 48_000.0).ceil() as usize).clamp(start + 1, 257);
+            let magnitude = self.buffer[start..end]
+                .iter()
+                .map(|bin| bin.norm())
+                .fold(0.0_f32, f32::max)
+                / 128.0;
+            let normalized = ((20.0 * magnitude.max(1e-8).log10() + 60.0) / 60.0).clamp(0.0, 1.0);
+            *value = normalized.max(*value * 0.8);
+        }
+    }
+}
+
 impl DspProcessor {
     pub fn new(settings: Arc<RwLock<AudioDspSettings>>, _model_dir: Option<PathBuf>) -> Self {
         Self {
@@ -1020,6 +1070,9 @@ impl DspProcessor {
             agc_envelope: 0.0,
             agc_smoothed_gain: 1.0,
             vad_fade: 1.0,
+            #[cfg(feature = "noise-suppression")]
+            spectrum_analyzer: None,
+            spectrum_tick: 0,
             raw_spectrum: vec![0.0; 64],
             processed_spectrum: vec![0.0; 64],
             accum_buffer: Vec::new(),
@@ -1045,6 +1098,7 @@ impl DspProcessor {
         }
 
         let raw_rms = compute_rms(data);
+        self.spectrum_tick = (self.spectrum_tick + 1) % 10;
         self.compute_spectrum(data, true);
 
         // Frame accumulation: align to 480*channels samples before processing,
@@ -1670,59 +1724,34 @@ impl DspProcessor {
 
     // ── Spectrum ─────────────────────────────────────────────────────────────
 
-    fn compute_spectrum(&mut self, data: &[f32], is_raw: bool) {
-        let bands = 64;
-        let target = if is_raw {
-            &mut self.raw_spectrum
-        } else {
-            &mut self.processed_spectrum
-        };
-        if target.len() != bands {
-            target.resize(bands, 0.0);
+    pub fn set_spectrum_enabled(&mut self, enabled: bool) {
+        #[cfg(feature = "noise-suppression")]
+        if enabled && self.spectrum_analyzer.is_none() {
+            self.spectrum_analyzer = Some(SpectrumAnalyzer::new());
+        } else if !enabled {
+            self.spectrum_analyzer = None;
+            self.raw_spectrum.fill(0.0);
+            self.processed_spectrum.fill(0.0);
         }
-        if data.is_empty() {
-            for v in target.iter_mut() {
-                *v = 0.0;
-            }
+        #[cfg(not(feature = "noise-suppression"))]
+        let _ = enabled;
+    }
+
+    fn compute_spectrum(&mut self, data: &[f32], is_raw: bool) {
+        if self.spectrum_tick != 1 {
             return;
         }
-
-        const BANDS: usize = 64;
-        static BAND_LIMITS: std::sync::OnceLock<[f32; BANDS + 1]> = std::sync::OnceLock::new();
-        let limits = BAND_LIMITS.get_or_init(|| {
-            let mut array = [0.0; BANDS + 1];
-            for (i, limit) in array.iter_mut().enumerate() {
-                *limit = (i as f32 / BANDS as f32).powf(1.5);
-            }
-            array
-        });
-
-        for (band_idx, band_val) in target.iter_mut().enumerate() {
-            let start = limits[band_idx] * data.len() as f32;
-            let end = limits[band_idx + 1] * data.len() as f32;
-            let start = start as usize;
-            let end = (end as usize).min(data.len());
-            if start >= end {
-                *band_val *= 0.85;
-                continue;
-            }
-            let mut sum = 0.0_f32;
-            for sample in &data[start..end] {
-                sum += sample * sample;
-            }
-            let rms = (sum / (end - start) as f32).sqrt();
-            let db = if rms > 1e-10 {
-                20.0 * rms.log10()
+        #[cfg(feature = "noise-suppression")]
+        if let Some(analyzer) = &mut self.spectrum_analyzer {
+            let target = if is_raw {
+                &mut self.raw_spectrum
             } else {
-                -100.0
+                &mut self.processed_spectrum
             };
-            let normalized = ((db + 60.0) / 60.0).clamp(0.0, 1.0);
-            if normalized > *band_val {
-                *band_val = normalized;
-            } else {
-                *band_val = *band_val * 0.85 + normalized * 0.15;
-            }
+            analyzer.analyze(data, target);
         }
+        #[cfg(not(feature = "noise-suppression"))]
+        let _ = (data, is_raw);
     }
 }
 
@@ -2095,11 +2124,13 @@ mod native_tests {
     #[test]
     fn dsp_processor_chain_smoke() {
         // Full chain: AEC → PureVox NS through the public DspProcessor API.
-        let mut settings = AudioDspSettings::default();
-        settings.ns_enabled = true;
-        settings.ns_type = "PureVox".to_string();
-        settings.ns_intensity = 100.0;
-        settings.aec_enabled = true;
+        let settings = AudioDspSettings {
+            ns_enabled: true,
+            ns_type: "PureVox".to_string(),
+            ns_intensity: 100.0,
+            aec_enabled: true,
+            ..Default::default()
+        };
         let settings = Arc::new(RwLock::new(settings));
         let mut dsp = DspProcessor::new(settings, None);
 
@@ -2126,3 +2157,42 @@ mod native_tests {
         );
     }
 }
+
+#[cfg(all(test, feature = "noise-suppression"))]
+mod spectrum_tests {
+    use super::*;
+
+    #[test]
+    fn spectrum_is_opt_in_and_tracks_frequency_without_changing_audio() {
+        let settings = Arc::new(RwLock::new(AudioDspSettings {
+            gain: 6.0,
+            ..Default::default()
+        }));
+        let mut plain = DspProcessor::new(Arc::clone(&settings), None);
+        let mut monitored = DspProcessor::new(settings, None);
+        assert!(plain.spectrum_analyzer.is_none());
+        monitored.set_spectrum_enabled(true);
+        let tone: Vec<f32> = (0..480)
+            .map(|i| (std::f32::consts::TAU * 1000.0 * i as f32 / 48_000.0).sin() * 0.05)
+            .collect();
+        for _ in 0..12 {
+            let mut a = tone.clone();
+            let mut b = tone.clone();
+            plain.process(&mut a, 1, 0.0);
+            monitored.process(&mut b, 1, 0.0);
+            assert_eq!(a, b);
+        }
+        let (raw, processed) = monitored.get_spectrums();
+        let peak = raw
+            .iter()
+            .enumerate()
+            .max_by(|a, b| a.1.total_cmp(b.1))
+            .unwrap()
+            .0;
+        assert!((30..=40).contains(&peak), "1 kHz peak at {peak}");
+        assert!(processed[peak] > raw[peak]);
+        monitored.set_spectrum_enabled(false);
+        assert!(monitored.spectrum_analyzer.is_none());
+        assert!(monitored.get_spectrums().0.iter().all(|v| *v == 0.0));
+    }
+}