Skip to content

perf(ci): FLOW-010 cut 1, drop aprender-compute dev-dep on aprender-core (Refs #5803) #6338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
noahgift wants to merge 2 commits into from
Closed

perf(ci): FLOW-010 cut 1, drop aprender-compute dev-dep on aprender-core (Refs #5803) #6338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ea8148
chore(roadmap): entries for PMAT-5446, PMAT-5909 and PMAT-5803 so #62…
Oct 25, 2026
5d910b2
perf(ci): FLOW-010 cut 1, drop aprender-compute dev-dep on aprender-c…
Oct 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions ci/explicit-test-commands.d/298-aprender-compute-tuner-falsification-5803.cmd
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
cargo test -p aprender-compute --test tuner_falsification
16 changes: 5 additions & 11 deletions crates/aprender-compute/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,9 @@ anyhow = "1.0.100"
# Hardware capability detection (PMAT-447)
chrono = "0.4"
toml = "0.8"
# ML tuner integration (SHOWCASE-BRICK-001, Section 12) — aprender RandomForest is a
# DEV-dependency only: the SIMD foundation must not depend on the ML layer (layer
# inversion). The `ml-tuner` showcase lives in examples/ml_tuner_demo.rs + the tuner
# falsification tests, which consume aprender directly. Dev-dep path cycles are legal
# in cargo and dropped on publish, keeping the workspace a self-contained DAG (APR-MONO).
# ML tuner integration (SHOWCASE-BRICK-001, Section 12) — the SIMD foundation takes no
# dependency, not even a dev one, on the ML layer. The aprender RandomForest tests live in
# crates/aprender-core/tests/tuner_training_data_quality.rs (FLOW-010, #5803).
# NOTE: trueno-graph (=aprender-graph) removed (APR-MONO self-containment): aprender-graph
# depends on trueno, so compute→graph closed a cycle. The `execution-graph` feature's only
# trueno-graph use was the thin `to_csr()`/`graph_to_csr()` CsrGraph exporters; CSR export
Expand Down Expand Up @@ -106,9 +104,6 @@ trueno-cuda-edge = { workspace = true }
# Sparse and solver crates for contract tests
trueno-sparse = { workspace = true }
trueno-solve = { workspace = true }
# ML-tuner showcase (SHOWCASE-BRICK-001): aprender RandomForest, DEV-only to avoid the
# compute→core layer inversion. Path dev-dep cycle is legal + dropped on publish.
aprender = { path = "../aprender-core", package = "aprender-core", default-features = false }
ndarray = "0.17.2"
faer = "0.24" # For GEMM benchmark comparison (fastest pure-Rust BLAS)
matrixmultiply = "0.3" # Direct matrixmultiply crate benchmark (engine behind ndarray)
Expand Down Expand Up @@ -139,9 +134,8 @@ hardware-detect = ["dep:dirs"]
tui-monitor = ["presentar-core", "presentar-terminal", "crossterm", "trueno-gpu", "dep:tracing", "tracing-subscriber", "tracing-appender", "dep:dirs"]
# CUDA support for TUI (real hardware monitoring)
cuda = ["trueno-gpu/cuda"]
# ML-based kernel selection and throughput prediction (SHOWCASE-BRICK-001)
# Showcase: feed trueno tuner features into aprender RandomForest (example + tests only;
# aprender is a dev-dependency, so this feature activates no lib dependency).
# DEPRECATED no-op, kept so removing it is not a semver break for 0.71.x users (activates nothing;
# the aprender RandomForest tests moved to aprender-core, #5803).
ml-tuner = []
# NOTE: `execution-graph` feature removed (APR-MONO self-containment) — its only purpose
# was the trueno-graph CsrGraph exporters (see above). The exec-graph profiler itself
Expand Down
107 changes: 9 additions & 98 deletions crates/aprender-compute/examples/ml_tuner_demo.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@
//!
//! Run with default (heuristic models):
//! cargo run --example ml_tuner_demo
//!
//! Run with RandomForest models (requires ml-tuner feature):
//! cargo run --example ml_tuner_demo --features ml-tuner

use trueno::tuner::{BrickTuner, KernelClassifier, QuantType, ThroughputRegressor, TunerFeatures};

Expand Down Expand Up @@ -163,101 +160,15 @@ fn demo_full_tuner(features: &TunerFeatures) {
println!();
}

/// Section 6: RandomForest models (ml-tuner showcase, SHOWCASE-BRICK-001).
///
/// The SIMD `trueno` tuner produces `TunerFeatures` vectors; this section feeds them
/// directly into `aprender::RandomForestRegressor`/`RandomForestClassifier`. aprender is
/// a **dev-dependency** of `aprender-compute`, so the compute lib itself never depends on
/// the ML layer — the showcase lives entirely here at the example layer, above both.
/// Section 6: RandomForest models (SHOWCASE-BRICK-001). The aprender half of the showcase
/// is tested in `crates/aprender-core/tests/tuner_training_data_quality.rs`: compute takes
/// no dependency on the ML layer, not even a dev one (FLOW-010, #5803).
fn demo_random_forest(features: &TunerFeatures) {
#[cfg(feature = "ml-tuner")]
{
use aprender::primitives::{Matrix, Vector};
use aprender::tree::{RandomForestClassifier, RandomForestRegressor};

println!("6. RandomForest Models (ml-tuner showcase)");
println!(" --------------------------------------");

let dim = TunerFeatures::DIM;
let n = 100usize;

// Regressor: trueno features -> predicted throughput (tok/s).
let mut xr = Vec::with_capacity(n * dim);
let mut yr = Vec::with_capacity(n);
for i in 0..n {
let batch = 1 + (i % 8) as u32;
let f = TunerFeatures::builder()
.model_params_b(1.5)
.batch_size(batch)
.quant_type(QuantType::Q4K)
.gpu_mem_bw_gbs(1000.0)
.cuda_graphs(batch == 1)
.build();
xr.extend(f.to_vector());
yr.push(200.0 + (batch as f32) * 80.0 + (i as f32 * 0.5));
}
println!(" Generated {n} regression samples");
match Matrix::from_vec(n, dim, xr) {
Ok(x_mat) => {
let mut rf = RandomForestRegressor::new(100);
match rf.fit(&x_mat, &Vector::from_vec(yr)) {
Ok(()) => {
let fx = Matrix::from_vec(1, dim, features.to_vector().to_vec())
.expect("single-row feature matrix");
let pred = rf.predict(&fx);
println!(
" RF regressor: SUCCESS — M=4 prediction {:.1} tok/s",
pred.as_slice()[0]
);
}
Err(e) => println!(" RF regressor: FAILED - {e}"),
}
}
Err(e) => println!(" RF regressor matrix: FAILED - {e}"),
}

// Classifier: trueno features -> kernel-type label (3=Batched, 2=Vectorized).
let mut xc = Vec::with_capacity(n * dim);
let mut yc: Vec<usize> = Vec::with_capacity(n);
for i in 0..n {
let batch = 1 + (i % 8) as u32;
let f = TunerFeatures::builder()
.model_params_b(1.5)
.batch_size(batch)
.quant_type(QuantType::Q4K)
.build();
xc.extend(f.to_vector());
yc.push(if batch >= 4 { 3 } else { 2 });
}
match Matrix::from_vec(n, dim, xc) {
Ok(xc_mat) => {
let mut rf = RandomForestClassifier::new(50);
match rf.fit(&xc_mat, &yc) {
Ok(()) => {
let preds = rf.predict(&xc_mat);
let correct =
yc.iter().zip(preds.iter()).filter(|(a, b)| **a == **b).count();
println!(
" RF classifier: SUCCESS — train accuracy {:.1}%",
correct as f32 / n as f32 * 100.0
);
}
Err(e) => println!(" RF classifier: FAILED - {e}"),
}
}
Err(e) => println!(" RF classifier matrix: FAILED - {e}"),
}
println!();
}

#[cfg(not(feature = "ml-tuner"))]
{
let _ = features;
println!("6. RandomForest Models");
println!(" --------------------");
println!(" [Disabled - enable with: --features ml-tuner]");
println!();
}
let _ = features;
println!("6. RandomForest Models");
println!(" --------------------");
println!(" [See aprender-core tests/tuner_training_data_quality.rs]");
println!();
}

fn main() {
Expand All @@ -279,7 +190,7 @@ fn main() {
println!(" - TunerFeatures: 42-dimension vector for ML models");
println!(" - Throughput prediction with roofline clamping (v1.1.0)");
println!(" - Kernel selection: BatchedQ4K for M>=4, VectorizedQ4K otherwise");
println!(" - RandomForest available with --features ml-tuner");
println!(" - RandomForest showcase: aprender-core tests/tuner_training_data_quality.rs");
println!();
println!("Next steps:");
println!(" cargo run --example quickstart # Basic trueno usage");
Expand Down
179 changes: 3 additions & 176 deletions crates/aprender-compute/tests/tuner_falsification/training_data_quality.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,101 +1,11 @@
//! F041-F060: Training Data Quality (20 points)
//!
//! The `ml-tuner` showcase (SHOWCASE-BRICK-001) feeds trueno `TunerFeatures` into
//! `aprender::RandomForestRegressor`/`Classifier`. aprender is a dev-dependency of
//! aprender-compute (the SIMD foundation never depends on the ML layer), so these
//! falsification tests drive the aprender RandomForest pipeline directly, mirroring the
//! `<10 samples → InsufficientData` guard the old lib wrapper enforced.
//! The aprender RandomForest half (F041-F044, F046-F048) lives in
//! `crates/aprender-core/tests/tuner_training_data_quality.rs`: the SIMD foundation
//! takes no dependency, not even a dev one, on the ML layer (FLOW-010, #5803).

use trueno::tuner::{ThroughputRegressor, TunerFeatures};

#[allow(unused_imports)]
use trueno::tuner::{KernelClassifier, QuantType};

/// Train an aprender RandomForest regressor on trueno features, with the showcase's
/// `<10 samples` guard. Returns `Err` on insufficient data or matrix/fit failure.
#[cfg(feature = "ml-tuner")]
fn fit_throughput_rf(
rows: &[(TunerFeatures, f32)],
n_trees: usize,
) -> Result<aprender::tree::RandomForestRegressor, String> {
use aprender::primitives::{Matrix, Vector};
use aprender::tree::RandomForestRegressor;
if rows.len() < 10 {
return Err(format!("insufficient data: {} < 10", rows.len()));
}
let dim = TunerFeatures::DIM;
let n = rows.len();
let mut x = Vec::with_capacity(n * dim);
let mut y = Vec::with_capacity(n);
for (f, t) in rows {
x.extend(f.to_vector());
y.push(*t);
}
let x_mat = Matrix::from_vec(n, dim, x).map_err(|e| e.to_string())?;
let mut rf = RandomForestRegressor::new(n_trees);
rf.fit(&x_mat, &Vector::from_vec(y)).map_err(|e| e.to_string())?;
Ok(rf)
}

/// F041: Empty training data should error
#[cfg(feature = "ml-tuner")]
#[test]
fn f041_empty_training_errors() {
let empty_data: Vec<(TunerFeatures, f32)> = vec![];
let result = fit_throughput_rf(&empty_data, 10);
assert!(result.is_err(), "F041 FALSIFIED: empty training data should error");
}

/// F042: Single sample training should error gracefully (below the 10-sample floor)
#[cfg(feature = "ml-tuner")]
#[test]
fn f042_single_sample_graceful() {
let features = TunerFeatures::builder().model_params_b(1.5).build();
let data = vec![(features, 100.0)];
// Should error (insufficient), not panic
let _ = fit_throughput_rf(&data, 10);
}

/// F043: Training with NaN labels should not produce NaN predictions
#[cfg(feature = "ml-tuner")]
#[test]
fn f043_nan_labels_error() {
use aprender::primitives::Matrix;
let mut data: Vec<(TunerFeatures, f32)> = (0..10)
.map(|i| (TunerFeatures::builder().batch_size(1 + (i % 8) as u32).build(), 100.0))
.collect();
data[0].1 = f32::NAN;
if let Ok(rf) = fit_throughput_rf(&data, 10) {
let fx = Matrix::from_vec(1, TunerFeatures::DIM, data[1].0.to_vector().to_vec()).unwrap();
let pred = rf.predict(&fx);
assert!(
pred.as_slice()[0].is_finite() || pred.as_slice()[0].is_nan(),
"F043: prediction must be a well-defined float"
);
}
}

/// F044: Training with negative labels should be handled without panic
#[cfg(feature = "ml-tuner")]
#[test]
fn f044_negative_labels_handled() {
let data: Vec<(TunerFeatures, f32)> = (0..12)
.map(|i| {
let f = TunerFeatures::builder().batch_size(1 + (i % 8) as u32).build();
(f, if i % 2 == 0 { -100.0 } else { 100.0 })
})
.collect();
// Must not panic; result may be Ok or Err.
let _ = fit_throughput_rf(&data, 10);
}

// Stub tests for non-ml-tuner builds
#[cfg(not(feature = "ml-tuner"))]
#[test]
fn f041_f044_ml_tuner_disabled() {
// Pass - these tests require ml-tuner feature
}

/// F045: Heuristic model should work without training
#[test]
fn f045_heuristic_no_training() {
Expand All @@ -106,89 +16,6 @@ fn f045_heuristic_no_training() {
assert!(pred.predicted_tps > 0.0, "F045 FALSIFIED: heuristic prediction failed");
}

/// F046: Training improves over heuristic (or doesn't regress)
#[cfg(feature = "ml-tuner")]
#[test]
fn f046_training_improves() {
// Generate training data that matches heuristic pattern
let training_data: Vec<(TunerFeatures, f32)> = (0..50)
.map(|i| {
let batch = 1 + (i % 8) as u32;
let features = TunerFeatures::builder()
.model_params_b(1.5)
.batch_size(batch)
.gpu_mem_bw_gbs(1000.0)
.build();
// Throughput scales with batch size
let throughput = 100.0 + (batch as f32) * 50.0;
(features, throughput)
})
.collect();

let result = fit_throughput_rf(&training_data, 50);
assert!(result.is_ok(), "F046 FALSIFIED: training failed: {:?}", result.err());
}

#[cfg(not(feature = "ml-tuner"))]
#[test]
fn f046_ml_tuner_disabled() {
// Pass
}

/// F047: Large training set should not OOM
#[cfg(feature = "ml-tuner")]
#[test]
fn f047_large_training_no_oom() {
let training_data: Vec<(TunerFeatures, f32)> = (0..1000)
.map(|i| {
let features = TunerFeatures::builder()
.model_params_b((i % 10) as f32 * 0.5 + 0.5)
.batch_size((i % 8 + 1) as u32)
.build();
(features, 100.0 + (i as f32))
})
.collect();

let result = fit_throughput_rf(&training_data, 10);
assert!(result.is_ok(), "F047 FALSIFIED: large training failed");
}

#[cfg(not(feature = "ml-tuner"))]
#[test]
fn f047_ml_tuner_disabled() {
// Pass
}

/// F048: Classifier training should work (aprender RandomForestClassifier on trueno features)
#[cfg(feature = "ml-tuner")]
#[test]
fn f048_classifier_training() {
use aprender::primitives::Matrix;
use aprender::tree::RandomForestClassifier;

let dim = TunerFeatures::DIM;
let n = 50usize;
let mut x = Vec::with_capacity(n * dim);
let mut y: Vec<usize> = Vec::with_capacity(n);
for i in 0..n {
let batch = 1 + (i % 8) as u32;
let f = TunerFeatures::builder().model_params_b(1.5).batch_size(batch).build();
x.extend(f.to_vector());
// Label: BatchedQ4K (3) for M>=4, VectorizedQ4K (2) otherwise
y.push(if batch >= 4 { 3 } else { 2 });
}
let x_mat = Matrix::from_vec(n, dim, x).expect("feature matrix");
let mut classifier = RandomForestClassifier::new(10);
let result = classifier.fit(&x_mat, &y);
assert!(result.is_ok(), "F048 FALSIFIED: classifier training failed: {:?}", result.err());
}

#[cfg(not(feature = "ml-tuner"))]
#[test]
fn f048_ml_tuner_disabled() {
// Pass
}

/// F049: Training data variance check
#[test]
fn f049_training_data_variance() {
Expand Down
22 changes: 22 additions & 0 deletions docs/roadmaps/entries/PMAT-5446.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
- id: PMAT-5446
github_issue: 5446
item_type: task
title: 'ci: shrink the slow quick-tier tests (apr-cli test_llm_band, contracts-cli ont4 gate tests) at the root'
status: in_progress
priority: medium
assigned_to: null
created: 2026-10-25T07:50:00Z
updated: 2026-10-25T07:50:00Z
spec: null
acceptance_criteria:
- 'contracts-cli ont4_relations_gate, ont4b_shapes_gate and ont5_consistency_gate lint a planted fixture or copied files, not the full contracts tree, and each finishes well under 60 s (landed by #5893 via the EMR-001 integration PR #6309)'
- the pv-sat corpus() lint tests use a planted .pv fixture (same change)
- apr-cli commands::test_llm_band tests no longer take 60-240 s each (separate PR; not covered by #6309)
phases: []
subtasks: []
estimated_effort: null
labels:
- kind:code
notes: |-
Entry minted for the #5893 planted-.pv tests carried by #6309 (quorum handoff triage-quorum-tickets-6294-6309-0718Z). Issue #5446 names both slow families; #5893 is the contracts-cli half (gate tests lint a fixture, 11/11 per its body). The test_llm_band half has no PR yet, so the item stays in_progress.
epic: null
Loading
Loading