From 41e61e57a0fd6516354212d7d55a13074f01d0a0 Mon Sep 17 00:00:00 2001 From: NaiveCoding <71413722+ReviveCoding@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:30:40 -0400 Subject: [PATCH 1/4] Preserve technical decision record without interview framing --- docs/RESEARCH_DECISIONS.md | 50 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 docs/RESEARCH_DECISIONS.md diff --git a/docs/RESEARCH_DECISIONS.md b/docs/RESEARCH_DECISIONS.md new file mode 100644 index 0000000..98cecf0 --- /dev/null +++ b/docs/RESEARCH_DECISIONS.md @@ -0,0 +1,50 @@ +# Research Decisions and Limitations + +## Maven Track A: frozen evaluation + +The complete official Maven source contains 8,800 opportunities and passed all five frozen hashes. A1 uses a 60-day horizon with three 60-day purge gaps (FIT 1,670; VALIDATION 434; POLICY 273; LOCKED 403). PREASSIGN excludes assignment fields and historical aggregates use only outcomes resolved strictly before engagement. Model choice used VALIDATION; calibration and bounded AP-EV selection used POLICY. + +The frozen AP-EV used discrete timing, conditional Platt win probability, XGBoost conditional value, `H=60`, and `rho=0.005`. On one-shot LOCKED it captured 24.2% of value versus 29.2% for the business heuristic; difference −5.0 points, 95% account-cluster CI [−14.5, 8.1]. Win capture was 12.8% versus 12.2%. G5 and G7 failed and G9 was not evaluable because there were no cold accounts. LambdaMART captured 23.2%; POSTASSIGN XGBoost captured 7.9% and has no causal interpretation. Decision: `RETAIN_BASELINE`. Any redesign requires a new protocol and new protected outcomes. + +## Business framing and estimands + +AccountPulse asks who to prioritize under capacity, not who is caused to convert by outreach. Predictive outcome/timing/value, causal treatment effect, and sequential allocation remain separate. ValueCapture@10% is primary; WinCapture@10% is co-primary. + +## Data and point-in-time design + +Maven is fictitious public CRM, not production evidence. Olist permits only first-contact fields; seller linkage is outcome-only. UCI histories, item text, and graph edges are truncated strictly before each cutoff. R2 adds 90-day gaps so FIT labels mature before VALIDATION scoring and VALIDATION labels mature before TEST. + +## Labels, leakage, and survival + +Fixed horizons distinguish mature positives, mature negatives, and censored rows. Track-A Lost must be a competing terminal event, never ordinary censoring. Leakage variants L1-L3 exist only as ineligible diagnostics. Cox on UCI failed to converge/usefully rank and was retained as a negative result. + +## Value, ranking, calibration, and capacity + +Olist XGBoost ValueCapture@10% was 17.4%; sparse AP-EV was 10.4%. Isotonic was selected on validation, but no post-test retuning followed. On UCI, RFM captured 57.3%. Adaptive fusion achieved AUC 0.869, Brier 0.154, and slope 0.913; fixed fusion captured 57.5%, but its incremental CI crossed zero. + +## Representation learning and ablations + +Transformer improved discrimination over tabular models but lost on the primary capacity endpoint. GraphSAGE nearly matched RFM; text was weak. Removing tabular caused the largest adaptive-fusion capture drop. Removing graph or text did not materially hurt adaptive performance, so their incremental contribution is unsupported. + +## Uncertainty, robustness, and cold entities + +Inference resamples stored predictions by customer/lead; Olist also uses temporal-block sensitivity. UCI cold/unseen RFM capture was 25.7% versus 60.7% for known customers. The selective 0.8 P(top10) policy covered 9.7% and remained a challenger. + +## GPU and MLOps + +One GPU lock serializes heavy jobs. Torch FP16 and XGBoost CUDA were requalified after uv sync. Track-A CatBoost/XGBoost/AFT/LambdaMART used CUDA. MLflow records separate Track-A development and LOCKED runs. Batch scoring supports historical UCI and frozen Maven replay only; shadow replay is not online deployment. + +## Causal bridge + +Criteo v2.1 came from Criteo's verified Hugging Face organization and passed the exact source +hash. With no business-time field, D2 uses a stable 60/20/20 hash split rather than inventing +chronology. Visit is primary and conversion secondary. Treatment is excluded from response +and T-learner features; the S-learner uses treatment only as the randomized intervention; +post-treatment exposure is excluded. A train-only propensity model and outcome nuisances feed +held-out AIPW evaluation. At 10% reach, the S-learner and response lists overlap 64.2%; uplift +adds 0.000933 visits per eligible user, CI [0.000633, 0.001255]. The advantage is supported at +5%/10%, not 20%/30%. This is causal evidence only within the released advertising benchmark. + +## Negative results and next experiments + +Track-A retained the business heuristic; Track-C retained RFM. AP-EV, Cox, text, Transformer, and adaptive fusion did not win their primary decision gates. Any Track-A follow-up requires a new protocol and genuinely new protected outcomes. Track D is complete and remains a separate causal leaderboard. From 18adb9e788e7be86f814ae9f2796122676b04d9d Mon Sep 17 00:00:00 2001 From: NaiveCoding <71413722+ReviveCoding@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:30:42 -0400 Subject: [PATCH 2/4] Replace interview notes with research decisions --- INTERVIEW_NOTES.md | 50 ---------------------------------------------- 1 file changed, 50 deletions(-) delete mode 100644 INTERVIEW_NOTES.md diff --git a/INTERVIEW_NOTES.md b/INTERVIEW_NOTES.md deleted file mode 100644 index 74a588f..0000000 --- a/INTERVIEW_NOTES.md +++ /dev/null @@ -1,50 +0,0 @@ -# Interview Notes - -## Track-A A1 update - -The complete official Maven source contains 8,800 opportunities and passed all five frozen hashes. A1 uses a 60-day horizon with three 60-day purge gaps (FIT 1,670; VALIDATION 434; POLICY 273; LOCKED 403). PREASSIGN excludes assignment fields and historical aggregates use only outcomes resolved strictly before engagement. Model choice used VALIDATION; calibration and bounded AP-EV selection used POLICY. - -The frozen AP-EV used discrete timing, conditional Platt win probability, XGBoost conditional value, `H=60`, and `rho=0.005`. On one-shot LOCKED it captured 24.2% of value versus 29.2% for the business heuristic; difference −5.0 points, 95% account-cluster CI [−14.5, 8.1]. Win capture was 12.8% versus 12.2%. G5 and G7 failed and G9 was not evaluable because there were no cold accounts. LambdaMART captured 23.2%; POSTASSIGN XGBoost captured 7.9% and has no causal interpretation. Decision: `RETAIN_BASELINE`. Any redesign requires a new protocol and new protected outcomes. - -## Business framing and estimands - -AccountPulse asks who to prioritize under capacity, not who is caused to convert by outreach. Predictive outcome/timing/value, causal treatment effect, and sequential allocation remain separate. ValueCapture@10% is primary; WinCapture@10% is co-primary. - -## Data and point-in-time design - -Maven is fictitious public CRM, not production evidence. Olist permits only first-contact fields; seller linkage is outcome-only. UCI histories, item text, and graph edges are truncated strictly before each cutoff. R2 adds 90-day gaps so FIT labels mature before VALIDATION scoring and VALIDATION labels mature before TEST. - -## Labels, leakage, and survival - -Fixed horizons distinguish mature positives, mature negatives, and censored rows. Track-A Lost must be a competing terminal event, never ordinary censoring. Leakage variants L1-L3 exist only as ineligible diagnostics. Cox on UCI failed to converge/usefully rank and was retained as a negative result. - -## Value, ranking, calibration, and capacity - -Olist XGBoost ValueCapture@10% was 17.4%; sparse AP-EV was 10.4%. Isotonic was selected on validation, but no post-test retuning followed. On UCI, RFM captured 57.3%. Adaptive fusion achieved AUC 0.869, Brier 0.154, and slope 0.913; fixed fusion captured 57.5%, but its incremental CI crossed zero. - -## Representation learning and ablations - -Transformer improved discrimination over tabular models but lost on the primary capacity endpoint. GraphSAGE nearly matched RFM; text was weak. Removing tabular caused the largest adaptive-fusion capture drop. Removing graph or text did not materially hurt adaptive performance, so their incremental contribution is unsupported. - -## Uncertainty, robustness, and cold entities - -Inference resamples stored predictions by customer/lead; Olist also uses temporal-block sensitivity. UCI cold/unseen RFM capture was 25.7% versus 60.7% for known customers. The selective 0.8 P(top10) policy covered 9.7% and remained a challenger. - -## GPU and MLOps - -One GPU lock serializes heavy jobs. Torch FP16 and XGBoost CUDA were requalified after uv sync. Track-A CatBoost/XGBoost/AFT/LambdaMART used CUDA. MLflow records separate Track-A development and LOCKED runs. Batch scoring supports historical UCI and frozen Maven replay only; shadow replay is not online deployment. - -## Causal bridge - -Criteo v2.1 came from Criteo's verified Hugging Face organization and passed the exact source -hash. With no business-time field, D2 uses a stable 60/20/20 hash split rather than inventing -chronology. Visit is primary and conversion secondary. Treatment is excluded from response -and T-learner features; the S-learner uses treatment only as the randomized intervention; -post-treatment exposure is excluded. A train-only propensity model and outcome nuisances feed -held-out AIPW evaluation. At 10% reach, the S-learner and response lists overlap 64.2%; uplift -adds 0.000933 visits per eligible user, CI [0.000633, 0.001255]. The advantage is supported at -5%/10%, not 20%/30%. This is causal evidence only within the released advertising benchmark. - -## Negative results and next experiments - -Track-A retained the business heuristic; Track-C retained RFM. AP-EV, Cox, text, Transformer, and adaptive fusion did not win their primary decision gates. Any Track-A follow-up requires a new protocol and genuinely new protected outcomes. Track D is complete and remains a separate causal leaderboard. From a74a864a372ffe1a194e3215fea5b3965fd480e2 Mon Sep 17 00:00:00 2001 From: NaiveCoding <71413722+ReviveCoding@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:30:53 -0400 Subject: [PATCH 3/4] Remove portfolio-only resume evidence from technical repository --- RESUME_EVIDENCE.md | 11 ----------- 1 file changed, 11 deletions(-) delete mode 100644 RESUME_EVIDENCE.md diff --git a/RESUME_EVIDENCE.md b/RESUME_EVIDENCE.md deleted file mode 100644 index fad182e..0000000 --- a/RESUME_EVIDENCE.md +++ /dev/null @@ -1,11 +0,0 @@ -# Resume Evidence - -Use only the bullets below, preserving their benchmark boundaries. - -- Built a resume-safe predictive decision-science pipeline spanning 8,800 fictitious Maven CRM opportunities, 8,000 Olist leads, and 824K customer-identified UCI transaction lines, with point-in-time contracts, maturity embargoes, DuckDB/Parquet evidence, MLflow tracking, and fail-closed CUDA execution. -- Executed a preregistered, one-shot Maven LOCKED evaluation after freezing source/split/model/code hashes; retained the business heuristic when AP-EV captured 24.2% versus 29.2% ValueCapture@10% (difference −5.0 points, 95% CI [−14.5, 8.1]). -- Benchmarked RFM, Logistic/XGBoost, Cox, a three-seed BF16 Transformer, PIT GraphSAGE, cached MiniLM text embeddings, and multimodal fusion on an RTX 4090 Laptop GPU; retained RFM because fusion's +0.30-point ValueCapture@10% estimate had a 95% CI crossing zero. -- Detected and corrected a temporal label-overlap defect through adversarial review, issued a new protocol identity, invalidated prior external-track results, and reran the full model ladder without opening protected CRM outcomes. -- Demonstrated a negative external result on sparse Olist data (AP-EV 10.4% versus CUDA XGBoost 17.4% ValueCapture@10%) and separately showed on verified randomized Criteo evidence that response and uplift top-decile lists overlapped only 64.2%, without claiming cross-domain transportability. - -Do not claim production deployment, enterprise CRM lift, AP-EV improvement, cold-account validation, or causal outreach impact outside the released Criteo benchmark. From a987112feb277677d68785ef58ae24bd14909cca Mon Sep 17 00:00:00 2001 From: NaiveCoding <71413722+ReviveCoding@users.noreply.github.com> Date: Wed, 14 Oct 2026 06:30:56 -0400 Subject: [PATCH 4/4] Link decisions and limitations instead of career material --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 6109ed5..987e1f3 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,8 @@ accountpulse score --as-of 2017-11-01 --capacity 0.10 --track maven-locked-repla The sole local project environment is `$HOME/.local/share/accountpulse/project-venv`; GPU stages fail closed without one visible CUDA GPU. See `BLOCKERS.md`, -`SOURCE_MANIFEST.yaml`, `SPLIT_MANIFEST.yaml`, and the reports under `reports/`. +`SOURCE_MANIFEST.yaml`, `SPLIT_MANIFEST.yaml`, the reports under `reports/`, +and [research decisions and limitations](docs/RESEARCH_DECISIONS.md). The frozen scientific checkpoint is `accountpulse-v1.0-complete`. Subsequent slicing, calibration and drift analysis, GPU TreeSHAP, grouped permutation, PDP/ICE, and AP-EV component diagnostics are preserved separately on branch