Skip to content

Fix CausalForestDML ATE and ATT standard errors for multiple outputs - #1067

Open
kbattocchi wants to merge 1 commit into
mainfrom
kebatt/causal-forest-standard-errors
Open

kbattocchi wants to merge 1 commit into
mainfrom
kebatt/causal-forest-standard-errors

Conversation

@kbattocchi

@kbattocchi kbattocchi commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

Fixes #1066.

Fix two errors in CausalForestDML's doubly robust training-data ATE/ATT inference:

  • Count non-missing predictions along the sample axis (axis=0), separately for each outcome-treatment pair and after ATT masking. Previously, pooling counts across outputs or treatment contrasts understated standard errors.
  • Preserve outcome-treatment ordering when constructing the doubly robust correction. The previous cross-product/reshape combination mixed the axes when both dimensions exceeded one, affecting point estimates as well as standard errors. Use an explicit einsum with sample, outcome, and treatment axes.

Keep standard-error docstrings focused on the public API. The separately identified multi-output model-selection scoring issue remains out of scope.

Regression coverage

  • Deterministic aggregation checks cover vector, single-column, and multi-output outcomes; binary and three-arm treatments; missing rows and unequal column missingness; and masked subsets.
  • Fitted-estimator checks cover ATE/ATT estimates, standard errors, confidence intervals, z-statistics, and p-values for every treatment group. They assert finite predictions and use ordinary means, standard deviations, and sample counts, with a comment explaining why the data and estimator seeds must be retained.
  • Duplicate-outcome invariance checks compare public ATE/ATT point estimates, standard errors, and confidence intervals before and after duplicating an outcome, for two, three, and four treatment arms. They do not recalculate a reference standard-error formula.

The invariance test exposed the correction-order error. The existing known-ground-truth test uses duplicate outcomes, identical true effects for both non-control treatments, and absolute tolerances that allowed this error to escape; recomputing inference from the estimator's internal corrected predictions also cannot detect incorrect corrections upstream.

Validation

  • All 45 focused regression cases plus the existing causal-forest ATE inference test passed (46 total).
  • Independently restoring the original missing-axis bug in memory makes all three invariance cases fail.
  • Independently restoring the original correction ordering in memory makes the three- and four-arm invariance cases fail; the binary-treatment control still passes.
  • Ruff passed for both changed Python files; git diff --check passed.

Local validation uses the repository's last-known-good Cython 3.2.4 build; the original Cython 3.3.0 native-code crash was addressed separately.

@kbattocchi
kbattocchi force-pushed the kebatt/causal-forest-standard-errors branch from a3c0ba5 to 983c0d7 Compare September 28, 2026 19:39
@kbattocchi
kbattocchi changed the base branch from main to kebatt/fix-ci-build-failures September 28, 2026 19:39
@kbattocchi
kbattocchi added this pull request to stack #1070 September 28, 2026 19:44
@kbattocchi
kbattocchi force-pushed the kebatt/causal-forest-standard-errors branch from 983c0d7 to d75bfa5 Compare September 28, 2026 19:51
Base automatically changed from kebatt/fix-ci-build-failures to main October 1, 2026 19:06
@kbattocchi
kbattocchi force-pushed the kebatt/causal-forest-standard-errors branch from d75bfa5 to 722add0 Compare October 1, 2026 19:06
@kbattocchi
kbattocchi force-pushed the kebatt/causal-forest-standard-errors branch from 722add0 to 5cb7897 Compare October 20, 2026 20:59
Count non-missing predictions along the sample axis and preserve outcome-treatment ordering when forming doubly robust corrections.

Cover masked and missing predictions, fitted-estimator inference, and duplicate-outcome invariance with two, three, and four treatment arms. Keep API docstrings focused on caller-facing behavior and make the finite-data reference calculation's seed dependence explicit.

Fixes #1066

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
@kbattocchi
kbattocchi force-pushed the kebatt/causal-forest-standard-errors branch from 5cb7897 to faec41e Compare October 22, 2026 15:40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CausalForestDML(drate=True): ate_stderr_ / att_stderr_ are too small by sqrt(d_y*d_t) with multiple treatments or outcomes

1 participant