Repository navigation
Fix DynamicDML covariance orientation and OrthoForest marginal standard errors - #1077
kbattocchi wants to merge 2 commits into
Conversation
9fe7557 to
1a9ff87
Compare
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
2 open issues
What changed in this PR
Fixes two inference bugs affecting reported uncertainty (not point estimates) in DynamicDML and OrthoForest, and adds regression tests to prevent recurrence.
Changes:
- Correct DynamicDML cross-period Jacobian block orientation used in sandwich covariance calculation.
- Fix OrthoForest BLB marginal-effect inference to return standard errors (sqrt of projected variance).
- Add focused regression tests covering multidimensional shape cases and transformation/Jacobian paths.
| File | Description |
|---|---|
| econml/orf/_ortho_forest.py | Fixes BLB marginal-effect scaling to report standard errors by taking sqrt of projected variance. |
| econml/panel/dml/_dml.py | Corrects Jacobian block outer-product order for DynamicDML covariance and clarifies ordering via comments/docstring. |
| econml/tests/test_orf.py | Adds regression tests validating BLB transformed marginal-effect stderr/CI behavior across shapes and transforms. |
| econml/tests/test_dynamic_dml.py | Adds regression test comparing DynamicDML covariance/effect inference vs. independent finite-difference score Jacobian reference. |
Review effort: Lite
| scales[tuple(me_index)] = final | ||
| projected_var = np.einsum('mj, mjk, mk -> m', jac, cov, jac) | ||
| # Inference needs the standard deviation of the projected coefficient uncertainty. | ||
| scales[tuple(me_index)] = np.sqrt(projected_var) |
There was a problem hiding this comment.
Agreed that roundoff can make a theoretically nonnegative projected variance slightly negative. I'd defer clipping here: unconditional maximum(..., 0) would also hide materially invalid covariance estimates, and the existing effect and constant-marginal inference paths take square roots without clipping. Numerical hardening should use a consistent, scale-aware policy across those paths, with near-singular regression cases. This fix stays focused on converting variance to standard error.
| from sklearn.linear_model import Lasso, LogisticRegression, LogisticRegressionCV | ||
| from sklearn.multioutput import MultiOutputRegressor | ||
| from sklearn.preprocessing import FunctionTransformer | ||
| from econml.orf import DMLOrthoForest, DROrthoForest |
There was a problem hiding this comment.
Direct use of BLBInference is deliberate here. The comment beside factor explains that the stub supplies known coefficient covariance to isolate inference propagation from forest fitting. The inference methods themselves are real, and the test checks an independent factor-based standard-deviation reference plus agreement between identity-transform marginal and finite-contrast inference. A fitted-forest-only test would lose that controlled reference, so I'd retain this focused component test alongside the existing estimator tests.
1a9ff87 to
0ac6961
Compare
Differentiate panel scores with score coordinates on rows and future coefficient coordinates on columns. Check the fitted covariance against numerical score derivatives, preserve point estimates, and verify public effect standard errors across treatment, feature, and outcome shapes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 818b2f7a-7f36-4a43-b9d0-3e9af6470ee5 Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
Take the square root of projected covariance before constructing marginal inference results. Regress identity invariance, linear and quadratic transformations, correlated coefficient errors, scalar/vector shapes, and confidence intervals. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 818b2f7a-7f36-4a43-b9d0-3e9af6470ee5 Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
0ac6961 to
7a2c9d3
Compare


Summary
Stacked follow-up to #1069. Contains two independently confirmed inference fixes after auditing the codebase for other issues with multidimensional arrays; point-estimation logic is unchanged but we had bugs in our confidence intervals for DynamicDML and for OrthoForest's marginal effects when using an explicit treatment featurizer.
Regression coverage
alpha=0.1). Check point estimates, standard errors, and both confidence-interval endpoints througheffect_inferenceandeffect_interval. Preserve scalar/column outcome shapes. Both inference calculations run unmodified against the same fixed estimator fixture.Validation