Question. Does the employment response to the 2021 round of state minimum-wage increases vary with each state-industry's pre-policy exposure to the new wage floor?
Method. A difference-in-differences design: 19 states that raised their minimum wage in 2021 Q1 vs. 26 states with no change that year, quarterly food-service and retail employment, 2015-2022, state and quarter fixed effects, state-clustered inference — extended with an interaction term letting the effect scale with a state-industry's pre-policy exposure share.
Finding. The baseline average effect is negative but not credible as a causal effect, and the exposure-gradient extension is underpowered and confounded. See Results at a Glance below.
Why the finding is uncertain. Parallel pre-trends — the assumption this design's causal claim actually rests on — are directly tested (Section 4.1 of the full report) and found to be rejected for both food service and retail by the event-study joint tests. That is this study's real identification problem, checked directly rather than assumed, and it's why every result below is reported with that caveat attached rather than as a clean causal estimate.
Reproduce: Rscript -e 'renv::restore()' && make all — see
How to Reproduce for the full command list.
Primary contribution — causal inference / econometrics: a Card-Krueger-style state minimum-wage DiD design, extended to ask how the effect scales with pre-policy exposure and stress-tested with every credibility check the data will support (event study, placebo test, COVID-sensitivity spec, wild cluster bootstrap, permutation test, specification curve, power analysis). This is the project's actual research contribution and the rest of this README is organized around it first.
Secondary — predictive modeling comparison: an ML/neural-network predictive-accuracy comparison (R and Python) and two exploratory checks (treatment predictability, state clustering) are included as extensions, not co-equal findings — see Extensions below. They inform the causal analysis's credibility checks; they don't replace them.
Primary: How does the employment response to state minimum-wage increases vary with pre-policy minimum-wage exposure across low-wage industries?
Secondary:
- Does minimum-wage treatment affect employment in food service and retail?
- Is the effect larger where exposure is greater?
- Does the effect differ across Census regions?
- How sensitive are these conclusions to alternative treatment definitions and inference procedures?
- Did the increases actually raise earnings in low-wage industries, and by how much relative to the size of the mandate (pass-through)?
- Is treatment status itself predictable from pre-period state characteristics, and do states form natural economic clusters that track treatment status? (Exploratory; see Extensions — this does not itself bear on whether the DiD design is valid, since DiD doesn't require random assignment.)
Which analyses were specified before results were seen, and which were added afterward, matters for how much weight a p-value can carry.
| Analyses | Status | |
|---|---|---|
| Confirmatory (pre-specified in the original proposal) | Model A (β₃), Model C (β₄, pre-registered as likely underpowered), event study, COVID-sensitivity spec, placebo test, wild cluster bootstrap, permutation test, power analysis | Reported regardless of significance |
| Exploratory (added after the original build, motivated by what the confirmatory results showed) | Bandwidth sensitivity, marginal effects, treatment intensity, leave-one-out, multiple-testing correction, specification curve, exposure–COVID correlation, COVID-controlled placebo, wage pass-through, treatment predictability, state clustering, ML/NN comparison | Informs interpretation; not held to the same inferential standard as the confirmatory family |
| Sample | 50 states, quarterly 2015-2022 (1,260 obs in the estimation panel) — 19 treated, 26 control, 5 excluded |
| Model A, β₃ (avg. effect) | -0.020 food service (p=0.22), -0.009 retail (p=0.44); only food service is significant in the ≥$0.50 subsample |
| Model C, β₄ (exposure gradient) | 0.143 food service (p=0.43), -0.006 retail (p=0.94) — noisy and inconsistent in sign |
| Robustness on β₄ | 4/4 independent checks (placebo, permutation, event study, spec curve) flag it as confounded |
| Inference | State-clustered SEs, wild cluster bootstrap, 10,000-rep permutation test, Monte Carlo power analysis |
| Wage pass-through | 13% (food service proxy, p=0.65) / 63% (retail proxy, p=0.22) of the mandated increase — neither significant |
| Treatment predictability | Logistic regression on pre-period growth + exposure: 87% accuracy, AUC 0.97 — highly predictable |
| State clustering | k-means (k=3) on the same features: 2 of 3 clusters are essentially pure treated/control (χ²p<0.0001) |
| Engineering | 32 R scripts, 369 R assertions and 61 Python tests (all passing); CI on every push; make all for full reproduction |
| Question | Result | Interpretation |
|---|---|---|
| Average employment effect | Negative but not robust | Weak causal evidence |
| Parallel trends | Rejected for both industries (event-study joint tests) | Major identification concern |
| COVID sensitivity | Large change once controlled for | Confounding likely |
| Exposure gradient (β₄) | Inconclusive and confounded | Underpowered, not "no effect" |
| Placebo / permutation / event study / spec curve (β₄) | All 4 flag a pre-existing exposure-outcome link | Convergent evidence, not one fluke |
| Exposure vs. COVID severity | Significant negative correlation | A candidate mechanism — but controlling for it barely moves the placebo effect |
| Multiple-testing correction | None of the 4 confirmatory tests survive Holm | Consistent with "not robust," now formal |
| Leave-one-state-out | Estimate stable across all 19 drops | No single state drives Model A |
| Wage pass-through | 13%/63% of the mandated increase, neither significant | Consistent with underpowered, not "no pass-through" |
| Is treatment predictable from pre-period data? | Yes — 87% accuracy, AUC 0.97 | Doesn't invalidate DiD, but motivates checking trends directly |
| Do states cluster by treatment status? | Yes — 2/3 k-means clusters are nearly pure | Same caveat — see event study (Section 4.1) for the actual test |
|
Which states were treated, and by how much
|
Event study: pre-trends don't hold for food service
|
|
Baseline vs. COVID-adjusted estimate
|
Pre-2021 exposure distribution by industry
|
Does the treatment effect scale with exposure?
All four confirmatory estimates at a glance
Every reasonable band × sample × industry choice, ranked
Exposure predicts COVID severity — a candidate mechanism for the confound
More figures — model diagnostics, permutation-test null distributions,
power-analysis curves, the beta4 event study and forest plot — are in
reports/figures/ and embedded in the
full report.
- Parallel trends violated for both industries in the event study; the separate two-sample test rejects for food service only — this study's central, load-bearing identification problem.
- COVID-era differential recovery confounds the treatment window — a real share of the baseline estimate, not the policy.
- Both β₃ and β₄ are underpowered at the observed effect sizes
(
R/16_power_analysis.R), not just assumed underpowered. - β₄ is also confounded, not just underpowered — 4 independent
checks find high/low-exposure states already differed before 2021. A
candidate mechanism (exposure correlates with COVID severity) was
tested directly by controlling for it in the placebo spec
(
R/25-R/26) — the effect barely shrank. See report Section 8.1. - The ≥$0.50 subsample and the legislated-only subsample are the exact same 10 states — the two cuts can't be disentangled in this data.
- Small cluster count (19 treated, 45 total) — addressed via clustered SEs and a wild bootstrap, but always some caution.
- Wage pass-through uses broader supersector proxies, not the exact food-service/retail industries — BLS/FRED don't publish earnings at that resolution by state (see Extensions).
- Exposure-measure construction, spillovers, anticipation effects, and external validity are documented but not fully resolved.
- Results are DiD estimates under this specification, not definitive causal effects — see the full report's Limitations section for detail.
Identification structure. Not a generic textbook DAG — the red path
below is this study's own confounding structure, established
empirically (exposure predicts treatment status: R/30; exposure
correlates with COVID severity: R/25-R/26), not assumed:
Read the rendered report (HTML) or download the PDF — no cloning required. Or render reports/final_report.Rmd directly:
Rscript -e 'rmarkdown::render("reports/final_report.Rmd")' # HTML
make report-pdf # + PDF, via headless Chrome
(needs pandoc — brew install pandoc on macOS — for
the HTML, and Chrome/Chromium installed locally for the PDF; see
scripts/render_report_pdf.R for why it's a headless-Chrome print
rather than a LaTeX pipeline).
Everything below is secondary to the causal analysis above, not co-equal with it: a second research question (wage pass-through), two exploratory checks on treatment assignment, and a predictive-modeling comparison. None of them revise the causal estimates or their limitations; they're included because they add real information, not to pad the project's scope.
A new research question, not a re-analysis of the one above: did the 2021 increases actually raise earnings in low-wage industries, and by how much relative to the dollar size of the mandate (pass-through)? Everything above asks whether employment fell; this asks whether the policy did the thing it was actually meant to do.
Data limitation, found by direct request (R/29_wage_passthrough.R):
BLS/FRED do not publish state-level average hourly earnings at the
detailed NAICS-722 (food service) or 44-45 (retail trade) level the
employment analysis uses — every candidate series ID at that resolution
404s. The finest available state-level breakdown is the CES
supersector: "Leisure and Hospitality" (nests food service, but also
accommodation and arts/entertainment/recreation) as the closest proxy for
food service, and "Trade, Transportation, and Utilities" (nests retail,
but also wholesale trade, transportation, and utilities) as the closest
proxy for retail. Both are broader than the industries used elsewhere in
this study, so this answers a related but not identical question —
not "did retail workers' wages rise" but "did wages in retail's broader
supersector rise."
| Proxy industry | treated_post (log pts) | p | Mandated $/hr | Estimated $/hr | Pass-through |
|---|---|---|---|---|---|
| Leisure & Hospitality (food service proxy) | 0.0044 | 0.647 | $0.54 | $0.07 | 13% |
| Trade, Transportation & Utilities (retail proxy) | 0.0144 | 0.217 | $0.54 | $0.34 | 63% |
Neither estimate is statistically significant, and the food-service proxy's confidence interval crosses zero on both sides — consistent with this study's broader theme that these designs are underpowered at the observed effect sizes, not evidence that pass-through is genuinely 13%/63% versus some other number. The likeliest reason both estimates sit well under 100%: the supersector proxies include large numbers of workers (hotel managers, arts/entertainment/recreation staff, wholesale and utilities employees) who were never near the minimum wage to begin with, diluting any real effect on the low-wage workers actually targeted.
Exploratory analyses investigating systematic differences in pre-treatment economic structure between treated and comparison states, using techniques not used elsewhere in this study (a chi-square test of independence, logistic regression as a classifier, k-means, PCA).
Is treatment status independent of Census region?
(R/30_treatment_predictability.R) A chi-square test of independence
says no (χ²=8.0, df=3, p=0.046) — the South is heavily under-represented
among treated states (2 of 20) relative to control (12 of 25).
Is treatment status predictable from pre-period state characteristics? A logistic regression on pre-period (2019-2020) GDP growth, population growth, and food-service exposure — the same variables used elsewhere in this study, not new ones picked for this check — classifies treated vs. control states with 87% accuracy and an AUC of 0.97 at n=45. Exposure alone is the strongest, most significant predictor (p=0.013).
Do states form natural economic clusters, and do those clusters track
treatment status? Unsupervised this time — no label is given to the
algorithm. A hand-rolled k-means (with k-means++ initialization; see
R/31_state_clustering.R) on the same pre-period features, k=3 chosen
from a gradual (not sharply-elbowed) WCSS curve, finds that 2 of the 3
clusters are almost pure treated or pure control (χ²=28.0, p<0.0001):
On interpretation — this does not, by itself, invalidate the causal design. DiD does not require random treatment assignment. States choosing to raise their minimum wage for identifiable political or economic reasons is entirely compatible with valid causal identification, provided the relevant conditional parallel-trends assumption holds. Treatment being this predictable doesn't do the identification work itself — what it does is raise the prior that treated and control states might follow different counterfactual trends, which is exactly what Section 4.1's event study tests directly rather than assumes. For both industries, that direct test finds pre-trends are violated (Section 4.1) — that is the actual identification problem this report leads with, not the predictability result. The predictability and clustering findings here are a reason to take the event study's verdict seriously, not a second, independent verdict of their own.
A predictive-accuracy comparison, separate from the causal claims above: how well can flexible ML methods predict quarterly employment growth for a state they never saw during training, versus a plain linear specification? Evaluated by holding out entire states (not rows) from training, since a row-level split would leak within-state structure the causal models' clustered SEs already treat as non-independent.
- R (
R/27_ml_prediction.R,R/28_neural_network.R): a linear baseline, a hand-rolled bagged-trees ensemble (bootstrap-aggregatedrparttrees, one fixed random feature subset per tree), a real random forest (therandomForestpackage — re-samples features at every split, not once per tree), gradient boosting (thegbmpackage, cross-validated to pick the number of trees), and a single-hidden-layer neural network (nnet). - Python (
python/): an independent replication in a separate ecosystem — numpy/pandas for data handling, a from-scratch regression tree with a bagging ensemble, a true random forest (per-split feature re-sampling), and gradient boosting (sequential trees fit to residuals) all hand-rolled since no ML library is usable here, plus a PyTorch feedforward network — reading the same panel the R side exports. See python/README.md for why it doesn't use scikit-learn.
On the held-out states, gradient boosting is the strongest predictor in both languages (R: R²=0.64; Python: R²=0.49), ahead of the neural network and both other tree ensembles — a sensible result, not a foregone one, since boosting's sequential residual-fitting is a genuinely different strategy from bagging/forests' independent, averaged trees.
This is a methods comparison, not a substitute for Model A/C's identification strategy — a model that predicts employment well isn't thereby estimating the policy's causal effect on it.
The Python package also provides state-grouped cross-validation, grid search,
cluster bootstrap intervals and permutation importance. Its exploratory
cross-fitted DML and linear R-learner use employment growth and region controls,
not Model A/C's outcome and fixed effects. Their identifying assumptions differ:
conditional unconfoundedness is not equivalent to DiD parallel trends.
DML standard errors now cluster by state with a G/(G-1) correction;
the CSV retains se_row for comparison. The 95% intervals use a normal
approximation and assume independent states, adequate nuisance fits and
residual treatment variation. They do not address cross-state shocks or
establish the identifying assumptions. See the Python results
for the comparison.
The read-only FastAPI service returns precomputed CSV results and responds with 404 when a result is unavailable. It does not refit models. The live site's wage and ML charts display saved R results, not API responses. See Python setup and endpoints.
A significant DiD estimate isn't sufficient evidence of a causal effect. The event study and COVID-sensitivity spec caught exactly the failure modes they were built to catch. Extending the same rigor to the exposure gradient afterward — power analysis, four robustness checks, then a direct test of the candidate confounding mechanism — reinforced it from another angle: a fragile result isn't evidence of "no effect," and proposing a mechanism isn't the same as confirming one.
| Source | Role |
|---|---|
| FRED (public API) | State-quarter employment, GDP, population, 2015-2022 |
| U.S. DOL minimum wage history | Treatment classification |
| QCEW Open Data API | Fallback employment (2 states); validation check |
| CPS-ORG via IPUMS-CPS | Exposure construction (2019-2020 earnings only) |
| FRED CES average hourly earnings (supersector) | Wage pass-through analysis (R/29) |
| Variable | Source | Frequency | Transformation | Role in identification |
|---|---|---|---|---|
log_employment (food service / retail) |
FRED CES; QCEW fallback for 2 states | Monthly → quarterly mean | Natural log | Outcome |
treated_post |
U.S. DOL minimum wage history | Static per state × time-invariant cutoff | Binary (treated × post-2021Q1) | Treatment indicator (Model A/C) |
exposure (12.5% band) |
CPS-ORG via IPUMS-CPS, 2019-2020 pooled | Static per state-industry | Weighted share of workers within band of pre-2021 minimum wage | Moderator — the exposure-gradient hypothesis itself (Model C) |
gdp_growth |
FRED (state GDP) | Quarterly | Year-over-year log growth | Control |
pop_growth |
FRED (state population) | Annual, repeated across quarters | Year-over-year log growth | Control |
region |
Census Bureau region classification | Static | Categorical (4 levels) | Heterogeneity / robustness (ANOVA, chi-square) |
covid_severity |
Derived from FRED employment (own construction) | Static per state | log(2020 Q2 employment) − log(2019 Q4 employment) | COVID-sensitivity covariate (Section 4.3) |
wage_food_service_proxy, wage_retail_proxy |
FRED CES average hourly earnings, supersector | Monthly → quarterly mean | Natural log | Outcome, wage pass-through (R/29) only |
Rscript -e 'renv::restore()' # pinned package versions (renv.lock, R 4.5.1)
make all # R/01-R/32, tests, figures, report end to end
make test # just the R test suite
make report # just render reports/final_report.Rmd (HTML)
make report-pdf # + reports/final_report.pdf (headless Chrome)
make python-setup # one-time: creates python/.venv, installs the minwage package
make python-test # Python ML/NN test suite (python/)
make all needs IPUMS_API_KEY in .env (see .env.example) for
R/03, and takes several minutes (CPS-ORG pull, wild bootstrap,
permutation test, power simulation are the slow steps). Rendering the
HTML report needs pandoc, separate from
renv.lock; make report-pdf additionally needs Chrome/Chromium
installed locally (no LaTeX distribution required).
make python-test needs make python-setup to have been run once
first; see python/README.md for why it's a
separate virtual environment rather than a renv-tracked dependency.
| Item | Value |
|---|---|
| R version | 4.5.1 (pinned in renv.lock; renv::restore()) |
| Python version | 3.10.16 locally, 3.11 in CI; numpy 2.2.6, pandas 2.3.3, torch 2.14.0 (exact pins in python/pyproject.toml) |
| Seed policy | Every stochastic step takes an explicit seed argument. R defaults to seed = 1 (R/11, R/15, R/16, R/27, R/28, R/31); the power simulations offset it per grid point (seed + i), and the β₃ power curve starts from seed = 100. All Python scripts use seed = 1. |
| Test counts | 369 R assertions across 32 test files (tests/), about 52 s; 61 Python tests (python/tests/), runtime varies by machine; Python coverage gate 90% (about 99% actual) |
| Data: FRED employment, GDP, population | Public CSV endpoint, pulled live with no vintage pinned, so a re-pull can differ if FRED revises a series. The local processed files were pulled 2026-08-23 |
| Data: FRED CES average hourly earnings | Same live pull, no pinned vintage; local file dated 2026-09-19 |
| Data: CPS-ORG (exposure) | IPUMS-CPS extracts for 2019 and 2020, requires IPUMS_API_KEY; extract definitions are in R/03, and the downloaded files are not committed |
| Data: DOL minimum wage history | Transcribed into data/treatment_classification.csv (committed) and cross-checked against two Wayback Machine snapshots of DOL's state table (2020-12-01 and 2021-02-02); provenance notes are in R/01_treatment_classification.R |
Runtime: R/32_modern_did.R |
about 4 s |
Runtime: python/train.py |
about 75 s (grouped-CV grid search is most of it) |
Runtime: python/causal_estimate.py |
about 28 s |
Runtime: full make all |
not measured end to end here; it needs a live IPUMS key and network, and the wild bootstrap, 10,000-rep permutation test and power simulations dominate |
Exact reproduction of the committed results needs the same FRED vintage, which isn't archived here; a fresh pull should agree up to data revisions, not bit for bit.
Repo layout: R/ (32 numbered scripts, run in order), tests/ (one
file per script), reports/ (final_report.Rmd, rendered HTML and PDF,
figures), data/processed/ (all results, gitignored, reproducible),
python/ (independent ML/NN replication — see
python/README.md). CI (.github/workflows/ci.yml)
parse-checks R/, verifies every script is wired into the Makefile and
has a matching test file (scripts/check_pipeline_sync.R — also
runnable locally as make check-sync), and runs both the R and Python
test suites, on every push.
- Includes the complete R analysis pipeline and report, with 32 numbered scripts and 369 R assertions across 32 test files.
- Includes the Python predictive and causal ML extension, read-only results API, and 61 Python tests.
- Reports rejected event-study pre-trends for both industries and describes the limits on causal interpretation throughout the report and site.
- Carries citation metadata in
CITATION.cff; the DOI and archive badge will be added after the Zenodo archive and release exist.
The v1.0.0 release is prepared locally. Zenodo archival, DOI assignment and
publication remain pending; no DOI is published yet.
Complete. The original 10-day build covered treatment classification through the final report;
11 new scripts (R/16-R/26) and 4 extended ones (R/08, R/10,
R/11, R/15) were added afterward for the power analyses, robustness
checks, and mechanism investigation summarized above; R/27-R/28 and
python/ add the ML/neural-network predictive comparison described
above, R/29 adds the wage pass-through research question, and
R/30-R/31 add the treatment-predictability and state-clustering
angles.
MIT — the code. Underlying data remains subject to each source's own terms (see Data Sources above); this license covers the analysis code, not the data itself.