Newcomers Help Most After Being Helped, Veterans Do Not

Replication package for:

Strahringer, L., Pruess, S. E., & Riemer, K. Newcomers Help Most After Being Helped, Veterans Do Not: Evidence from Stack Overflow. Computers in Human Behavior.

The repository contains everything needed to rebuild the study datasets from a public Stack Overflow data dump, refit the survival models, and regenerate every table and figure in the paper. It does not ship the data itself — see Getting the data.

• paper/manuscript.pdf — accepted manuscript
• paper/supplementary_material.pdf — online supplementary material (S1–S8)

---

What the study does

Each observation is a Stack Overflow question. Questions that received a first answer ("helped") are matched to comparable questions that did not, via propensity-score matching within year × tag strata. Each matched pair is then followed through a symmetric window around the question, and we estimate a difference-in-differences Cox model of the rate at which the asker goes on to answer other people's questions. The headline coefficient is the treated × post-question interaction (β₄), estimated separately by asker tenure.

---

Repository layout

preprocessing/   Stages 1–3 and 5–6 of the ETL: XML dump → question-centered dataset
                 → processed metrics → event histories. See preprocessing/README.md.
matching/        Stage 4: propensity-score matching, balance diagnostics, love plot.
analysis/        Cox model fitting, robustness checks, and table/figure generation.
  output_tables/   LaTeX tables included by the manuscript (committed).
  output_figures/  Figures in .pdf/.png/.eps (committed).
data/            Empty scaffolding. All contents are gitignored except the small
                 descriptive CSV in data/output/.
paper/           Accepted manuscript, supplementary material, and highlights.

The committed contents of analysis/output_tables/ and analysis/output_figures/ are the exact artifacts used in the published paper. Re-running the pipeline overwrites them, so git diff after a run is a direct reproduction check.

---

Getting the data

The study uses the public Stack Overflow data dump, which is released under CC BY-SA and is not redistributed here.

1. Download a dump. Stage 1 parses Posts.xml, Votes.xml, Users.xml, Comments.xml, and PostHistory.xml; Badges.xml is not used. See Stack Overflow's data-dump documentation and the schema reference.
2. Place the XML files in data/input/.
3. Run stage 1 (preprocessing/processing_data_dump.py) to convert them to Parquet alongside them in data/input/. Stage 1 skips any table whose .parquet already exists, so it is safe to re-run.

The published results use a dump current through 2025-04-01; that date is the cutoff_date in preprocessing/main.py and preprocessing/processing_reciprocity_dataset.py. A later dump will produce more observations and slightly different point estimates. Exact numerical reproduction requires the same dump vintage.

preprocessing/REVISION_DATA_PATHS.md maps each input file to the pipeline stage that consumes it, including the extra columns (ViewCount, comment and edit history, answer body length) added for the revision.

---

Environment

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Developed on Python 3.11–3.13. The pipeline scripts default to $REPO/.venv/bin/python; override with the PYTHON environment variable if your interpreter lives elsewhere.

---

Running the pipeline

After stage 1 has produced the Parquet inputs, the end-to-end driver runs stages 2 through 7:

bash preprocessing/run_revision_pipeline.sh

Stage by stage:

Stage	Command	Output
1	python preprocessing/processing_data_dump.py	data/input/*.parquet
2	python preprocessing/creating_raw_reciprocity_dataset.py	data/input/question_centered_model_7d_all_questions.parquet
3	python preprocessing/processing_reciprocity_dataset.py	data/study_datasets/*_processed.parquet
4	python matching/create_matched_dataset.py	data/study_datasets/*_matched.parquet
5a	python preprocessing/calculate_matched_questions_helping.py	matched_questions_helping_metrics.parquet
5b	python preprocessing/create_matched_event_histories.py	data/event_history/study_{timelines,events}.parquet
6	cd analysis && python fit_cox_models.py --input ../data/event_history	analysis/model_cache/*.csv
7	cd analysis && python create_figures.py	analysis/output_tables/*.tex, analysis/output_figures/*

preprocessing/main.py is a convenience wrapper around stages 2 and 3. analysis/clear_caches.py deletes the model and data caches so a run rebuilds from scratch; fit_cox_models.py --no-cache does the same for model fits only.

Resume points exist for every stage — preprocessing/run_revision_pipeline_from_stage5.sh, ..._from_stage6.sh, ..._resume.sh — because a full run takes many hours.

Robustness and supplementary analyses

create_figures.py is the single renderer for the paper's LaTeX tables: the robustness scripts below write their results into analysis/model_cache/*.csv, and stage 7 turns those caches into analysis/output_tables/*.tex. Run the relevant script before stage 7, or the corresponding table will be skipped.

Script	Feeds / produces
analysis/revision_robustness.py	Caches behind observable_controls.tex, answer_quality_robustness.tex, score_coding_sensitivity.tex, composite_outcome.tex
analysis/cohort_robustness.py, analysis/cohort_post2020.py	Caches behind cohort_robustness.tex
analysis/pair_bootstrap_se.py	Caches behind pair_bootstrap.tex (matched-pair bootstrap SEs for β₄)
analysis/newcomer_preperiod_placebo.py	Caches behind newcomer_robustness.tex (pre-period placebo)
analysis/effect_sizes.py	Writes absolute_effects.tex directly
analysis/selection_sensitivity.py	Selection/churn sensitivity summaries
analysis/help_rate_over_time.py	Writes help_rate_pooled.*, help_rate_by_tenure.*, help_rate_adoption_*.*
analysis/rt_bin_figure_from_tables.py	Rebuilds interaction_effect.* from the response-time-bin tables, without the model cache
matching/create_matched_dataset.py	Writes matching/balance_check.tex as a side effect of stage 4
matching/refresh_matching_plots.py	Writes matching/love_plot.pdf, matching/common_support.pdf
preprocessing/investigate_question_loss.py	Observation counts at each pipeline stage
preprocessing/sanity_checking_reciprocity_dataset.py	Null/consistency/range checks on the processed dataset

---

Compute requirements

This is a large-data pipeline. The full Stack Overflow dump is tens of gigabytes, stages 2–4 use DuckDB with high memory limits, and the Cox fits hold millions of interval rows in memory (COX_MAX_FIT_ROWS, default 8,000,000).

The published results were produced on an HPC cluster. preprocessing/slurm_*.sbatch, analysis/slurm_*.sh, and the submit_*.sh wrappers are the SLURM job scripts used, sharded where the work parallelizes (matching, bootstrap). They are included for transparency and are cluster-agnostic: submit them from the repository root, since REPO defaults to $SLURM_SUBMIT_DIR and the #SBATCH log paths are repo-relative. Override REPO or PYTHON in the environment if you need different locations.

Memory and time budgets in the #SBATCH headers (up to 200 GB, several hours per stage) reflect what these jobs actually needed.

---

Notes for readers of the code

• ISS-NN tags in comments (e.g. ISS-04, ISS-37) are our internal revision-tracking IDs from the review process. They mark code written in response to a specific reviewer point; the surrounding comment explains the substance.
• Estimand. HEADLINE_ESTIMAND = "arrival" in analysis/cox_config.py selects β₄ as the reported effect. The summed difference-in-differences is computed internally by some robustness scripts but is deliberately never emitted into a table or caption.
• Primary outcome. PRIMARY_HELP_TYPES = ["answer"] — help means answering another user's question. Comment- and edit-inclusive composites appear only in the outcome decomposition.
• Caches. DATA_VERSION in cox_config.py is bumped whenever data or covariate construction changes, so stale caches miss rather than silently mixing generations.

---

Citation

See CITATION.cff.

License

Code is released under the MIT License. The Stack Overflow data dump is not included and remains under its own CC BY-SA terms. The manuscript PDFs in paper/ are subject to the publisher's copyright.