From 8edd5defc1d71b65ba962cda0787989836f1b9b9 Mon Sep 17 00:00:00 2001 From: "Joshua C. Macdonald" Date: Sat, 3 Oct 2026 06:02:04 -0400 Subject: [PATCH] docs: present serosurvey notebook for a public audience --- docs/guide/serosurvey.md | 42 +++- examples/serosurvey_study.ipynb | 361 ++++++++++++++------------------ 2 files changed, 195 insertions(+), 208 deletions(-) diff --git a/docs/guide/serosurvey.md b/docs/guide/serosurvey.md index 5676287..f5cefad 100644 --- a/docs/guide/serosurvey.md +++ b/docs/guide/serosurvey.md @@ -12,6 +12,8 @@ to read the narrative, code, tables and saved figures. The accompanying [Python script](https://github.com/jcm-sci/trade-study/blob/main/examples/serosurvey_study.py) contains the simulator and plotting helpers and regenerates these figures. Use both files from a checkout; the notebook imports the companion module. +The notebook addresses the presentation audience directly. Setup commands, +export instructions and the suggested running order are collected in this guide. ## Run or present the notebook @@ -40,6 +42,16 @@ to HTML for an additional presentation copy: uv run --extra notebook jupyter nbconvert --to html examples/serosurvey_study.ipynb ``` +To omit the collapsed setup code from a presentation copy while keeping the +analysis code visible: + +```bash +uv run --extra notebook jupyter nbconvert --to html \ + --TagRemovePreprocessor.enabled=True \ + --TagRemovePreprocessor.remove_input_tags='["hide-input"]' \ + examples/serosurvey_study.ipynb +``` + To verify execution in a fresh kernel without modifying the saved notebook: ```bash @@ -99,7 +111,7 @@ signed errors before taking their absolute value would measure something differe ## Inspect feasible alternatives -A `Constraint` imposes an illustrative $40,000 financial budget. The Pareto +A `Constraint` imposes an illustrative \$40,000 financial budget. The Pareto set minimizes cost, overall MAE and underserved-group MAE simultaneously. Subgroup error is a narrow measure of information equity, not a comprehensive measure of equity in health outcomes. @@ -118,7 +130,7 @@ not simultaneous post-selection confidence guarantees. The notebook passes the raw results to `preference_sweep()` with three explicit preference vectors and `normalization="reference"`. Fixed reference ranges are -$0–70,000, 0–4 percentage points overall MAE, and 0–10 percentage points subgroup +\$0–70,000, 0–4 percentage points overall MAE, and 0–10 percentage points subgroup MAE. These anchors scale preferences; they are not feasibility thresholds and do not clip values. Scenario weights are hypothetical, not elicited stakeholder values. @@ -130,7 +142,7 @@ can change with further simulation or different assumptions. Any reported selection fraction describes the supplied preference scenarios, not a probability that a design is best. -For a short live interaction, change the budget to $30,000 in the decision +For a short live interaction, change the budget to \$30,000 in the decision cell and rerun that cell and the figures below it. Restore the budget and edit the subgroup-priority weights to compare another preference. No simulation rerun is needed. If the budget admits no alternative, the example reports no choice. @@ -151,10 +163,11 @@ The optional cost breakdown in the appendix supports an economics discussion: | 12–13 | Assumptions a real project would replace | | 13–15 | Discussion | -The notebook includes presenter notes and slideshow cell metadata. Leave the -model details and cost breakdown as appendices for the main talk. The saved -notebook and HTML export make a Beamer build unnecessary for the current -fast-running example. +The notebook's setup cells are collapsed, and slideshow metadata identifies +the main narrative and its figures. The model details and cost breakdown are +technical appendices. Use this guide's running order and budget/priority changes +to prepare the presentation; saved notebook outputs and HTML support a version +without live execution. ## What a real project would replace @@ -166,8 +179,19 @@ fixed and does not equate antibody-status prevalence with complete protection. Communities are sampled without selection bias by construction. Real selection and nonresponse can introduce bias that more simulation cannot remove. -The source notebook links the IVAC projects and member profiles that informed -its scope. Those connections do not imply endorsement of the example. +The notebook's references link to related serosurveillance work and its costing +literature. They provide context for the scientific question; the numerical +assumptions are illustrative. + +## Export the results + +From a notebook code cell, export the per-design scores and decision metadata: + +```python +report.summary.to_dataframe(include_metadata=True).to_csv( + "serosurvey_designs.csv", index=False +) +``` To regenerate documentation figures: diff --git a/examples/serosurvey_study.ipynb b/examples/serosurvey_study.ipynb index ac7096a..93c398e 100644 --- a/examples/serosurvey_study.ipynb +++ b/examples/serosurvey_study.ipynb @@ -11,48 +11,15 @@ "source": [ "# How much survey is enough—and whose uncertainty matters?\n", "\n", - "### A 15-minute trade-study demonstration for IVAC\n", + "### Balancing cost, accuracy and representation in serosurveys\n", "\n", - "**Decision:** choose a serosurvey design that balances financial cost,\n", - "overall estimation accuracy, and accuracy for an underserved group.\n", + "How can we design an affordable survey that gives useful estimates\n", + "for the whole population and for communities that are often underserved?\n", "\n", - "Every population, prevalence, price and priority below is **hypothetical**.\n", - "We estimate antibody-status prevalence; this example does not equate it\n", - "with complete protection or prescribe a real survey." - ] - }, - { - "cell_type": "markdown", - "id": "69e09179", - "metadata": { - "slideshow": { - "slide_type": "notes" - } - }, - "source": [ - "## Presenter route and setup\n", - "\n", - "| Minutes | Story |\n", - "|---|---|\n", - "| 0–2 | One decision, two population groups |\n", - "| 2–4 | What can we change, and what counts as success? |\n", - "| 4–6 | Run 18 designs with repeated simulated surveys |\n", - "| 6–10 | Inspect budget feasibility and Pareto alternatives |\n", - "| 10–12 | Change priorities without rerunning the model |\n", - "| 12–13 | What would a real project replace? |\n", - "| 13–15 | Discussion |\n", - "\n", - "From a checkout of the repository, start with:\n", + "We will compare 18 survey designs, examine their trade-offs, and see how\n", + "the preferred design changes with the budget and the priorities we choose.\n", "\n", - "```bash\n", - "uv run --extra notebook jupyter lab examples/serosurvey_study.ipynb\n", - "```\n", - "\n", - "Choose the environment's Python kernel, then **Restart Kernel and Run All Cells**.\n", - "The saved outputs also support a presentation without execution. Model and\n", - "plotting helpers live beside this notebook in `serosurvey_study.py`; the\n", - "trade-study orchestration is visible here. Installation needs network access;\n", - "execution after installation uses local code and synthetic data." + "*An illustrative example using synthetic populations and costs.*" ] }, { @@ -66,9 +33,15 @@ "iopub.status.idle": "2026-10-03T09:22:20.187824Z", "shell.execute_reply": "2026-10-03T09:22:20.185881Z" }, + "jupyter": { + "source_hidden": true + }, "slideshow": { "slide_type": "skip" - } + }, + "tags": [ + "hide-input" + ] }, "outputs": [], "source": [ @@ -104,9 +77,15 @@ "iopub.status.idle": "2026-10-03T09:22:20.476987Z", "shell.execute_reply": "2026-10-03T09:22:20.475256Z" }, + "jupyter": { + "source_hidden": true + }, "slideshow": { "slide_type": "skip" - } + }, + "tags": [ + "hide-input" + ] }, "outputs": [], "source": [ @@ -152,19 +131,20 @@ } }, "source": [ - "## 1. A familiar decision\n", + "## 1. Where should we invest in better evidence?\n", "\n", - "We need useful evidence about a population, but have a finite survey budget.\n", - "An overall estimate can be reasonably accurate while a smaller, underserved\n", - "group remains poorly measured.\n", + "A survey has a finite budget. An accurate overall estimate can still\n", + "leave a smaller, underserved group poorly measured.\n", "\n", - "This problem connects to IVAC's [SISS work](https://publichealth.jhu.edu/ivac/our-work/strengthening-immunization-systems-through-serosurveillance-siss)\n", - "and the [serosurvey costing analysis by Carcelen, Patenaude, Moss and colleagues](https://pmc.ncbi.nlm.nih.gov/articles/PMC7561102/).\n", - "The paper motivates separating study, community and participant costs;\n", - "we use **invented coefficients**, not its historical prices.\n", + "Serosurvey design brings these questions together: how many people to\n", + "sample, how many communities to visit, and how to allocate effort across\n", + "population groups. IVAC's [SISS project](https://publichealth.jhu.edu/ivac/our-work/strengthening-immunization-systems-through-serosurveillance-siss)\n", + "explored the design and use of serological surveillance. A related\n", + "[costing study by Carcelen and colleagues](https://pmc.ncbi.nlm.nih.gov/articles/PMC7561102/)\n", + "distinguished study-, community- and participant-level costs.\n", "\n", - "**Audience question:** Would you spend additional resources on more people,\n", - "more communities, or better representation of a smaller group?" + "**Where would you invest the next dollar: more participants, more\n", + "communities, or better measurement of a smaller group?**" ] }, { @@ -211,20 +191,24 @@ "source": [ "## 2. Three choices, three objectives\n", "\n", - "| Choice | Levels |\n", + "| Design choice | Alternatives |\n", "|---|---|\n", "| Participants | 300, 600, 1,200 |\n", "| Communities | 10, 20, 40 |\n", "| Allocation | Proportional (80:20) or oversampling (50:50) |\n", "\n", - "That gives **3 × 3 × 2 = 18 designs**. Allocation applies to both participants\n", - "and communities. Within each group, participants are spread as evenly as\n", - "possible across its communities, preserving the total exactly.\n", + "These choices produce **18 survey designs**. Allocation applies to both\n", + "participants and communities. Within each group, participants are spread\n", + "as evenly as possible across the sampled communities.\n", + "\n", + "We compare three outcomes, each of which we want to minimize:\n", + "\n", + "- Financial cost.\n", + "- Mean absolute error in the overall prevalence estimate.\n", + "- Mean absolute error in the underserved group's prevalence estimate.\n", "\n", - "We minimize **financial cost**, **overall mean absolute error**, and\n", - "**underserved-group mean absolute error**. Errors are in percentage points.\n", - "Subgroup error is a specific measure of *information equity*, rather than a\n", - "complete measure of equity in health outcomes." + "Errors are measured in percentage points. Subgroup accuracy tells us how\n", + "well the smaller group is measured: one aspect of information equity." ] }, { @@ -325,19 +309,22 @@ } }, "source": [ - "## 3. The simulator and scorer do different jobs\n", + "## 3. How do we compare estimation accuracy?\n", "\n", - "**Simulator:** draw community-level probabilities around the fixed group means,\n", - "then draw antibody-status counts within communities. The illustrative\n", - "within-community correlation is 0.06. Each design/replicate uses its own\n", - "reproducible random stream.\n", + "In a simulation, the population prevalences are known. We can repeat a\n", + "survey and measure how far its estimates are from those target values.\n", "\n", - "**Scorer:** compute the absolute difference from known synthetic truth and attach\n", - "the financial cost. Average these absolute errors across repeated surveys to\n", - "estimate **mean absolute error (MAE)**.\n", + "The **simulator** draws antibody-status counts from communities with\n", + "different underlying prevalences. The assumed within-community\n", + "correlation is 0.06. Random streams are reproducible and independent\n", + "across designs and repeated surveys.\n", "\n", - "**Important:** use the population shares, 80:20, when combining group estimates.\n", - "Sampling 50:50 does not make the population 50:50." + "The **scorer** records the financial cost and absolute estimation errors.\n", + "Averaging the errors across repeated surveys gives **mean absolute error\n", + "(MAE)**: the typical size of an estimation error under this model.\n", + "\n", + "We combine the group estimates using the population shares, **80:20**.\n", + "Oversampling changes who we sample; the population weights stay the same." ] }, { @@ -446,15 +433,18 @@ } }, "source": [ - "## 4. Trade-study orchestrates the comparison\n", + "## 4. Compare all 18 designs\n", + "\n", + "`trade-study` organizes the designs, runs the simulated surveys, and\n", + "collects their costs and errors in a results table.\n", "\n", - "First run 100 simulated surveys per design, then run 1,000 per design with an\n", - "**independent phase seed**. Both phases evaluate the same 18 designs. We keep\n", - "all of them because this model is cheap; premature elimination is unnecessary.\n", + "We begin with 100 simulated surveys per design, then refine the comparison\n", + "with an independent set of 1,000 surveys per design. All 18 designs are\n", + "evaluated in both phases.\n", "\n", - "Refinement increases the number of *simulated surveys*, not the participant\n", - "count in a survey. Its estimates replace the screening estimates; the two\n", - "phases are not pooled." + "More simulation replication improves the precision of our estimated\n", + "performance. Each design's participant count stays fixed. The refined\n", + "results provide the evidence for the comparisons that follow." ] }, { @@ -515,19 +505,19 @@ } }, "source": [ - "## 5. Separate eligibility from preference\n", + "## 5. Which designs fit the budget?\n", "\n", - "A financial ceiling is a **hard constraint**. Within that ceiling, some designs\n", - "have lower cost, some have lower overall error, and some have lower subgroup error.\n", + "A financial ceiling determines which designs are eligible. Among those\n", + "designs, lower cost, better overall accuracy and better subgroup accuracy\n", + "may favor different choices.\n", "\n", - "A design is **Pareto dominated** if another eligible design is no worse on\n", - "all three objectives and strictly better on at least one. The feasible Pareto\n", - "set supplies alternatives; preferences select among them.\n", + "A design is **Pareto dominated** when another eligible design is no worse\n", + "on all three outcomes and strictly better on at least one. The remaining\n", + "**Pareto alternatives** give us a set of trade-offs to consider.\n", "\n", - "The following three priorities are hypothetical scenarios, not estimates of\n", - "stakeholder values. Fixed reference ranges make dollars and percentage points\n", - "comparable. They are scaling anchors, not feasibility thresholds, and values\n", - "beyond them are not clipped." + "We will compare three priority scenarios: cost first, overall accuracy,\n", + "and subgroup accuracy. Fixed reference ranges put costs and errors on\n", + "comparable scales before applying the preference weights." ] }, { @@ -599,7 +589,7 @@ } ], "source": [ - "# Live-demo control: change the budget, then rerun this cell and the figures below.\n", + "# Budget and preference assumptions for this comparison.\n", "budget = BUDGET\n", "priorities = {\n", " \"Cost first\": {\n", @@ -680,20 +670,18 @@ } }, "source": [ - "### Read the trade-offs\n", + "### Reading the trade-offs\n", "\n", "- Blue circles use proportional allocation; orange diamonds oversample.\n", - "- Gray points exceed the financial ceiling; the dashed line shows that ceiling.\n", - "- Black outlines mark the Pareto set computed using **all three objectives**.\n", - " These panels are projections, not separate two-objective fronts.\n", - "- Labels identify the preference winners; they can change when inputs change.\n", - "- Vertical bars are approximately ±2 **Monte Carlo standard errors of estimated\n", - " MAE**. They describe simulation precision under this model, not uncertainty\n", - " in a real survey's prevalence estimate or in the model assumptions. They are\n", - " marginal bars, not simultaneous confidence guarantees after selection.\n", + "- Gray points exceed the budget, shown by the dashed line.\n", + "- Black outlines identify the feasible Pareto alternatives across all\n", + " three objectives. Each panel shows a different view of the same set.\n", + "- Design labels identify the choices favored by the three priority scenarios.\n", + "- Error bars show approximately two Monte Carlo standard errors of\n", + " estimated MAE: the precision of these simulation-based comparisons.\n", "\n", - "**Audience question:** Which alternative would you defend, and what information\n", - "would you need before committing resources?" + "**Which design would you choose, and what would you want to know before\n", + "committing resources?**" ] }, { @@ -705,15 +693,18 @@ } }, "source": [ - "## 6. Same evidence, different priorities\n", + "## 6. Different priorities, different choices\n", + "\n", + "The same results can support different choices. Here we give greater\n", + "emphasis to affordability, overall accuracy, or subgroup accuracy.\n", "\n", - "We can change the budget or preference weights using the already computed\n", - "results. This step calls `preference_sweep`, **not the simulator**.\n", + "`preference_sweep` compares these priorities using the results already\n", + "collected. Changing the budget or the preference weights is a new\n", + "decision analysis; it does not require another simulation run.\n", "\n", - "The full feasible Pareto set is shown below, sorted by cost. Rank 1 is best\n", - "within a priority scenario; ranks are calculated among all feasible designs.\n", - "Nearby point estimates can change order with additional simulation or different\n", - "assumptions, so a rank is not a statistical guarantee of superiority." + "The rows below are feasible Pareto designs, sorted by cost. Rank 1 is the\n", + "preferred design in a scenario. Ranks are calculated among all feasible\n", + "designs using their estimated mean outcomes." ] }, { @@ -840,19 +831,20 @@ "id": "0a44a734", "metadata": { "slideshow": { - "slide_type": "notes" + "slide_type": "fragment" } }, "source": [ - "### Two safe live changes\n", + "### What changes when resources or priorities change?\n", + "\n", + "Suppose the financial ceiling falls from **\\$40,000 to \\$30,000**.\n", + "Which designs remain available, and which trade-offs become harder?\n", "\n", - "1. Set `budget = 30_000` in the decision cell. Rerun that cell and the figures:\n", - " which previously attractive designs are now excluded?\n", - "2. Restore the budget and edit the weights in `priorities[\"Subgroup accuracy\"]`\n", - " in the decision cell. Rerun that cell and the figures below it.\n", + "Or suppose accurate measurement of the underserved group becomes a\n", + "higher priority. How much additional cost or overall error would we accept?\n", "\n", - "The results table remains unchanged. These scenarios expose value judgments;\n", - "they do not establish that one set of stakeholder priorities is correct." + "These are choices about what matters. Making the priorities explicit\n", + "helps explain why a recommendation changes." ] }, { @@ -864,22 +856,30 @@ } }, "source": [ - "## 7. What would a real project replace?\n", + "## 7. From an illustration to a real decision\n", + "\n", + "A substantive application would bring together:\n", "\n", - "- **Population model:** locally relevant prevalence, clustering, nonresponse,\n", - " sampling frames, and uncertainty in these assumptions.\n", - "- **Measurement model:** validated assay characteristics and their uncertainty.\n", - "- **Cost model:** context-specific financial and economic costs, including\n", - " participant and staff burden where appropriate.\n", - "- **Decision rules:** stakeholder-defined objectives, acceptable error, budget,\n", - " and field-capacity constraints.\n", + "- Locally relevant prevalence, clustering, sampling frames and nonresponse.\n", + "- Validated assay characteristics and uncertainty in their performance.\n", + "- Context-specific financial and economic costs, including participant\n", + " and staff burden where appropriate.\n", + "- Stakeholder-defined objectives, acceptable error, budgets and\n", + " field-capacity constraints.\n", "\n", - "Our target is the model's fixed group mean, not the realized mean of sampled\n", - "communities. Sampling communities is unbiased by construction; real selection\n", - "and nonresponse can introduce bias that more replication cannot remove.\n", + "The population, prices and preferences here are hypothetical. Our target\n", + "is antibody-status prevalence; interpreting protection requires further\n", + "evidence. Communities are sampled without selection bias in this model.\n", + "Real selection and nonresponse can introduce bias that more replication\n", + "cannot remove.\n", "\n", - "**Takeaway:** trade-study makes alternatives, evidence and priorities inspectable.\n", - "The domain model and the decision assumptions still need substantive expertise." + "Simulation error bars describe the precision of estimated MAE under the\n", + "stated assumptions. Uncertainty in an individual survey's prevalence,\n", + "uncertainty in the assumptions, and confidence in a selected design\n", + "require their own assessment.\n", + "\n", + "`trade-study` makes the alternatives, evidence and priorities inspectable.\n", + "Substantive expertise determines the model and the decision criteria." ] }, { @@ -893,13 +893,13 @@ "source": [ "## Discussion\n", "\n", - "What is the closest decision in your current work?\n", + "What decision in your work involves competing objectives like these?\n", "\n", - "- Comparing designs for surveillance or program evaluation?\n", - "- Comparing delivery strategies under cost and capacity constraints?\n", - "- Explaining a recommendation when several objectives matter?\n", + "- Choosing a surveillance or evaluation design.\n", + "- Comparing delivery strategies under cost and capacity constraints.\n", + "- Explaining a recommendation when several outcomes matter.\n", "\n", - "Which objective or constraint is missing from this demonstration?" + "**Which objective or constraint would you add to this comparison?**" ] }, { @@ -913,13 +913,20 @@ "source": [ "## Appendix: what drives financial cost?\n", "\n", - "The illustrative coefficients are $3,000 setup, $250/$700 per community visit,\n", - "and $30/$40 per participant, for other/underserved communities respectively.\n", - "They represent a hypothetical access-cost difference, not observed local prices.\n", + "| Cost component | Unit | Illustrative price |\n", + "|---|---|---:|\n", + "| Setup | Per survey | \\$3,000 |\n", + "| Community visit: other communities | Per community | \\$250 |\n", + "| Community visit: underserved communities | Per community | \\$700 |\n", + "| Participant: other communities | Per participant | \\$30 |\n", + "| Participant: underserved communities | Per participant | \\$40 |\n", + "\n", + "These illustrative prices represent a hypothetical access-cost difference.\n", + "The model adds setup, community-visit and participant costs. Participant\n", + "burden is not monetized here.\n", "\n", - "Fixed, community and participant costs are additive. Average cost per participant\n", - "and average cost per community are **not** added together as if they were\n", - "independent marginal costs. Participant burden is not monetized in this version." + "The selected designs use different combinations of participants,\n", + "communities and allocation, so their cost components differ." ] }, { @@ -964,11 +971,15 @@ } }, "source": [ - "## Appendix: inspect weighting and Monte Carlo precision\n", + "## Appendix: population weighting and simulation precision\n", + "\n", + "The population-weighted estimate uses the fixed 80:20 composition.\n", + "A sample-weighted average instead reflects the survey's allocation.\n", + "The first comparison below shows why those weights matter.\n", "\n", - "For the same group estimates, a sample-weighted average changes when allocation\n", - "changes. The population-weighted estimate uses the fixed 80:20 composition.\n", - "The comparison below demonstrates the estimand, not an accuracy claim from one draw." + "The second table compares Monte Carlo precision after 100 and 1,000\n", + "simulated surveys per design. Its standard errors describe uncertainty\n", + "in our estimates of design performance under the same population model." ] }, { @@ -1057,45 +1068,6 @@ ")" ] }, - { - "cell_type": "markdown", - "id": "5f4e667b", - "metadata": { - "slideshow": { - "slide_type": "slide" - } - }, - "source": [ - "## Appendix: export and present without execution\n", - "\n", - "The notebook stores tables and figures. Generate a standalone HTML presentation\n", - "from those saved outputs:\n", - "\n", - "```bash\n", - "uv run --extra notebook jupyter nbconvert --to html examples/serosurvey_study.ipynb\n", - "```\n", - "\n", - "For a fresh headless run, with a 60-second limit per cell:\n", - "\n", - "```bash\n", - "uv run --extra notebook jupyter nbconvert --execute --to notebook \\\n", - " --ExecutePreprocessor.timeout=60 --output serosurvey_executed.ipynb \\\n", - " --output-dir /tmp examples/serosurvey_study.ipynb\n", - "```\n", - "\n", - "Export results from a code cell if needed:\n", - "\n", - "```python\n", - "report.summary.to_dataframe(include_metadata=True).to_csv(\"serosurvey_designs.csv\", index=False)\n", - "```\n", - "\n", - "The companion script regenerates the documentation's PNG figures:\n", - "\n", - "```bash\n", - "uv run --extra notebook python examples/serosurvey_study.py\n", - "```" - ] - }, { "cell_type": "markdown", "id": "531c85ec", @@ -1105,27 +1077,18 @@ } }, "source": [ - "## Sources and audience connections\n", + "## References\n", "\n", - "- [IVAC's portfolio](https://publichealth.jhu.edu/ivac/projects): coverage/equity,\n", - " epidemiology, economics/finance, operations research and policy.\n", - "- [SISS](https://publichealth.jhu.edu/ivac/our-work/strengthening-immunization-systems-through-serosurveillance-siss)\n", - " and [Serosurvey Tools](https://serosurveytools.org/about/): design and use of serological surveillance.\n", - "- [Carcelen et al., 2020](https://pmc.ncbi.nlm.nih.gov/articles/PMC7561102/):\n", - " *How much does it cost to measure immunity?* This is motivation, not a reproduction.\n", - "- [Andrea Carcelen](https://publichealth.jhu.edu/faculty/4158/andrea-carcelen):\n", - " serosurveillance and reaching vulnerable populations.\n", - "- [Bryan Patenaude](https://publichealth.jhu.edu/faculty/3683/bryan-n-patenaude):\n", - " economic evaluation, financing and equity measurement.\n", - "- [Shaun Truelove](https://publichealth.jhu.edu/faculty/3998/shaun-truelove):\n", - " modeling to inform prevention and response.\n", - "- [Chizoba Wonodi](https://publichealth.jhu.edu/faculty/2206/chizoba-barbara-wonodi)\n", - " and [Molly Sauer](https://publichealth.jhu.edu/faculty/3466/molly-sauer):\n", - " immunization delivery, implementation and prioritization.\n", - "- [Svea Closser](https://publichealth.jhu.edu/faculty/3657/svea-closser):\n", - " health systems and the experiences of frontline workers.\n", + "- [Strengthening Immunization Systems through Serosurveillance (SISS)](https://publichealth.jhu.edu/ivac/our-work/strengthening-immunization-systems-through-serosurveillance-siss).\n", + " International Vaccine Access Center, Johns Hopkins Bloomberg School of Public Health.\n", + "- [Carcelen AC and colleagues (2020). *How much does it cost to measure immunity? A costing analysis of a measles and rubella serosurvey in southern Zambia.*](https://pmc.ncbi.nlm.nih.gov/articles/PMC7561102/)\n", + " PLOS ONE, 15(10), e0240734.\n", + "- [Serosurvey Tools](https://serosurveytools.org/about/).\n", + " Resources for the design, analysis and interpretation of serosurveys.\n", "\n", - "These connections informed the example's scope; they do not imply endorsement." + "The costing paper motivates separating study-, community- and\n", + "participant-level costs. The numerical assumptions in this illustration\n", + "were chosen independently of that study." ] } ],