Add JOSS paper for submission

[vahid-ahmadi]

Summary

• Adds paper.md and paper.bib for submission to the Journal of Open Source Software (JOSS)
• Paper covers all 6 required JOSS sections: Summary, Statement of Need, State of the Field, Software Design, Research Impact Statement, and AI Usage Disclosure
• Cites adoption by 10 Downing Street, HM Treasury (Algorithmic Transparency Record), NIESR UK Living Standards Review 2025, INET Oxford (Seventh Carbon Budget), IEA reports, and House of Lords parliamentary debate
• Funded by the Nuffield Foundation

What is JOSS?

JOSS is a peer-reviewed open-access journal for research software. It publishes short papers (750–1,750 words) describing open-source software with research impact. There are no submission fees. The review process happens on GitHub via openjournals/joss-reviews. See the submission guide and paper format for full details.

JOSS requirements checklist

• OSI-approved open-source license (AGPL-3.0)
• Public repository with browsable source code
• Issue tracker readable without registration
• Public development history > 6 months
• Demonstrated research impact (government, academic, policy citations)
• Good open-source practices (CI, tests, docs, PRs, tagged releases)
• Iterative development over time
• paper.md with all 6 required sections
• paper.bib with BibTeX references
• AI Usage Disclosure
• Funding acknowledgement (Nuffield Foundation)
• ORCID IDs for all authors

Review suggestions status

From the self-review comment:

• Add architecture diagram — architecture.png added with proper Figure 1 reference in text
• Fix placeholder ORCIDs — All 4 authors have real ORCIDs
• Add a minimal code example — No code snippet yet (QuantEcon-style)
• Add comparison table in State of the Field — Feature table vs TAXSIM/UKMOD/OpenFisca
• Add reproducibility framing to Statement of Need — Added: "Because existing proprietary models cannot be independently verified..."
• Add scope/limitations sentence — Added: "PolicyEngine models static fiscal impacts; it does not model macroeconomic feedback effects or general equilibrium responses"
• Condense policy research paragraph — Research Impact still ~60% of body
• Add foundational microsimulation references — No Orcutt 1957 or Bourguignon & Spadaro 2006 yet

Additional fixes applied

• Replaced fake pe_uk_data / pe_us_data repo citations with real Woodruff & Ghenis (2024) Enhanced CPS working paper
• Fixed Claude citation year 2025 → 2026
• Fixed broken Atlanta Fed Policy Rules Database URL → GitHub repo
• Moved architecture figure after intro paragraph (text-first, then diagram)
• Added explicit Figure 1 references in Software Design text
• Verified all 27 bib URLs resolve correctly
• Synced both HTML previews with paper.md

Steps before submission

• Complete the full name of all authors in paper.md
• ORCID IDs — All authors have real ORCIDs:
  • Max Ghenis — 0000-0002-1335-8277
  • Nikhil Woodruff — 0009-0009-5004-4910
  • Pavel Makarchuk — 0009-0003-4869-7409
  • Vahid Ahmadi — 0009-0004-1093-6272
• All co-authors confirm agreement to be listed
• Review and approve paper text
• Merge this PR to main
• Submit at https://joss.theoj.org/papers/new with the repo URL
• Disclose conflicts of interest on the submission form (e.g., all authors employed by PolicyEngine)
• Note related publications on the submission form (NIESR report, No 10 Innovation Fellows work, IEA reports)

After submission

1. A JOSS editor will open a pre-review issue at joss-reviews
2. Two reviewers will be assigned; review happens via GitHub issues
3. Respond to reviewer feedback within 2 weeks, complete changes within 4-6 weeks
4. Upon acceptance: create a tagged release, deposit to Zenodo for a DOI, update the review thread

Test plan

• Verify paper.md renders correctly with JOSS preview tools
• Confirm all paper.bib references are valid and resolve correctly

[vahid-ahmadi]

Redundancy review: repeated points in paper.md

Several points are stated multiple times across sections. Consider consolidating to stay within the JOSS word limit:

1. Four-layer architecture (country packages delegate to Core, policyengine sits above) — described in Summary (line 33), State of the Field (lines 47, 59), and Software Design (line 63). Three tellings of the same stack.
2. Reproducibility/certification boundary (runtime bundle pinning country-model version, microdata version, dataset artifact, provenance) — described in Summary (line 33), Statement of Need (line 39), and Software Design (lines 65, 71). The specific list of what gets pinned appears nearly verbatim in all three.
3. Baseline-versus-reform workflow — mentioned in Statement of Need (line 37, 39), State of the Field (line 47), and Software Design (line 63).
4. Country packages contain statutory logic / policyengine centralizes analysis — stated in Summary (line 33), State of the Field (lines 47, 59), and Software Design (lines 63, 69).
5. Household vs. population-level simulation (Simulation class for households, representative survey datasets with calibrated weights for population) — described in both Summary (line 33, implicitly) and Statement of Need (line 39).
6. Reusable output types (Aggregate, ChangeAggregate, IntraDecileImpact, economic_impact_analysis()) — mentioned in State of the Field (line 47) and Software Design (line 63).
7. Enhanced survey microdata / companion data repositories (CPS, FRS) — mentioned in State of the Field (line 59) and Software Design (line 63).
8. OpenFisca fork origin — mentioned in State of the Field (line 45) and Acknowledgements (line 128).

[vahid-ahmadi]

Updated the JOSS manuscript on joss-paper in commit 463b328.

This pass made three paper-only changes:

• Added an explicit conflict-of-interest disclosure in the Acknowledgements section noting that all authors are employed by PolicyEngine.
• Simplified the Summary for non-specialist readers by replacing the more technical reproducibility language with a plainer description of what the package does and who it is for.
• Compressed the Research Impact Statement by merging overlapping examples and trimming repeated context while keeping the main evidence categories and citations.

I also synced paper-preview.html with the updated paper.md.

[vahid-ahmadi]

@MaxGhenis — flagging for the readiness conversation.

End-to-end test pass on joss-paper (4.2.1)

Editable install in python313, then ran the headline pipelines and the full test suite. Summary of what works and what doesn't.

Works

• Imports / public API: pe.uk, pe.us, provenance, results, graph, tax_benefit_models.common all reachable.
• Household calculators (README + paper) — UK £50k earner: £7,486 income tax / £39,520 net. US $60k CA single: $5,020 fed tax / $48,007 net. Paper's verbatim US example (lines 74–89) reproduces: $40,702 → $42,634 after doubling SINGLE std deduction.
• UK microdata pipeline: ensure_datasets downloads enhanced FRS in ~26s; Simulation.run() ~7s; UC aggregate £82.18bn. PA reform £12.57k → £15k drops income tax by ~£21.8bn.
• examples/paper_repro_uk.py (after the fix in this push): 53,508 households, baseline Gini 0.3170 → reform Gini 0.3265, decile-1 mean change −£712.76, 10 programmes.
• Other UK examples (employment_income_variation_uk, income_bands_uk, policy_change_uk, household_impact_example, employment_income_variation_us): all run cleanly.
• pytest tests --no-cov: 429 passed, 0 failures, 15 min wall-clock. 84 warnings, all RuntimeWarning: invalid value encountered in divide from policyengine_us (QBI / labor-supply variables) — not from this repo.
• Paper API claims verified: flat-dict policy={...} on Simulation works (paper §Software Design); Aggregate, ChangeAggregate, IntraDecileImpact, economic_impact_analysis() all present.

Issues

1. examples/paper_repro_uk.py:73 referenced analysis.programme_statistics (UK spelling) but the API is analysis.program_statistics — script crashed at the final print. Fixed in 4b12ac0.
2. examples/us_budgetary_impact.py is OOM-killed on the simulation step: peak ~30 GB RSS on a Mac, killed by the OS (rc=137). Loads enhanced CPS 2024 fine (101k people, 41k households), dies running baseline + reform. Worth profiling before submission since this is the canonical US population example.
3. examples/income_distribution_us.py doesn't auto-fetch its dataset like the other examples — errors with "run create_datasets() first". Informative but inconsistent UX vs. the rest of the examples.
4. examples/us_budgetary_impact.py:38 prints the entire dataset object via print(f"Loaded: {dataset}") — dumps every dataframe to stdout. Cosmetic.

Bottom line

Package itself is in solid shape — tests green, all the headline pipelines work, paper API claims hold. The remaining blockers are concentrated in the US population example (issue 2 above). Everything else is polish.

[anth-volk]

I ran a profiled duplicate of examples/us_budgetary_impact.py to get a clearer picture of the US example's runtime and resource behavior.

Summary of the run:

• The instrumentation smoke test passed before running the full example.
• The full US budgetary impact run completed the dataset load, baseline simulation, reform simulation, and quick aggregate outputs.
• Dataset loaded successfully for 2026, including roughly 101k people, 55k tax units, and 41k households.
• baseline_sim.run() took 546.6s wall time and 357.4s process CPU.
• reform_sim.run() took 596.5s wall time and 337.3s process CPU.
• Total instrumented runtime was 1,307.3s wall time, about 21m 47s, with 767.1s process CPU.
• Quick aggregate outputs were produced:
  • Tax revenue change: -$101.90B
  • Winners: 45.73M households
  • Losers: 0.35M households
• The later economic_impact_analysis(baseline_sim, reform_sim) step failed with a StopIteration from src/policyengine/outputs/aggregate.py, while looking up a variable in simulation.tax_benefit_model_version.variables.

The memory behavior was the main concern:

• Peak resident memory was 7.48 GiB RSS.
• Peak virtual memory was 82.35 GiB VMS.
• baseline_sim.run() pushed memory into the multi-GiB range and ended around 3.3 GiB RSS / 57.4 GiB VMS.
• reform_sim.run() then produced the largest spike. VSCode crashed while the run was in reform_sim.run(), shortly after memory had climbed past roughly 6 GiB RSS and 64 GiB VMS. The process continued long enough to finish logging afterward, and VMS eventually climbed to about 82 GiB.
• Even after RSS dropped, VMS stayed very high, which is consistent with the local machine becoming sluggish due to memory pressure, compression, and swap activity.

This strongly supports the concern raised above that the US country package / examples are effectively too memory-heavy for normal local test runs. The UK package may be practical to exercise locally, but the US run is heavy enough that it can destabilize a developer workstation, at least on my machine.

[anth-volk]

@vahid-ahmadi one follow-up from the instrumented US budgetary-impact run: the final StopIteration looks like a real analysis-layer bug, separate from the VSCode crash/resource-pressure issue.

The run completed the expensive baseline and reform simulations and produced the quick outputs:

• Tax revenue change: -$101.90B
• Winners: 45.73M households
• Losers: 0.35M households

The failure happened later inside economic_impact_analysis, specifically in ProgramStatistics -> Aggregate.run(), where Aggregate does a bare next(...) lookup over simulation.tax_benefit_model_version.variables. The likely immediate mismatch is the hard-coded US program entry:

"payroll_tax": {"entity": "person", "is_tax": True}

The US model/output path appears to expose/materialize employee_payroll_tax instead, not a plain payroll_tax. There are likely other fragile entries in the same list too, e.g. medicare looks like it may not correspond to a plain aggregate variable in the US package.

So my read is: if this lookup mismatch were fixed or skipped, the code probably would have continued past this point, albeit after a very resource-intensive run. But simply suppressing StopIteration would not be correct, because the analysis still needs a valid variable object and matching output column/entity mapping.

More generally, this is an inherently fragile pattern for the US package given the velocity of policy/model variable updates: hard-coded program-statistics variable names in policyengine.py can drift from policyengine-us variable names or from the subset of variables actually materialized into the output dataset. This should probably either validate the configured program list up front with a clear error, derive the list from model metadata, or use an explicit country-versioned mapping that is tested against current US outputs.