The Utility of AI Tools in Auditing Adherence to Pre-Analysis Plans -- by Jeffrey Clemens, Anwita Mahajan
EconomicsData ScienceHigher Education
THE AI ANGLE
Auditing adherence to pre-analysis plansResearchers Jeffrey Clemens and Anwita Mahajan show that large language models can systematically audit adherence to pre-analysis plans by identifying precommitted design choices, evaluating deviations, and diagnosing specification gaps. While these AI tools substantially reduce the human labor required for authors and reviewers to verify empirical reproducibility, output variability across different models highlights the continued necessity of human judgment. This offers economics and data science faculty an emerging methodology for scaling research transparency while reinforcing the need for methodological scrutiny.
THE TEACHING ANGLE
Students can examine the pedagogical tension between using scalable AI audits to enforce research reproducibility and relying on human judgment to manage inconsistent outputs across different models.Read the original at nber.org Generate teaching or study materials
More in Economics
- Europe must build own AI or risk getting cut off by US or China, says ECB’s LagardeThe Guardian — AI · September 15, 2026
- AI’s best coding agent fails 60% of the time — and the data backs it upThe New Stack · September 15, 2026
- AI, Redistribution, and the Size of the PieMarginal Revolution · September 15, 2026
- A costly mistake? Report claims a third of employees fired due to AI will need to be rehired in the next few yearsTechRadar · September 15, 2026
- Exclusive: Paying for frontier AI models buys 4-month head start at 5x the costArs Technica · September 15, 2026