Skip main navigation

Angrist and Pischke discuss how the credility of econometrics has improved

In this article, Joshua Angrist and Jörn-Steffen Pischke discuss how credibility in econometrics research has improved due to better research designs.
© Center for Effective Global Action
Here, I’d like to dive a little deeper into Joshua Angrist and Jörn-Steffen Pischke’s article “The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics,” which I discussed in the last video. In the article, Angrist and Pischke explore Ed Leamer’s view of econometrics as explained in his well-known 1983 paper “Lets take the con out of econometrics” and seek to determine “whether the quality and the credibility of empirical work has increased since Leamer’s pessimistic assessment.”
In 1983, Leamer argued that contemporary empirical work “suffer[ed] from a distressing lack of robustness to changes in key assumptions – assumptions he called ‘whimsical’ because one seemed as good as another.” The ideal research design, he believed, was randomized trials and the “best way to use nonexperimental data to get closer to the experimental ideal [was] to explore the fragility of nonexperimental estimates.” However, he didn’t actually advocate searching for credible natural experiments or even performing randomized trials, and none of the central figures in his debate had much to say about research design.
To confront the “whimsical nature” of the key assumptions made in regression analysis, Leamer proposed that a process of sensitivity analysis should be implemented, featuring an “explicitly Bayesian agenda” and “extreme bounds analysis” — an estimation of regressions using different sets of covariates as controls.
Angrist and Pischke argue that empirical econometric work has vastly improved since Leamer’s article was published in 1983 and that “a clear-eyed focus on research design is at the heart of the credibility revolution in empirical economics.” They list other key factors contributing to this improvement, including the availability of more and better data and advances in theoretical econometric understanding, as well as sensitivity analysis for which Leamer advocated. They offer explanations for the improvements in empirical work today:
“Better and more robust estimation methods are part of the story, as is a reduced emphasis on econometric considerations that are not central to a causal interpretation of the main findings. But the primary force driving the credibility revolution has been a rigorous push for better and more clearly articulated research designs.”
They also state that a “well-done observational study can be more credible and persuasive than a poorly executed randomized trial.”
Further expressing the importance of research design, Angrist and Pischke argue that “good designs have a beneficial side effect: they typically lend themselves to a simple explanation of empirical methods and a straightforward presentation of results.” They go on to say that they “find the empirical results generated by a good research design more compelling than the conclusions derived from a good theory…”
In response to this “revolution” in credibility and research designs, concerns have arisen, especially with regards to external validity — whether or not the evidence from an experiment is applicable to situations beyond those of the original study. Angrist and Pischke respond to such concerns saying that “empirical evidence on any given causal effect is always local, derived from a particular time, place, and research design” and that a constructive response would be to “look for more evidence, so that a more general picture begins to emerge.”
In conclusion, Angrist and Pischke argue that improvement in empirical work in econometrics is not due to the use of extreme bounds sensitivity analysis, as Leamer suggests, but due to better research design and “careful implementation of quasi-experimental methods.” The progress that has been made in this field has produced a “credibility revolution” in fields as varied as labor, public finance, and development economics.
If you want to dive deeper into the material, you can read the entirety of each paper by clicking on the links in the SEE ALSO section at the bottom of this page.
Angrist, Joshua D, and Jörn-Steffen Pischke. 2010. “The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics.” Journal of Economic Perspectives 24 (2): 3–30. doi:10.1257/jep.24.2.3.
© Center for Effective Global Action
This article is from the free online

Transparent and Open Social Science Research

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education