Experimental and Quasi-experimental Variations
Chicago Housing Voucher Experiment
In their 2015 study, economists Brian Jacob, Max Kapustin, and Jens Ludwig examine the effect on crime of a housing voucher lottery in Chicago in 1997.
As in many other U.S. cities, Chicago offers housing vouchers to income-eligible households, but the number of eligible applicants has been much larger than the number of available vouchers and its housing voucher waiting list was officially closed off in 1985.
Twelve years later, in July 1997, the Chicago Housing Authority opened its waiting list once again, and within a month received 82,607 applications from eligible low-income households. As usual, the housing authority could not provide housing assistance to all eligible households and decided to run a randomized lottery to determine who would receive the assistance.
All applicants were given a random number between 1 and 82,607, and were told that housing vouchers would be given to those in the top 35,000 positions only.
In this case, the randomization makes it a lot easier to estimate the causal effect of a housing voucher on criminal participation. We can simply compare the offending rate between those who win the lottery and those who do not, and view the difference as the causal effect of the housing voucher on crime. Because of the randomization, there should be minimal difference between lottery winners and losers on average, in terms of individual characteristics that may be relevant to their criminal decisions. Any substantial difference in their criminal behavior can then be attributed to the causal effect of whether they won or lost the lottery.
After comparing the offending rates of children of lottery winners and losers (14 years after the randomization), Jacob, Kapustin, and Ludwig conclude that there was no significant difference in criminal outcomes of children who won the housing vouchers and those who did not.
This is pretty disappointing, given the generosity of the housing voucher program. On average, the value of a housing voucher was about $12,000, which was more than 60 percent of an average applicant’s annual income!
As we have seen, an experiment in which a government support program is randomly allocated among similarly eligible applicants enables researchers to recover the causal effect of the program in a straightforward way. (Let’s call the variation in housing assistance from this randomized experiment an “experimental” variation.) However, it would be difficult to run such experiments for most government programs. Just imagine what would happen if government suddenly begins to provide cash and food assistance for a small number of randomly chosen low-income families only.
But researchers can still learn a great deal about the causal effect of a government program without running a randomized experiment. Consider the following example. While all eligible American households can apply for the same government welfare programs, actual implementation of the programs are usually done by state governments. As a result, the timing of welfare transfers often differs across different states. Some states make their welfare transfers once in a month in the beginning of each calendar month, others in the middle of a month, and some others make staggered and/or biweekly payments.
Economist Fritz Foley predicted that the timing of welfare transfers would be closely related to crime rates. In states where welfare transfers are made once in the beginning of a month, welfare recipients may find themselves out of cash and benefits near the end of the month. They then may look for additional income from criminal activities, causing crime rates in these states to increase near the end of a calendar month. Similarly, crime rates would be higher near the middle of each month in states where the transfers are made in the middle of the month. He tested this hypothesis using data on welfare transfer timing and crime rates from multiple U.S. cities, and found the results in support of his prediction. This finding suggests that government welfare transfers reduce recipients’ criminal risk.
Importantly, it is unlikely that the variation in the timing of welfare transfers across different states was driven by local crime rates and other crime-relevant factors (such as age structure, gender ratio, police force size, etc.). If states with different welfare transfer timings have similar observable characteristics on average, the difference in their crime rates can be credibly viewed as the causal effect of the welfare transfer on crime.
In this sense, this temporal variation in the timing of welfare transfers across states closely resembles a variation from a hypothetical randomized experiment, in which a researcher asks a randomly chosen group of state governments to make welfare transfers in the beginning of a month and others in the middle of a month. Economists often use such variations (called a “quasi-experimental variation”) to empirically investigate a causal relationship of relationship without having to run a randomized experiment. We will see many more examples in later weeks.
- Foley, C. Fritz. “Welfare Payments and Crime.” Review of Economics and Statistics 93.1 (2011): 97-112.
- Jacob, Brian A., Max Kapustin, and Jens Ludwig. “The Impact of Housing Assistance on Child Outcomes: Evidence from a Randomized Housing Lottery.” Quarterly Journal of Economics 130.1 (2015): 465-506
© Songman Kang, Hanyang University