Skip to 0 minutes and 1 secondBy now, we have seen examples of randomized control trials that show how some types of policy interventions may be or may not be very effective in lowering crime. The findings from randomized control trials usually have very strong causal interpretations, compared to studies based on non-experimental data. For example, suppose we compare crime rates in highly patrolled neighborhoods with crime rates in neighborhoods that do not have a lot of police presence. If these neighborhoods differ in other aspects, such as their economic and demographic characteristics, then the difference in their crime rates may be driven by the difference in the intensity of police patrol, or it may be driven by the difference in these other characteristics.
Skip to 0 minutes and 50 secondsOn the other hand, in the randomized control trials we saw earlier, police randomly chose neighborhoods to receive unusually high levels of police presence. In this case, the neighborhoods chosen to receive more police presence closely resemble the neighborhoods that did not receive more police presence, and the difference in their crime rates can be plausibly attributed to the difference in the level of police presence. Likewise, when we compare the criminal behavior of individuals living in high poverty and low poverty neighborhoods, we may find a very large difference in their crime rates. But we cannot easily conclude that this difference is caused by the difference in the neighborhoods that they live in.
Skip to 1 minute and 37 secondsAnd that’s because people living in high-poverty and low-poverty neighborhoods probably differ in many other aspects. But when a government policy provides a randomly chosen group of low-income households with housing vouchers that they can use to move to low poverty neighborhoods, the difference in the crime rates between low-income households that received the housing vouchers and those that did not receive the voucher can be plausibly attributed to the difference in their residential locations. But, studies based on random experiments are not without limitations. First, the findings from these studies may not be easily extended to the general population.
Skip to 2 minutes and 20 secondsFor example, the MTO experiment result we saw earlier shows that the opportunity to relocate to low-poverty neighborhoods lowers the probability of committing crime among the female program participants. But if government actually implements a housing support program that enables all low-income households to move to low poverty neighborhoods, the result of this housing program may be very different from the result from the MTO experiment. And that’s because, when a large number of low-income households use housing vouchers to move to new locations, there will be big changes to the local housing market and the spatial distribution of low-income and high-income households across neighborhoods.
Skip to 3 minutes and 7 secondsThis change in the housing market can have a large impact on city crime rates by itself, but of course, this possibility was not captured in the MTO experiment, which only covered a fraction of low-income households in the city. Second, the causal interpretation of a random experiment may be compromised if the experiment leads to a change in sample composition. For example, suppose a local government offers some sort of cognitive and behavioral support programs to a randomly chosen group of high schools. A simple comparison of delinquency rates between high schools that participate in this support program and high schools that do not participate in the program may not give us the causal effect of this support program on delinquency.
Skip to 3 minutes and 58 secondsA potential problem is that high schools that receive this program may attract different types of incoming students than high schools that do not receive this support program. And we may expect that parents who care a lot about their children’s educational experience will want to send their children to high schools that participate in this support program and children from these households may be less likely to commit delinquency in the first place. Then, when we compare the rates of delinquency between the high schools that received the support program and high schools that do not received the support program, the first group should have lower delinquency rates even if the support program actually had no impact on delinquency.
Skip to 4 minutes and 44 secondsAnd the results from this experiment can tell us no longer the causal effect of the support program on delinquency.
Strengths and limitations of randomized control trials
How do findings from non-experimental data and randomized control trials complement each other?
What are some potential problems that researchers may face when applying their findings from RCTs to real-life policy decisions?
© Songman Kang, Hanyang University