Skip main navigation

Strengths and limitations of randomized control trials

Strengths and limitations of randomized control trials
By now, we have seen examples of randomized control trials that show how some types of policy interventions may be or may not be very effective in lowering crime. The findings from randomized control trials usually have very strong causal interpretations, compared to studies based on non-experimental data. For example, suppose we compare crime rates in highly patrolled neighborhoods with crime rates in neighborhoods that do not have a lot of police presence. If these neighborhoods differ in other aspects, such as their economic and demographic characteristics, then the difference in their crime rates may be driven by the difference in the intensity of police patrol, or it may be driven by the difference in these other characteristics.
On the other hand, in the randomized control trials we saw earlier, police randomly chose neighborhoods to receive unusually high levels of police presence. In this case, the neighborhoods chosen to receive more police presence closely resemble the neighborhoods that did not receive more police presence, and the difference in their crime rates can be plausibly attributed to the difference in the level of police presence. Likewise, when we compare the criminal behavior of individuals living in high poverty and low poverty neighborhoods, we may find a very large difference in their crime rates. But we cannot easily conclude that this difference is caused by the difference in the neighborhoods that they live in.
And that’s because people living in high-poverty and low-poverty neighborhoods probably differ in many other aspects. But when a government policy provides a randomly chosen group of low-income households with housing vouchers that they can use to move to low poverty neighborhoods, the difference in the crime rates between low-income households that received the housing vouchers and those that did not receive the voucher can be plausibly attributed to the difference in their residential locations. But, studies based on random experiments are not without limitations. First, the findings from these studies may not be easily extended to the general population.
For example, the MTO experiment result we saw earlier shows that the opportunity to relocate to low-poverty neighborhoods lowers the probability of committing crime among the female program participants. But if government actually implements a housing support program that enables all low-income households to move to low poverty neighborhoods, the result of this housing program may be very different from the result from the MTO experiment. And that’s because, when a large number of low-income households use housing vouchers to move to new locations, there will be big changes to the local housing market and the spatial distribution of low-income and high-income households across neighborhoods.
This change in the housing market can have a large impact on city crime rates by itself, but of course, this possibility was not captured in the MTO experiment, which only covered a fraction of low-income households in the city. Second, the causal interpretation of a random experiment may be compromised if the experiment leads to a change in sample composition. For example, suppose a local government offers some sort of cognitive and behavioral support programs to a randomly chosen group of high schools. A simple comparison of delinquency rates between high schools that participate in this support program and high schools that do not participate in the program may not give us the causal effect of this support program on delinquency.
A potential problem is that high schools that receive this program may attract different types of incoming students than high schools that do not receive this support program. And we may expect that parents who care a lot about their children’s educational experience will want to send their children to high schools that participate in this support program and children from these households may be less likely to commit delinquency in the first place. Then, when we compare the rates of delinquency between the high schools that received the support program and high schools that do not received the support program, the first group should have lower delinquency rates even if the support program actually had no impact on delinquency.
And the results from this experiment can tell us no longer the causal effect of the support program on delinquency.
How do findings from non-experimental data and randomized control trials complement each other?
What are some potential problems that researchers may face when applying their findings from RCTs to real-life policy decisions?
This article is from the free online

Economics of Crime

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education