Want to keep learning?

This content is taken from the University of California, Berkeley, Center for Effective Global Action (CEGA) & Berkeley Initiative for Transparency in the Social Sciences (BITSS)'s online course, Transparent and Open Social Science Research. Join the course to learn more.

Skip to 0 minutes and 1 second It’s actually easier to illustrate some of these ideas with a specific example. So, I’m going to talk about meta-analysis of studies in a climate of conflict literature. Which is a very interdisciplinary literature. So, maybe we’ll touch on the interest of many people in this room. It is a long tradition of studying this specific question of how climate or natural resources and the environment more broadly affect violent human conflict. These sorts of, you know, ideas go back centuries. People have been talking about them for a long time.

Skip to 0 minutes and 32 seconds And what’s been new, relatively new in the last decade there’s a growing quantitative literature to understand how things like droughts or floods or high temperatures or basic extreme climatic conditions affect human societies and in particular violence in human societies. The goal of this meta-analysis that Sol Hsiang, Marshall Burke and I worked on was to try to understand is there a coherent message that comes out of this literature? Now that there’s all these studies, can we see anything meaningful if we take all the studies together? As we sort of dove into this literature and we’re really interested in the big picture question. We were trying to find out all, discover all the studies as we could. There were different timeframes.

Skip to 1 minute and 13 seconds They were using different data sets. There were lots of different methods. and I think it’s interesting making the contrast with the minimum-wage literature. The minimum-wage literature, let’s say, focused on the U.S. or rich countries for the most part. You kind of know what the outcome is. It’s going to be like some measure of unemployment. Explanatory variable is going to be like the minimum wage. You know, there’s some basic stuff that’s set. This literature in some ways was much less well-defined. There’s temperature studies that look at high temperature. There’s studies that look at extreme rainfall. There’s studies that look at El Niño, this big weather phenomenon. Okay, how do we kind of put these together? And then there’s different outcomes.

Skip to 1 minute and 49 seconds There’s many forms of conflict. There’s civil war, there’s violent crime. There’s all kinds of studies that are in this broad umbrella of violent human conflict. We had to make a decision about what sorts of approaches we found credible and which ones we didn’t. So, for instance, the studies that have longitudinal data but don’t include location, fixed effects or time controls at all, we felt like that’s not the appropriate way to use longitudinal data. So, we made a decision and we said, “You know what, we’re going to impose a certain methodological standard.

Skip to 2 minutes and 22 seconds If you can justify it, if you can really make the case, that methodologically there’s an appropriate approach to analyzing the data which we try to do in this paper. It helps you sort of undo some of the specification-searching and data-mining that might have existed in some of the original studies. There were three types of studies. Some of which we were able to sort of combine more easily than others in our analysis. There are these long-term historical studies, we do not include these in the meta-analysis. Like the data is really different, they’re from totally different time period. These are studies that use things like tree rings or sediment deposits. The second set of studies were laboratory studies. These were really fun.

Skip to 2 minutes and 59 seconds You know, there’s lab experiments where police trainees are much more violent in a hot training room than a cool training room. And then there’s some other really awful experiments where psychologists in Arizona on hot days in the summer time just wouldn’t move at green lights and they just – And they basically just recorded how crazy people in the cars behind them went. And they went nuts on hot days. But on cool days, they only went kind of nuts. The main literature we focused on are observational studies with longitudinal data or panel data. There’s this very natural distinction between interpersonal studies, interpersonal violence like crime, violent crime, person-to-person and organized violence or intergroup violence.

Skip to 3 minutes and 41 seconds So, like a civil war is a different outcome than violent assault or murder. But we felt like the interpersonal studies have some commonality like assault and murder and whatnot have much more in common with each other. And things like civil war, insurgency, conflict, even rioting, they were more common to each other. Now, there’s some in the middle, maybe rioting is in the middle. Maybe there’s others that are hard to classify but we made this distinction. There’s also some differences in the population study. Most of the intergroup violence studies are poor countries studies. They’re like civil war in Africa, Hindu-Muslim rioting in India, things like that.

Skip to 4 minutes and 20 seconds And most of the interpersonal violence is like really good high quality, high frequency data from rich countries. So, they just felt like two different sets of studies and that’s how we divided them up. But you might disagree with that, you might want to pool them. You might also want to divide these even more. So, these are the kinds of debates that are always going to ensue when you start combining literatures. What we put together here is some sense of where these studies fall, like what do they look like? So, the left panel here is the time period of the study, in a kind of reverse log scale. Tons of data go back to the post-war period, basically.

Skip to 4 minutes and 56 seconds That’s where you see a lot of data. A lot of the studies are basically, post-war period but then there’s a bunch of these historical studies that go way back. The right-hand panel here shows the scales, geographic scales and time scales. A ton of these really are country-year observations. A bunch of these are province-year, pixel-year, municipality-year. This is a lot of mass of studies that get annual data of a national or sub-national data and try to look for relationships. There’s also a fair number of studies and these are more of the longer-term studies, again, that go way out to the centuries or whatnot. These are the tree ring studies or the sediment deposits at the national scale.

Skip to 5 minutes and 35 seconds But, you know, there’s some variability like this Vrij et al. study was the lab experiments. So, some of these are just literally variation at the hourly level and then some are global. This is a global El Niño study by Sol as well. So, there’s a lot of range in terms of the types of studies and we started of finding all these patterns. Like 24 of these studies focus on temperature as the key climate variable. In 24 out of 24, higher temperature the point estimate is positive. Higher temperature is associated with more violence. We’re like, “Okay. That seems pretty unlikely to happen by chance.” In the precipitation studies, not all of them but almost all of them give this result.

Skip to 6 minutes and 13 seconds That more extreme precipitation is associated with more violence. Okay, that seems to go in the right direction. Three quarters of the individual studies are statistically significant. So, again, like gosh, you know, the bulk of the evidence seems to point towards extreme climate leading to more conflict. One of the figures we put together early on, this is, you know, using African data, this is just to say that whether we look at villages – this is a study looking across Tanzanian villages. We look at the pixel level in East Africa. And actually this exact area is that pixel, so we line those up.

Skip to 6 minutes and 47 seconds If we go from the pixel level to all of Africa and then we go up to the globe, you basically see this upward sloping relationship, more extreme climate, more conflict. These are just a few of the studies but there’s just dozens of these. So, in the paper, we have all these plots and we say, “We’re going to look across all these estimates.” This is before the meta-analysis. This is what we did before the meta-analysis. We said like, “Look at the data.” More extreme temperature moving to the right. More conflict that holds for, you know – this one up here is violent crime in the U.S. on the top left. This is rape in the U.S.

Skip to 7 minutes and 23 seconds This is retaliation in baseball games on hot days. This is Hindu-Muslim rioting in India. So, it’s pretty much, almost all of them you see this upward sloping relationship. This may seem kind of compelling but we may want more than pictures, even these beautiful pictures that Sol put together.

Skip to 7 minutes and 44 seconds What is the average effect? Give me a number. I’m working for the U.N. I’m trying to plan for climate change. It’s going to warm by 2 degrees Celsius, maybe 3 degrees Celsius. Give me a number, how much will violence increase? We couldn’t do that just from the pictures. We need to combine these estimates in a more systematic way. How much uncertainty is there? So, you have a bunch of pictures but maybe the standard errors are really big and maybe we shouldn’t have much confidence in these results. We want to understand uncertainty. We may want to quantify heterogeneity or understand the source of heterogeneity. That was sort of like some of the comments before.

Skip to 8 minutes and 16 seconds Maybe we can do better than just looking at pictures, we can do things systematically. And then we wanted to look at publication bias. We wanted to get back to this initial question in the literature. Like, is there so much publication bias that we just don’t believe the literature fundamentally? So, yes, we can come up with a meta-analysis estimate but if we think the whole literature is riddled with publication bias, maybe we don’t put much weight on it. So, this is really the goal of the next step is to take all those estimates and build something that’s a little firmer. First thing, to combine estimates across studies, we need to have comparable units.

Skip to 8 minutes and 51 seconds This was a big challenge in the climate and conflict literature. No one that we knew of had come up with a way to standardize measures. There’s actually a straightforward way to do it which is what we ended up doing. We consider all the climate shocks as deviations from the local mean and we just put them in standard deviation units. We say like, “Hey, you know, if you’re a degree Celsius warmer in tropical Africa, that’s like one-and-a-half standard deviations. Maybe it’s only one standard deviation in India. But we’re going to basically use the local distribution as the relevant measure of shocks. Now, that is the relevant measure of the shocks that populations face there.

Skip to 9 minutes and 31 seconds You might complain and say, “Well, that means you’re sort of studying small temperature shocks in Africa and large temperature shocks in Northern Latitudes.” So, you might say, “Well, maybe you’re really not comparing apples to apples anymore.” So you can do all this analysis for temperature and we have done some of this, but this is actually a very natural thing to consider. Maybe in the long run, you really get adapted to your local distribution of climate shocks. And a two standard deviation shock, that’s meaningful for you, yeah, that temperature variation may not be meaningful for others. But anyway, this provided a very simple standardization of the climate variables.

Skip to 10 minutes and 7 seconds Second, we then represent or we display the effects as percentage changes with respect to the local mean of the outcome variable. So, now all of a sudden, instead of saying, “You know, temperature shock of one degree increases civil war by eight percentage points and rainfall in India does something else.” We can say, “A one standard deviation climate shock affects this conflict outcome by 12 percent.” And we have that sort of potential for comparability. And I would say that even if it’s imperfect, at least it allows for some comparability, some ability to compare.

A meta-analysis example: "Quantifying the Influence of Climate on Human Conflict" Part I

In 2013, Professor Miguel and two of his colleagues, Solomon Hsiang and Marshall Burke, conducted a meta-analysis of studies examining the impacts of climate and environmental variability on violent human conflict. There was no shortage of studies to include, but the abundance of literature, as well as the interdisciplinary nature of the field and the broad umbrella under which impacts fell, made the exclusion of some studies and standardization across other necessary. For example, they omitted long-term historical studies and reanalyzed primary data to make comparisons between them more feasible. Their analysis revealed a striking pattern across multiple types of environmental variation, regions, and geographical scales.

Read our full paper on BITSS.org.


Hsiang, Solomon M., Marshall Burke, and Edward Miguel. 2013. “Quantifying the Influence of Climate on Human Conflict.” Science 341 (6151): 1235367. doi:10.1126/science.1235367.

Share this video:

This video is from the free online course:

Transparent and Open Social Science Research

University of California, Berkeley