Difference-in-Differences (DiD) quantitative framework
A highly effective statistical evaluation technique.
In the context of natural disasters, DiD technique can serve a dual purpose: assessing the impact of the disaster itself and evaluating the effectiveness of disaster relief/support programs during the recovery phase.
For the first purpose, this method evaluates the impact of disasters by comparing outcome changes over time between the group affected by disasters and the group unaffected by them. Likewise, to evaluate the effectiveness of disaster relief or support programs for an affected group, DiD will compare the trajectory of the outcomes before and after receiving the support programs between the group that received the support and its counterpart, given that both groups were affected by disasters.
Treatment group, control group, and event time
Essential to the design and implementation of a DiD analysis are the treatment group, control group, and event time. Consider the following diagram for a better understanding of these elements.
The DiD statistical method
Click the image to open the PDF in fullscreen
In terms of disaster impacts, the treatment group includes those affected, marked by the solid red line in the diagram above, while the control group, represented by the continuous blue line, remains unaffected.
In the context of disaster recovery, the treatment group, comprising those affected by disasters and receiving support programs, is indicated by a solid red line, while the control group consists of those affected by disasters without access to support programs and is depicted by the solid blue line.
The orange vertical line or the event time serves the purpose of marking either the time of occurrence of a disaster (for measuring the impact of a disaster), or the time of the introduction of disaster relief or disaster support programs that we want to evaluate for their effectiveness.
The diagram shows the control and treatment groups both before (on the left side of the event time) and after the event (on the right side of the event time).
Without intervention, both the treatment and control group will follow the same trajectory. The pre-event trajectory or the trend, as shown in the diagram is the same for both groups. If the intervention were not implemented or the treatment had not happened, the treatment group would continue its pre-event trend, illustrated by the dashed line. Consequently, the deviation between the assumed trend (dashed line) and the actual trend (solid line) for the treatment group is attributed to the effect of the intervention or treatment.
How to calculate the effect
Suppose that you observe two data points pre and post treatment for both the control and treatment group. In the diagram above, the outcomes for the control pre and post treatment are denoted by Yc1 and Yc2 (the subscript c denotes for control while the subscripts 1 and 2 denote the two points of time). Similarly, Yt1 and Yt2 denote for outcomes of treatment group pre and post treatment respectively.
We now take the change in outcome of the control as a reference. In other words, in the absence of a disaster strike or the support program, the change in the outcome is (Yc2 – Yc1). Likewise, if the treatment had not received the intervention, the change in the outcome would be (Yt2’ – Yt1).
As the treatment and control trend similarly these two changes would be equal in the absence of the intervention i.e., (Yt2’ – Yt1) – (Yc2 – Yc1) = 0. Thus, the difference between the observed change of the treatment group compared with the control is attributed to the treatment effect i.e., (Yt2 – Yt1) – (Yc2 – Yc1).
The term “difference-in-differences” becomes clear when you consider it in this context.
Evaluating DiD’s core premise
It’s crucial to bear in mind that the effectiveness of DiD relies on the validity of its underlying assumption. As mentioned earlier, this assumption asserts that the control and treatment groups would exhibit the same trajectory in the absence of the intervention (disaster strike or support programs) or, in a testable manner, share a similar trajectory before the event occurs. However, this assumption often falters in observed data due to numerous confounding factors influencing the outcomes’ evolution for these two groups, leading to disparate trajectories.
For instance, consider assessing the impact of the latest flood on individuals’ income using DiD, comparing the income of the treatment group (residents in the affected area) with that of the control group (residents outside the affected area). If the affected area is inherently more susceptible to floods, it’s plausible that individuals’ income in the affected region has already decreased before the recent flood due to the lasting impact of past floods. Similarly, a similar pattern may emerge if the flood-affected area has experienced a negative shock, causing a decline in the region’s economy and individuals’ income even before the studied flood.
Event study
The technique commonly employed for verifying the parallel trend assumption is known as the ‘event study’. It involves utilising econometric modelling to test if the parallel trend assumption holds and to fix or to minimise the divergence of the trend between treatment and control groups if any.
While we don’t cover the details of event study techniques in this course, participating in the following task can improve your comprehension of the method.
Reflect and share
The event study method examines how a specific event, not limited to natural disasters, affects a chosen outcome by analysing patterns before and after the event.
Reflect on a significant turning point in your life – decisions, achievements, or challenges. Analyse the periods before and after this event, taking into account its effects on emotions, relationships, and personal growth. Examples of such significant events, aside from disasters, could include participating in a job training program, enrolling in a new course, experiencing job loss, undergoing hospitalisation, or going through a divorce.
Now, compare your life before and after the turning point. Are there noticeable differences or trends? How do you interpret the before and after phases of the event? Keep in mind that outcomes may not always translate easily into numerical data.
Feel free to share your thoughts in the comment section.
Note: We simplified the explanations for easier comprehension of the DiD method. However, it’s crucial to acknowledge that DiD calculations and analysis often involve the use of complex econometric models.
Natural Disaster Recovery and Management
Reach your personal and professional goals
Unlock access to hundreds of expert online courses and degrees from top universities and educators to gain accredited qualifications and professional CV-building certificates.
Join over 18 million learners to launch, switch or build upon your career, all at your own pace, across a wide range of topic areas.
