Back to Blog

No Control Group? No Problem. An Intro to Quasi-Experimental Methods

By Cedric Mwesigwa · 5/20/2024
A diagram comparing two different groups for a study.

While Randomized Controlled Trials (RCTs) are the gold standard, it's not always ethical or practical to randomly assign a program. What if a program has already been rolled out? Or what if it's a nationwide policy? This is where quasi-experimental methods come in. They are a set of techniques used to estimate causal impact when you can't do a perfect experiment.

The Challenge: Creating a Credible Counterfactual

The goal of any impact evaluation is to answer the question: "What would have happened to the participants if they hadn't received the program?" This imaginary scenario is called the "counterfactual." In an RCT, the control group provides a perfect counterfactual. In quasi-experiments, we have to use clever statistical methods to construct a believable one.

A line graph showing two parallel trends before an intervention and diverging after.

Method 1: Difference-in-Differences (DiD)

Imagine a new fertilizer subsidy is introduced in one district but not in a neighboring, similar district. The DiD method compares the change in crop yields *before and after* the subsidy in the program district to the change in crop yields over the same period in the neighboring (control) district. By comparing the "difference in differences," we can strip out other factors that might have affected yields in both districts (like rainfall) and isolate the effect of the subsidy.

Method 2: Regression Discontinuity Design (RDD)

This is one of the most powerful quasi-experimental methods. It can be used when a program has a specific cutoff point for eligibility. For example, a scholarship is given to all students with an exam score of 80% or higher. We can then compare the long-term outcomes (like university graduation rates) of students who scored just above 80 (e.g., 80-82%) to those who scored just below (e.g., 78-79%). Since these two groups are likely very similar in every other way, any sharp "discontinuity" or jump in graduation rates right at the 80% cutoff can be attributed to the scholarship.

A scatter plot showing a clear jump or 'discontinuity' at a specific cutoff point.

These are just two examples of a broad toolkit. The key is to find a naturally occurring situation that mimics a random experiment. While more complex than an RCT, quasi-experimental methods are essential for evaluating a wide range of real-world programs and policies, providing the robust evidence needed to make informed decisions.