A spillover experiment is a research trial designed to measure how an intervention applied to one group affects the outcomes of other individuals within the same network. By analyzing these indirect effects, researchers can determine the broader impact of a policy, such as how a vaccine campaign might protect unvaccinated neighbors or how a marketing discount influences non-purchasing peers.
Recent research from the National Bureau of Economic Research (NBER) indicates that optimizing the distribution of treatment assignments can increase the effective sample size of these experiments by 50% to 100% or more (Borusyak, Hull, & Munro, 2024). When you structure your experiment to maximize signal strength while balancing the diffusion of treatment effects, you significantly reduce the variance of your results.
The optimal treatment assignment in a spillover experiment is one that balances the intensity of the treatment signal against the need to maintain variation across the network. By using a regression-based approach that accounts for the exposure mapping—the mathematical relationship between your treatment and the surrounding network—you minimize the worst-case asymptotic variance in your estimates (Borusyak et al., 2024). This means that instead of assigning treatments randomly, you apply them in a way that ensures the spillover effects are distinct and measurable, rather than blurred by noise.
In practice, this requires a shift from simple randomized controlled trials (RCTs) toward a design that considers the structure of your network. If your experiment is too concentrated, you may fail to see the spillover; if it is too diffuse, the signal becomes weak and indistinguishable from random data fluctuations. A robust design identifies the "sweet spot" where the treatment group and the spillover group provide the most statistical leverage.
A recentered instrumental variable (IV) estimator is a statistical tool used to isolate the causal effect of spillovers by accounting for the random nature of treatment assignment. Because spillover effects often correlate with underlying network characteristics, standard regression models can produce biased results. The recentered IV approach adjusts for these correlations, allowing you to extract a cleaner, more reliable signal from the observed data (Borusyak et al., 2024).
When you apply this estimator, you are essentially correcting for the "noise" created by the network's structure. This method is particularly effective in clustered settings, such as classrooms or villages, where participants are naturally grouped together. By recentering the data, you ensure that the observed outcomes are tied specifically to the intervention rather than to the pre-existing traits of the network members.
Keep exploring
More from GroundworkFor complex or large-scale networks, you can use computationally tractable approximations to design experiments that would otherwise be too difficult to map mathematically. While exact optimization is ideal for small, well-defined groups, general networks often require simplified models that approximate the ideal treatment distribution without requiring exhaustive computing power (Borusyak et al., 2024). These approximations are highly effective in bipartite settings, where two distinct groups interact in a predictable way.
To implement this, you should first map your network to identify the exposure pathways. Once the pathways are identified, use the approximation tools to determine the optimal density of your treatment assignments. This allows you to scale your experiment to larger populations while maintaining the statistical rigor of a smaller, more controlled study.
Spillover analysis is critical in development economics because it captures the total welfare impact of a policy, rather than just the direct effect on the recipient. Studies have shown that policies designed for individuals often have massive ripple effects that traditional experiments miss, leading to an underestimation of the policy's true cost-effectiveness (Borusyak et al., 2024). By using the new design tools discussed here, researchers can prove the efficacy of interventions that might otherwise appear to have negligible impacts.
For example, in a microfinance program, the spillover effect might include the economic benefit to the entire village, not just the borrower. If you fail to measure these spillovers, you are effectively ignoring a large portion of the intervention's success. Implementing a robust design allows you to capture these externalities accurately, providing better data for future financial and social policy decisions.
To improve the accuracy of your spillover experiments, follow these steps:
- Map your exposure: Define exactly how one person’s treatment is expected to influence another. Are you looking at geographic proximity, social connection, or economic interaction?
- Select your assignment strategy: Avoid simple random assignment if your network is clustered. Instead, use a distribution that spreads the treatment to maximize the variation of spillover exposure.
- Utilize a recentered IV estimator: When analyzing your data, do not rely on standard OLS regression. Use a recentered instrumental variable approach to account for the assignment process and network noise.
- Validate with semi-synthetic testing: Before running the full experiment, simulate your design using existing data to see how the estimator performs under different assignment distributions.
