Description:

Danil Fedchenko (Northwestern):

The Treatment Effect on a Summary Index: Interpretation, Inference, Power (paper attached)

Abstract: This paper examines the common practice of combining multiple outcomes into summary indices to estimate the effect of an intervention. While the interpretation of such estimates is often intuitive – a weighted average of component-level effects – the weights used by procedures more complex than the simple average can be problematic. Correct inference requires accounting for data-dependent weights when computing standard errors. The resulting correction depends on the variability of the weighting scheme and the magnitude of the true treatment effect, and disappears when either is zero. Since none of the commonly used summary index procedures incorporate sources of variability relevant to the estimator’s variance, the resulting estimates do not exhibit the high power often attributed to them. Common data manipulations, such as imputing missing values in index components, often produce indices that differ from what they are intended to represent. In such cases, the interpretation of the resulting estimates is no longer that of a weighted sum; instead, it depends on the specific estimation method and the approach used to handle missing data. This study aims to help applied researchers better understand the econometric procedures they use and to assist readers of these studies in interpreting the results.