An Evaluation of Balance Metrics in Propensity Score Analyses

Abstract

Propensity score analysis is one of many options for controlling for confounding variables in observational studies. A properly specified propensity score can be used to create a balanced sample of treated and control subjects. The sample can be created using matching, weighting or stratification. After a sample has been created it is important to assess whether or not the treated and control subjects are balanced on the confounding variables. If the control and treated subjects are not balanced on the confounds then the propensity score needs to be recalculated. Assessing balance is, therefore, of utmost necessity when conducting a propensity score analysis. Previous work has proposed certain methods for assessing balance. To date there have been few attempts to compare various methods for assessing balance. When attempts have been made to compare methods, they have been limited to a small handful of methods (e.g., five) and limited (e.g., fewer than a dozen) conditions. The current study examines over a dozen possible methods for assessing balance across more than 4,000 conditions. Results suggest that some of the newest available methods do not perform as well as older and simpler methods.