A Unified Effect Size Index and Its Application to Improve Replicability in Brain-Behavior Association Studies

Abstract

Reporting of effect sizes (ES), such as Cohen's d and odds ratios, alongside their confidence intervals, has gained attention for its ability to convey both the strength and precision of scientific findings simultaneously. However, existing ES indices are model-specific, presenting a challenge for researchers attempting to compare effect sizes across studies addressing similar questions but using different statistical models. In the first chapter of my dissertation work, I first introduce a robust ES index (RESI) that is not conditional on statistical models to facilitate ES reporting within the cross-sectional study setting. However, the newly proposed ES index hasn’t solved the systematic differences in ESs between cross-sectional and longitudinal study designs yet, thereby complicating comparisons between the two. To resolve this, in the second chapter, I propose a new version of RESI tailored for longitudinal studies, which estimates ES as if the study were conducted cross-sectionally, thereby improving comparability across different study designs. The proposed ES index unifies ES reporting across studies using different models and/or different designs (i.e., cross-sectional or longitudinal), bridging a critical gap in ES communication. Recently, several studies raised concerns about the low replicability of brain-behavior association studies and showed that thousands of study participants are required for good replicability. However, massive sample sizes are often infeasible in practice. In the last chapter of my dissertation work, I apply the proposed ES index and systematically investigate how we can leverage the modifiable study design features to improve the ESs (and therefore, the replicability) of brain-behavior association studies, using the large-scale data from the Lifespan Brain Chart Consortium. Based on strong empirical evidence and pragmatic statistical theory, concrete and actionable study design and analysis procedures are provided to the neuroscientist to help them improve the replicability of their studies, given their different research objectives and the nature of their target associations.