Evaluation of machine learning methods for prognostic and adverse event risk in Major Depressive Disorder

Abstract

Clinical risk prediction models trained on electronic health record (EHR) data can support clinicians in decision making by estimating prognostic risk. Major Depressive Disorder (MDD) risk modeling presents particular challenges due to data sparsity and a developing understanding of risk factors. Autoencoders are an unsupervised learning method that have potential to account for some of the data challenges by learning feature dependencies, reducing dimensionality, and denoising. We evaluated two use cases for autoencoders—feature embedding and pretraining—and found that pretrained models outperform comparable models from the literature.

Treatment decisions often involve switching antidepressants and the safety of this practice is an important consideration in treatment decisions. EHR data reflect clinical practice and can be used to measure adverse event risk, but the observational nature of EHR data can result in biased analyses. We evaluated two common methods for treatment effect estimation—propensity score weighting and propensity score matching—and used control outcomes to measure bias and calibrate effect estimates of adverse event risk. We found propensity score matching estimates were more robust, potentially due to clinical expertise informing selection of the study population.

Prior research suggests that adverse event risk may be heterogeneous. We apply our finding of bias reduction in the matched population to estimation of heterogenous treatment effects (HTE). First, we used semi-synthetic outcomes to measure model performance under varying data generating processes. Followed by significance testing to identify adverse event HTE. Analysis of semi-synthetic and adverse event outcomes identified limited evidence for heterogeneity and highlights steps necessary to implement clinically useful HTE models. Our work improved prognostic modeling, showed evidence for robust adverse event effect estimation, and provided insight into variation in HTE model performance in the MDD phenotype.