Assessing Risk Score Calculation in the Presence of Uncollected Risk Factors

Abstract

Risk scores developed from risk prediction models assist clinicians and patients as a decision support tool. Additionally, well supported risk scores can also be used as an adjusting risk factor in clinical research. This application of risk scores presents a set of challenges, especially in retrospective analyses, when the required risk factors are uncollected for all patients in the study. Uncollected risk factors cannot be handled with traditional missing data techniques. We performed a simulation study to understand how a risk factor's prevalence, weight, and relationship with other risk factors impacts the risk score when it is uncollected. We simulated the true risk score along with two alternatives, an omit model with the risk score calculated assuming the risk factor is absent, and a refit model to understand what information could be gained if all of the data were available to refit the model and calculate a new risk score. The performance of the alternative risk scores were measured by risk score correlation, discrimination, calibration, integrated discrimination improvement (IDI) and difference in risk. We will illustrate our finding using the Framingham stroke risk score.