Evaluating an Existing Suicide Risk Prediction Tool in Transgender Patients

Abstract

Validated in a general hospital population, the Vanderbilt Suicide Attempt and Ideation Likelihood (VSAIL) model predicts risk of suicide 30 days post-visit. VSAIL performance differed between clinical settings, noting worse performance in behavioral health. Creating “site-aware” VSAIL models may improve performance in clinics but does not guarantee equitable performance for historically underrepresented groups, like the transgender and gender diverse (TGD) community. We first performed a systematic review of methods used to identify TGD patients in electronic health record (EHR) data, finding that combining structured and unstructured data offered the most consistent results. We then developed a method to ascertain the status of TGD patients using novel identifiers (pronouns and ICD-10 codes alone) and verified TGD status via manual review of clinical notes; successfully identifying 2523 patients and achieving a positive predictive value (PPV) of 99.49%. We leveraged the confirmed and verified TGD cohort to evaluate the performance of VSAIL in TGD patients. We found that VSAIL was not well calibrated for the TGD cohort with a Spiegelhalter’s p-value <0.001. VSAIL had high false negative rates (between 27.5% and 77.2%, depending on the selected score threshold), and PPV values (between 6.1% and 19.3%). This indicates that visits with high VSAIL scores are likely followed by a suicide event but between 27.5% and 77.2% of visits that are followed by a suicide event do not generate a sufficiently high VSAIL score.