Postdischarge Suicide Prediction Improved With Social Determinants of Health

Performance of machine learning models that predict suicide within a year following Veteran Health Administration psychiatric hospitalization can be improved with clinical notes and social determinant of health public records, according to study findings published in the Journal of the American Medical Association Psychiatry.

Investigators sought to determine whether adding information from clinical notes and public records could improve machine learning (ML) model prediction of suicide within 1 year among patients discharged from a psychiatric hospital. The primary endpoint was suicide within 12 months following psychiatric hospital discharge.

They conducted a prognostic study that included 448,788 psychiatric hospitalizations. They used models trained (January 2010 through August 2012) to predict suicides in the 12 months (916) following Veterans Health Administration (VHA) short-term (less than 365 days) psychiatric hospitalizations (299,050; >93% men; median age 55 years). These models were subsequently validated in hospitalizations (149,738; >93% men; median age 55 years) from September 2012 through December 2013 followed by suicides (393) within 12 months of hospital discharge.

Both training sample and validation sample were about 62% non-Hispanic White, 24% non-Hispanic Black, and 8% Hispanic. The 12-month suicide rate was significantly higher in the training sample than in the validation sample. The mean suicide rate in both samples decreased with time since discharge, however the reasons were unclear.

Investigators focused validation on net benefit across a range of plausible decision thresholds. They used Shapley additive explanations (SHAP) values to assess predictor importance. Investigators used the National Death Index to define suicides. The base model predictors included electronic health records of the VHA and patient residential data.

Additional predictors came from a social determinants of health (SDOH) public records database (LexisNexis SDOH database of public records as of the month before hospitalization) and from natural language processing (NLP) of clinical notes (consolidated free text file of VHA clinical notes for all inpatient and outpatient visits in the 12 months before and including hospitalization).

Investigators noted that there were 4 broad predictor classes expanding predictors in the earlier model (psychopathologic risk factors [diagnoses, treatments, suicidality], physical disorders and treatments and counts of medications classified by the US Food and Drug Administration [FDA] as increasing suicide risk, indicators of SDOH at patient level and geospatial level, and facility-level quality indicators over the 12 months prior to the date of hospitalization).

They found the model that included both SDOH and NLP generally showed the highest net benefit up to 12 months following discharge (area under the receiver operating characteristic curve range, 0.747-0.780; area under the precision recall curve relative to the suicide rate range, 3.87-5.75).

They noted that physical disorders had larger proportional SHAP values (40.3%) than psychopathologic risk factors (34.2%). A count of medications classified by the US FDA as increasing suicide risk prescribed the year before hospitalization was the single highest positive variable-level SHAP value (15.0%). The highest predictor class-level SHAP values were NLP predictors (64.0%) and SDOH predictors (49.3%).

Study limitations include absence of consensus on minimum level of risk needed to justify implementing aggressive suicide prevention intervention, lack of generalizability beyond the VHA, and uncertain longitudinal prospective generalization within the VHA.

Investigators concluded, “Improvements could be made to a standard ML model for postdischarge suicides by adding predictors extracted from clinical notes and public records.” They wrote, “The model had positive net benefit over 3-month to 12-month risk horizons for plausible decision thresholds.” They urge caution inferring causality based on predictor importance.

Disclosure: Some study authors declared affiliations with biotech, pharmaceutical, and/or device companies. Please see the original reference for a full list of authors’ disclosures.

This article originally appeared on Psychiatry Advisor