Next Article in Journal
Intraoperative Fluoroscopy Allows the Reliable Assessment of Deformity Correction during Periacetabular Osteotomy
Previous Article in Journal
Expression of ABCA1 Transporter and LXRA/LXRB Receptors in Placenta of Women with Late Onset Preeclampsia
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 11
Issue 16
10.3390/jcm11164813
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review

by
Shu Huang
1,
Motomori O. Lewis
1,
Yuhua Bao
2,
Prakash Adekkanattu
2,
Lauren E. Adkins
3,
Samprit Banerjee
2,
Jiang Bian
4,5,
Walid F. Gellad
6,7,
Amie J. Goodin
1,8,
Yuan Luo
9,
Jill A. Fairless
10,
Theresa L. Walunas
11,
Debbie L. Wilson
1,
Yonghui Wu
4,
Pengfei Yin
4,
David W. Oslin
12,13,
Jyotishman Pathak
2 and
Wei-Hsuan Lo-Ciganic
1,8,*
1
Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, FL 32610, USA
2
Department of Population Health Sciences, Weill Cornell Medicine, New York, NY 10065, USA
3
Health Science Center Libraries, University of Florida, Gainesville, FL 32610, USA
4
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL 32610, USA
5
Cancer Informatics Shared Resource, University of Florida Health Cancer Center, University of Florida, Gainesville, FL 32610, USA
6
Division of General Internal Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA
7
Center for Health Equity Research Promotion, Veterans Affairs Pittsburgh Healthcare System, Veterans Health Administration, Pittsburgh, PA 15240, USA
8
Center for Drug Evaluation and Safety, College of Pharmacy, University of Florida, Gainesville, FL 32610, USA
9
Division of Health and Biomedical Informatics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
10
Department of Psychiatry, University of Florida, Gainesville, FL 32610, USA
11
Department of General Internal Medicine and Geriatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
12
Veterans Integrated Service Network 4 Mental Illness Research, Education, and Clinical Center (MIRECC), Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA 15240, USA
13
Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2022, 11(16), 4813; https://doi.org/10.3390/jcm11164813
Submission received: 2 August 2022 / Revised: 12 August 2022 / Accepted: 15 August 2022 / Published: 17 August 2022
(This article belongs to the Section Mental Health)
Browse Figure
Review Reports Versions Notes

Abstract

:
Suicide is a leading cause of death in the US. Patients with pain conditions have higher suicidal risks. In a systematic review searching observational studies from multiple sources (e.g., MEDLINE) from 1 January 2000–12 September 2020, we evaluated existing suicide prediction models’ (SPMs) performance and identified risk factors and their derived data sources among patients with pain conditions. The suicide-related outcomes included suicidal ideation, suicide attempts, suicide deaths, and suicide behaviors. Among the 87 studies included (with 8 SPM studies), 107 suicide risk factors (grouped into 27 categories) were identified. The most frequently occurring risk factor category was depression and their severity (33%). Approximately 20% of the risk factor categories would require identification from data sources beyond structured data (e.g., clinical notes). For 8 SPM studies (only 2 performing validation), the reported prediction metrics/performance varied: C-statistics (n = 3 studies) ranged 0.67–0.84, overall accuracy(n = 5): 0.78–0.96, sensitivity(n = 2): 0.65–0.91, and positive predictive values(n = 3): 0.01–0.43. Using the modified Quality in Prognosis Studies tool to assess the risk of biases, four SPM studies had moderate-to-high risk of biases. This systematic review identified a comprehensive list of risk factors that may improve predicting suicidal risks for patients with pain conditions. Future studies need to examine reasons for performance variations and SPM’s clinical utility.

1. Introduction

Suicide was the 10th leading cause of death in the United States (USA) in 2019 and remains a major public health concern [1]. The age-adjusted suicide rate in the US increased by 35% from 10.5 per 100,000 population in 1999 to 14.2 in 2018 [2,3]. Patients with pain conditions have elevated risks. For example, the prevalence of suicidal ideation among individuals with chronic pain is approximately three times as high as in those without [4,5]. Furthermore, increasing rates of suicide have coincided with the epidemic of nonmedical opioid use and overdose over the past two decades [6]. Together, suicide and opioid overdose contribute to excess adverse health consequences [7], with growing evidence indicating opioid use disorder (OUD), opioid overdose, and suicide are associated [6].
The relationship between pain conditions/severity, pain management (including prescription opioids), OUD, opioid overdose, and suicide is complex [8,9,10,11]. Individuals with OUD, opioid overdose, and suicide share common risk factors, such as mental health disorders. OUD is a predominant precursor to overdose and suicide deaths [12,13,14]. Other potential interactions of opioid use and suicide among patients with pain conditions include patients suffering from pain being more likely to develop suicidal ideations from suppression and sensitivity of their neural reward pathways [10,15].
Given the unique vulnerabilities of patients with pain conditions and complex interactions of predisposing and enabling factors in this population, suicide prediction models (SPMs) may be effective tools to identify patients at risk and prevent adverse outcomes, including death [16,17]. Although previous SPM studies have shown promising results, few have focused on patients with pain conditions. Belsher et al.’s systematic review evaluated predictive accuracy of SPM studies from 2015 to 2018, focusing on the general population [18]. The existing SPMs had good C-statistics for suicide deaths, but very low positive predictive value (PPV), and the overall predictive performance remained poor for the general population. To develop and improve the prediction performance for suicide-related outcomes in patients with pain conditions, this study aimed to conduct a systematic review to (1) to evaluate the performance of existing SPMs among patients with pain conditions, and (2) to identify risk factors and derived data sources of suicidal ideation (SI), suicide attempts (SA), suicide deaths (SD), and suicide behaviors (SB) [19,20] among patients with pain conditions [21,22].

2. Materials and Methods

2.1. Data Sources and Search Strategies

This systematic review followed internationally accepted gold standard guidelines stated in the Cochrane Handbook for Systematic Reviews of Interventions and was complied with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines [23,24]. The study protocol was registered with PROSPERO (i.e., International Prospective Register of Systematic Reviews; Registration ID: CRD42020215887). An initial search was performed on 13 January 2021 of grey literature sources and the bibliographic databases PubMed, PsycINFO, EMBASE, SCOPUS, Cochrane Library, Web of Science, ProQuest Thesis Dissertations, and CINAHL for studies published from January 2000 to December 2020. According to the Cochrane Handbook, to reduce the risk of publication bias and retrieve all relevant evidence, the inclusion of gray literature sources such as conference abstracts and proceedings, theses, dissertations, and reports are necessary [23]. Additionally, we manually screened the reference lists from published suicide-related systematic reviews.
In consultation with a librarian (L.E.A.), the search strategy combined database-specific controlled vocabulary truncated and phrase-searched keywords in titles and abstracts as available for the concepts pain conditions, suicide-related outcomes, and risk prediction or predictive models. Search results were limited to the English language. Table S1 provides a complete list of search queries for databases used.

2.2. Study Selection and Data Extraction

We restricted our search to observational studies identifying risk or protective factors associated with suicide-related outcomes for adult patients aged ≥18 years with any pain condition. Since most studies did not specify pain conditions or differentiate between acute and chronic pain, we included studies focusing on patients with physical pain (e.g., fibromyalgia) and/or psychological pain. Psychological pain was defined as having intense emotional pain, often associated with psychiatric disorders (e.g., depression) or emotional trauma [25,26]. We excluded case reports, opinions, animal research, and commentaries, as well as interventional studies intended to evaluate interventional efficacy (e.g., ketamine use) on suicide-related outcomes [27,28]. Eligible studies were determined to be SPM studies if they described all of the following: (1) data sources and predictor candidates (or features), (2) modeling methods or prediction procedures (e.g., randomly split data into training and testing), and (3) SPM’s prediction performance (e.g., discrimination and calibration measures) [29]. Discrimination measures the extent to which predicted high-risk patients exhibit higher rates of suicide-related outcomes compared to those predicted as low risk (e.g., C-statistic) [30]. Calibration measures risk estimates’ accuracy, considering agreement between suicide-related events estimated and observed numbers [31]. Given our goal was identifying SPM studies or studies reporting suicide risk factors for patients with pain conditions, we excluded purely descriptive studies without any regression analysis or otherwise adjusted analysis.
After a comprehensive literature search and removal of duplicates, three investigators (S.H., M.O.L., P.Y.) double screened the articles’ titles and abstracts, and independently screened for inclusion and exclusion eligibility based on full text using Covidence (Melbourne, Australia) [32]. We extracted study information and details using a standardized Microsoft Excel (Redmond, WA, USA) form. We extracted information regarding author, year of publication, country, type of study, data source, patient population (e.g., sample size, age, pain conditions), outcomes of interest and definitions, outcome rates, statistical or prediction modeling methods, and significant risk or protective factors and risk estimates (e.g., point estimate of risk ratios [RR] or odds ratios [OR], 95% confidence intervals [95%CI], and p-values). We categorized data-source types identifying and measuring each risk factor into: (1) “structured data” that naturally occur (e.g., clinical documentation or billing activities results) and outside a research context such as prespecified fields using standardized terminologies or codes in electronic medical records (EMR) or administrative claims data; (2) “unstructured data” included text-based clinical notes and reports in EMR requiring detailed manual review or sophisticated machine learning technologies such as natural language processing (NLP) to extract information; or (3) “collected data” that are not found in the EMR and require linkage to additional data sources such as a survey or registry. For studies developing an SPM, we collected prediction performances, including C-statistic, accuracy, sensitivity, and PPV. Disagreements between reviewers were resolved by consultation with a fourth investigator (W.-H.L.-C.).

2.3. Risk of Bias Assessment

Using a modified Quality In Prognosis Studies (QUIPS) tool, two investigators (S.H., M.O.L.) independently assessed included SPM studies’ quality using bias domains including (1) study participation, (2) study attrition, (3) prognostic factor measurement, (4) outcome measurement, (5) study confounding, and (6) statistical analysis and reporting [33]. We chose QUIPS, because its bias assessments were designed for prognostic modeling studies and were more appropriate for SPM study designs. We rated each SPM study in the 6 potential bias domains as having low, moderate, or high risk of bias. A third investigator (W.L-C.) led discussions to resolve disagreements.

2.4. Study Outcomes of Interest

Our outcome of interest was any suicide-related outcome including SI, SA, SD, and any other SB. SI (or suicidal thoughts) is thinking or planning to commit suicide, but without taking any action [34]. SA is when someone harms themself with the intent to end their life, which may or may not result in death [35]. SD is mortality resulting from self-injury with the intent to end one’s life [35]. SB includes SI, SA, and SD [35]. For included SPM studies, given suicide-related events’ rarity, C-statistics will not account for outcome prevalence information and can overestimate an SPM’s utility [36]. Thus, we extracted other prediction performance metrics reported such as overall accuracy or misclassification rate, sensitivity, or recall (measured as proportion of patients having suicide-related outcomes correctly identified as being at risk), PPV or precision (measured as proportion of patients identified as being at risk of suicide who have suicide-related outcomes), and negative predictive values (NPV) (i.e., fraction of patients identified with no or low suicide risk lacking suicide-related outcomes) [37].

3. Results

3.1. Characteristics of Included Studies

Figure 1 shows the initial literature search yielded 1802 records, of which 1579 remained after removing duplicates. Title and abstract screening resulted in excluding articles unrelated to the study objective (n = 1294) or duplicates (n = 3). We then identified four additional studies from prior suicide-related systematic reviews’ references, which led to 286 full-text studies assessed for eligibility. Full-text review further excluded 199 studies for the following: (1) not target population/no pain conditions (n = 109), (2) descriptive studies without any regression analysis or otherwise adjusted analysis (n = 31), (3) commentary, abstract, or no full-text (n = 21), (4) ineligible study design (n = 21), (5) no suicide-related outcome (n = 12), and (6) no predictor/risk factor identified (n = 5). This systematic review identified 87 studies reporting suicide risk factors among patients with pain conditions, including eight studies developing SPMs.
Table 1 and Table S2 summarize characteristics for each study. Sixty-four percent (n = 56) of the 87 included studies were cross-sectional, followed by retrospective cohort studies (n = 14, 16%), and case-control studies (n = 12, 14%). Sample sizes ranged from 50 to 4,863,086. Nearly half were conducted in the USA (n = 38, 44%), followed by South Korea (n = 7, 8%), Canada (n = 6, 7%), and the United Kingdom (UK) (n = 6, 7%). Eleven studies (13%) included cancer patients [38,39,40,41,42,43,44,45,46,47,48]. The most common physical pain conditions specified included fibromyalgia, cancer pain, and back pain/low back pain. Besides patients with pain conditions (physical pain and/or psychological pain), many included studies featured special populations such as veterans and older adults. The most common data sources used were multisite questionnaire (n = 28, 32%), single-site questionnaire (n = 25, 29%), and EMR data (n = 19, 22%).

3.2. Factors Associated with Suicide-Related Outcomes among Patients with Pain Conditions

Table 2 and Table S3 summarize identified suicide-related outcome risk factors among patients with pain conditions. We further grouped risk factors into 27 categories based on characteristics and clinical conditions (Table S4). The top ten most frequently reported risk factor categories were: depression/depressive disorders and their severity (reported by 33% of included studies), other patient reported factors (e.g., sexual/physical abuse, hopelessness; 29%), any unspecified physical health illness or comorbidity index (24%), other mental health conditions (15%), pain duration/severity/intensity (15%), anxiety disorders and their severity (14%), other specific pain conditions (13%), history of suicide-related outcomes (9%), sleep disorders including insomnia (9%), and social determinants of health (9%). Only six (7%) studies identified opioid use/dosage as a risk factor for suicide among patients with pain conditions. Based on our analysis, 67% of all risk factors were routinely identifiable in structured data, 19% were available primarily from unstructured data, and the remaining 15% of risk factors were potentially identifiable from both.
Depression was the top reported risk factor, and was also considered a type of psychological pain when selecting eligible studies. To avoid tautology, we checked the four articles that included patients with depression [49,50,51,52]. Three studies included patients with depression only, and identified risk factors other than depression [49,50,51]. One study included depressed patients and healthy controls, and therefore identified depression as a suicide risk factor [52]. Among the remaining 83 studies, including patients without depression as study cohort, 28 identified depression as a risk factor focused on patients with physical/psychological pain conditions other than depression. Therefore, depression was considered as an independent suicide risk factor.

3.3. Performance of Studies Developing Suicide Prediction Models

Table 3 summarizes the eight SPM studies’ main characteristics [19,20,53,54,55,56,57,58]. None comprehensively reported model prediction performance metrics. Three reported C-statistics, which ranged from 0.67 to 0.84 [19,20,57]. Five reported overall accuracy, which ranged from 0.78 to 0.96 [53,54,55,56,58]. Two reported sensitivities, which ranged from 0.65 to 0.91 [19,20]. Three reported PPV, which ranged from 0.01 to 0.43 [19,20,57]. Two (25%) validated their predictive model using external validation and bootstrap cross-validation methods [57,58]. Two used structured EMR to develop SPMs (C-statistics: 0.67–0.82; PPV: 0.01–0.14) [19,57]. Table 3 presents QUIPS risk of bias assessment results. The quality of the eight SPM studies for pain patients varied. For four studies (50%) [19,56,57,58], all risk of bias domains were rated low. The remaining four had at least one risk of bias domain rated moderate or high risk [20,53,54,55].
One research team (Fishbain et al.) conducted half of the studies [53,54,55,56], three of which used the same study sample (~2000 USA community and rehabilitation facility residents) [53,54,55]. Using a 600-item survey, they developed multivariable logistic regression models to predict various suicide-related outcomes including SI and SB with an overall accuracy ranging from 0.78 to 0.96. Several QUIPS bias domains in the three studies were rated moderate or high risk due to the potential for selection bias and attrition bias in the study sample, lack of definitions for suicide-related outcome measures, inadequate adjustment for potential confounders, and insufficient statistical reporting. The fourth SPM study (Fishbain et al. 2009) [56] developed a multivariate logistic regression model to predict SI risk among smokers with chronic low back pain (n = 81) from a single pain facility, with an overall accuracy of 78%. Risk of bias across all domains for this study was low. However, none of these four performed a validation analysis of their SPMs.
Lopez-Morinigo et al. [19] conducted a retrospective cohort study using single-site EMR data to predict suicide deaths among UK patients receiving secondary mental health care (referred mental health care) (n = 13,758). SDs were identified from death certificates using International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes. Using multivariable Cox proportional hazards regression models, the SPM C-statistic was 0.67, with 0.65 sensitivity and 0.01 PPV at the balanced risk threshold. This study had low risk of bias but did not perform SPM validation.
McKernan et al. [57] conducted a case-control study predicting SI and SA in 8879 patients with fibromyalgia using EMR data. Using bootstrapped L-1 penalized regression, C-statistics were 0.80 for SI and 0.82 for SA. However, PPV remained low for SI (0.14) and SA (0.08). Risk of bias across all domains for this study was low. This study used an independent sample to evaluate published SPMs’ external validity.
Sun et al.’s [20] cross-sectional study used chart review and survey data to develop hierarchical regression and multivariable logistic regression models to predict post suicidal ideation and suicide attempts among 137 patients with major depressive disorders in China. At the balanced cutoff threshold, C-statistic was 0.84. This study had high risk of bias for the prognostic factor measurement domain due to unclear temporality of the suicide-related outcome’s measurement, and low risk biases for remaining domains. It did not perform SPM validation.
Tektonidou et al. [58] conducted a cross-sectional study using 2007–2008 National Health and Nutrition Examination Survey to identify US adults aged ≥40 years with arthritis, diabetes, or cancer (n = 2344). They developed a random forest model with bootstrapping and cross-validation methods to predict suicidal ideation among this high-risk group. The prediction metric reported in the study was 0 misclassification error (i.e., 100% accuracy). This study had low risk of bias.

4. Discussion

In this systematic review, including 87 studies reporting suicide risk factors, we identified 107 risk factors associated with suicide-related outcomes among patients with pain conditions. Two-thirds of these risk factors are identifiable from data routinely collected for clinical documentation or billing purposes, such as structured data within an EMR. Only eight studies developed SPMs to predict suicide-related outcomes for patients with pain conditions, of which two used structured data sources. Given suicide-related outcomes’ rarity, PPVs were low, as expected. Most SPM studies had major limitations including lacking rigorous study designs, lacking robust processes for SPM development (e.g., validation), lacking thorough reporting of prediction metrics, and risk of biases. One SPM study by McKernan et al. [57] had low risk of biases and provided good quality of evidence.
Risk of SD was at least doubled in patients with chronic pain compared to those without [59]. Consistent with prior studies predicting suicide-related outcomes in general populations, common suicide risk factors for patients with pain conditions identified in our review included depressive disorders, unspecified physical or somatic pain conditions, and anxiety disorders. Unique risk factors for patients with pain conditions identified included migraine/headaches, fibromyalgia, opioid use and dosage, and perceived burdensomeness. Of the identified risk factors, modifiable risk factors valuable for designing targeted interventions are physical health conditions and mental health disorders, which may be managed with medication, behavioral therapy, and other treatment. Furthermore, this study provided a comprehensive list of risk factors critical for improving future SPM studies for identify individuals at risk of suicide among patients with pain conditions [60].
Nearly 20% of risk factors (e.g., perceived hopeless) identified in our review are only collected from survey or unstructured EMR data (e.g., clinical notes) [61]. Developing and administering a validated questionnaire instrument for patients with pain is time-consuming. Unstructured EMR data often contain valuable information for prediction such as patient behaviors and social and behavioral determinants of health infrequently captured in structured data. With advancements in computational linguistics and machine learning, NLP systems are capable of processing free-text clinical notes and producing structured outputs useful as predictors [62]. Identify additional features from unstructured clinical notes using NLP may improve SPMs’ prediction accuracy [63].
No SPM study identified in our review thoroughly reported prediction metrics unbiasedly [29,64]. Relying on C-statistics to predict rare outcomes can be misleading and uninformative [64]. Consistent with Belsher et al.’s [18] systematic review of SPMs in the general population, rarity of suicide-related outcomes in patients with pain conditions led to low PPVs (5/11 studies reported models with PPVs > 0.01) in the included SPM studies. Despite very high sensitivity and specificity, low PPVs limit a model’s clinical utility because of false positives [65]. Applying advanced machine learning methods such as neural networks may address shortcomings of current SPMs and improve identification of patients with pain conditions at high-risk of suicide-related outcomes for targeted interventions. This will require large sample sizes, which may only be attainable with large EMR- or claims-based datasets linked to death records.
Risk of bias assessments identified several major weaknesses in existing SPMs for patients with pain conditions. In total, 75% of the SPM studies used cross-sectional designs, which lack temporality between exposures or predictors and outcomes [66]. For example, Sun’s study did not clearly specify measuring risk factors before suicide-related outcomes [20]. When developing SPMs, longitudinal and prognostic modeling designs better simulate continuous population screening [29]. Furthermore, most SPM studies did not conduct any validation analyses to avoid model overfitting. For the two SPM studies that conducted validation, the performance of the model was not clearly reported. For example, Tektonidou et al. [58] developed random forest models, with a misclassification error of 0 reported for cross-validated test sets. Achieving perfect prediction in real-world settings is implausible, especially for rare suicide-related outcomes. Future studies should include robust validation processes to ensure developed SPMs are well-calibrated and reproducible in real-world data. Future SPM studies’ should make their data and methods publicly available according to Findability, Accessibility, Interoperability, and Reuse (FAIR) principles of digital assets.
Our study had two major limitations. First, we excluded studies and theses not written in English. Second, because of the limited number of SPM studies identified for patients with pain conditions, mixed study designs, and effect estimates’ varied reporting, we could not conduct pooled analyses combining the eight SPM studies’ results.
We propose the following recommendations for future studies to develop SPMs for patients with pain conditions. First, using a composite measure of any suicide-related outcomes in SPM may increase event frequency and improve PPVs, but might limit the utility of predicting suicide deaths. Second, an SPM’s utility is influenced by availability of important predictors that may require better data collection in routine clinical care or the billing process. Third, thoroughly reporting different prediction metrics to allow researchers to evaluate SPMs in an unbiased manner is important [67]. SPMs’ development should include validation processes to minimize overfitting and increase model calibration ability, and to evaluate performance variations across subpopulations and multiple datasets. Fourth, although opioid use, OUD, overdose, and suicide may impact or interact among patients with pain conditions, only a few studies in our review included opioid use as a suicide risk factor, and no studies have examined OUD or overdose history as risk factors. An aggressive prescription opioid supply regulation to curb the opioid crisis can result in unintended consequences of patients with pain no longer having access to adequate pain management (including prescription opioids) and thus lead to an increased risk for depression and suicide. Future studies should include opioid use and OUD and overdose history when developing SPMs. Finally, clinical and policy utility of SPMs is influenced by their outcome risk and benefit profile, interventions type, and resource availability. Using SPMs with proper risk stratification (e.g., top 1st percentile, top 5th percentile as high risk) may more effectively identify high-risk individuals to better target time-sensitive interventions. For rare outcomes and SPMs with low PPVs, additional screening and assessment are needed to avoid unintended consequences resulting from false positives.

5. Conclusions

Although our findings revealed major limitations of existing SPMs and the need for mature and robust predictive models to predict suicide-related outcomes for patients with pain conditions, the systematic review provides a comprehensive summary of evidence of the risk factors of suicide-related outcomes and SPMs for patients with pain conditions that can be used to improve SPMs’ performance. Risk factors identified from this review may be valuable for clinicians to identify patients with elevated suicide risks for target interventions.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm11164813/s1, Table S1: Suicide Prediction Modeling Systematic Review Search Queries (Searches Were Conducted on 9 December 2020); Table S2: Characteristics of Included Studies (n = 87); Table S3: Summary of Individual Risk Factors Identified from Less Than 3 Studies by Data Source for Identification; Table S4: Identified Risk Factors in Each Risk Factor Category.

Author Contributions

S.H. and W.-H.L.-C. had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Conceptualization, S.H. and W.-H.L.-C.; methodology, S.H. and W.-H.L.-C.; software, S.H.; validation, S.H., M.O.L. and W.-H.L.-C.; formal analysis, S.H. and M.O.L.; investigation, S.H. and W.-H.L.-C.; resources, S.H. and L.E.A.; data curation, S.H., M.O.L. and W.-H.L.-C.; writing—original draft preparation, S.H., D.L.W. and W.-H.L.-C.; writing—review and editing, S.H., M.O.L., Y.B., P.A., L.E.A., S.B., J.B., W.F.G., A.J.G., Y.L., J.A.F., T.L.W., D.L.W., Y.W., P.Y., D.W.O., J.P., W.-H.L.-C.; visualization, S.H.; supervision, W.-H.L.-C.; project administration, S.H., W.-H.L.-C., J.B., D.W.O., J.P. and D.L.W.; funding acquisition, Y.B., S.B., J.B., W.F.G., Y.W., D.W.O., J.P. and W.-H.L.-C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded in part by the NIH, grant numbers R01MH121907 and R01DA050676.

Institutional Review Board Statement

Ethical review and approval were waived for this study due to the use of de-identified data.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data are publicly available.

Conflicts of Interest

W.-H.L.-C. has received grant funding from Merck, Sharp & Dohme, and Bristol Myers Squibb. T.L.W. has received grant funding from Gilead Sciences. None of these are relevant to this study. The remaining authors report no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS). 2003. Available online: https://webappa.cdc.gov/sasweb/ncipc/leadcause.html (accessed on 1 August 2022).
  2. Hedegaard, H.; Curtin, S.C.; Warner, M. Increase in Suicide Mortality in the United States, 1999–2018; NCHS Data Brief; National Center for Health Statistics: Hyattsville, MD, USA, 2020; pp. 1–8. [Google Scholar]
  3. National Institute of Mental Health. Suicide is a Leading Cause of Death in the United States. 2021. Available online: https://www.nimh.nih.gov/health/statistics/suicide.shtml (accessed on 1 August 2022).
  4. Breslau, N. Migraine, suicidal ideation, and suicide attempts. Neurology 1992, 42, 392–395. [Google Scholar] [CrossRef] [PubMed]
  5. Magni, G.; Rigatti-Luchini, S.; Fracca, F.; Merskey, H. Suicidality in chronic abdominal pain: An analysis of the Hispanic Health and Nutrition Examination Survey (HHANES). Pain 1998, 76, 137–144. [Google Scholar] [CrossRef]
  6. Ashrafioun, L.; Bishop, T.M.; Conner, K.R.; Pigeon, W.R. Frequency of prescription opioid misuse and suicidal ideation, planning, and attempts. J. Psychiatr. Res. 2017, 92, 1–7. [Google Scholar] [CrossRef] [PubMed]
  7. Webster, L.R. Risk Factors for Opioid-Use Disorder and Overdose. Anesth. Analg. 2017, 125, 1741–1748. [Google Scholar] [CrossRef]
  8. Campbell, G.; Bruno, R.; Darke, S.; Shand, F.; Hall, W.; Farrell, M.; Degenhardt, L. Prevalence and Correlates of Suicidal Thoughts and Suicide Attempts in People Prescribed Pharmaceutical Opioids for Chronic Pain. Clin. J. Pain 2016, 32, 292–301. [Google Scholar] [CrossRef]
  9. Brown, L.A.; Lynch, K.G.; Cheatle, M. Pain catastrophizing as a predictor of suicidal ideation in chronic pain patients with an opiate prescription. Psychiatry Res. 2020, 286, 112893. [Google Scholar] [CrossRef]
  10. Racine, M. Chronic pain and suicide risk: A comprehensive review. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 2018, 87 Pt B, 269–280. [Google Scholar] [CrossRef]
  11. Manchikanti, L.; Helm, S., II; Fellows, S.H.B.; Janata, J.W.; Pampati, V.; Grider, J.S.; Boswell, M.V. Opioid epidemic in the United States. Pain Physician 2012, 15 (Suppl. 3), Es9–Es38. [Google Scholar] [CrossRef]
  12. Bohnert, A.S.B.; Ilgen, M.A. Understanding Links among Opioid Use, Overdose, and Suicide. N. Engl. J. Med. 2019, 380, 71–79. [Google Scholar] [CrossRef]
  13. Bohnert, A.S.B.; Ilgen, M.A.; Ignacio, R.V.; McCarthy, J.F.; Valenstein, M.; Blow, F.C. Risk of Death From Accidental Overdose Associated With Psychiatric and Substance Use Disorders. Am. J. Psychiatry 2012, 169, 64–70. [Google Scholar] [CrossRef]
  14. Bohnert, K.M.; Ilgen, M.A.; Louzon, S.; McCarthy, J.F.; Katz, I.R. Substance use disorders and the risk of suicide mortality among men and women in the US Veterans Health Administration. Addiction 2017, 112, 1193–1201. [Google Scholar] [CrossRef]
  15. Elman, I.; Borsook, D.; Volkow, N.D. Pain and suicidality: Insights from reward and addiction neuroscience. Prog. Neurobiol. 2013, 109, 1–27. [Google Scholar] [CrossRef]
  16. Espinet, S.; Corrin, T.; Baliunas, D.; Quilty, L.; Zawertailo, L.; Rizvi, S.J.; De Ruiter, W.; Bonato, S.; De Luca, V.; Kennedy, S.; et al. Predisposing and protective factors influencing suicide ideation, attempt, and death in patients accessing substance use treatment: A systematic review and meta-analysis protocol. Syst. Rev. 2019, 8, 115. [Google Scholar] [CrossRef]
  17. McCarthy, J.F.; Cooper, S.A.; Dent, K.R.; Eagan, A.E.; Matarazzo, B.B.; Hannemann, C.M.; Reger, M.A.; Landes, S.J.; Trafton, J.A.; Schoenbaum, M.; et al. Evaluation of the Recovery Engagement and Coordination for Health-Veterans Enhanced Treatment Suicide Risk Modeling Clinical Program in the Veterans Health Administration. JAMA Netw. Open 2021, 4, e2129900. [Google Scholar] [CrossRef]
  18. Belsher, B.E.; Smolenski, D.J.; Pruitt, L.D.; Bush, N.E.; Beech, E.H.; Workman, D.E.; Morgan, R.L.; Evatt, D.P.; Tucker, J.; Skopp, N.A. Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation. JAMA Psychiatry 2019, 76, 642–651. [Google Scholar] [CrossRef]
  19. Lopez-Morinigo, J.-D.; Fernandes, A.C.; Shetty, H.; Ayesa-Arriola, R.; Bari, A.; Stewart, R.; Dutta, R. Can risk assessment predict suicide in secondary mental healthcare? Findings from the South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLaM BRC) Case Register. Soc. Psychiatry Psychiatr. Epidemiol. 2018, 53, 1161–1171. [Google Scholar] [CrossRef]
  20. Sun, X.; Li, H.; Song, W.; Jiang, S.; Shen, C.; Wang, X. ROC analysis of three-dimensional psychological pain in suicide ideation and suicide attempt among patients with major depressive disorder. J. Clin. Psychol. 2020, 76, 210–227. [Google Scholar] [CrossRef]
  21. Bryant, R.A.; O’Donnell, M.L.; Forbes, D.; McFarlane, A.C.; Silove, D.; Creamer, M. The course of suicide risk following traumatic injury. J. Clin. Psychiatry 2016, 77, 648–653. [Google Scholar] [CrossRef]
  22. Campbell, G.; Darke, S.; Degenhardt, L.; Townsend, H.; Carter, G.; Draper, B.; Farrell, M.; Duflou, J.; Lappin, J. Prevalence and Characteristics Associated with Chronic Noncancer Pain in Suicide Decedents: A National Study. Suicide Life Threat. Behav. 2020, 50, 778–791. [Google Scholar] [CrossRef]
  23. Higgins, J.P.T.; Thomas, J.; Chandler, J.; Cumpston, M.; Li, T.; Page, M.J.; Welch, V.A. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022): Cochrane. 2021. Available online: www.training.cochrane.org/handbook (accessed on 1 August 2022).
  24. Hutton, B.; Salanti, G.; Caldwell, D.M.; Chaimani, A.; Schmid, C.H.; Cameron, C.; Ioannidis, J.P.A.; Straus, S.; Thorlund, K.; Jansen, J.P.; et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann. Intern. Med. 2015, 162, 777–784. [Google Scholar] [CrossRef]
  25. Merskey, H.E. Classification of Chronic Pain. In Classification of Chronic Pain, 2nd ed.; IASP Press: Washington, DC, USA, 1994; pp. 209–214. [Google Scholar]
  26. Tossani, E. The Concept of Mental Pain. Psychother. Psychosom. 2013, 82, 67–73. [Google Scholar] [CrossRef]
  27. Grunebaum, M.F.; Galfalvy, H.C.; Choo, T.H.; Keilp, J.G.; Moitra, V.K.; Parris, M.S.; Marver, J.E.; Burke, A.K.; Milak, M.S.; Sublette, M.E.; et al. Ketamine for Rapid Reduction of Suicidal Thoughts in Major Depression: A Midazolam-Controlled Randomized Clinical Trial. Am. J. Psychiatry 2018, 175, 327–335. [Google Scholar] [CrossRef]
  28. Miller, I.W.; Camargo, C.A., Jr.; Arias, S.A.; Sullivan, A.F.; Allen, M.H.; Goldstein, A.B.; Manton, A.P.; Espinola, J.A.; Jones, R.; Hasegawa, k.; et al. Suicide Prevention in an Emergency Department Population: The ED-SAFE Study. JAMA Psychiatry 2017, 74, 563–570. [Google Scholar] [CrossRef]
  29. Collins, G.S.; Reitsma, J.B.; Altman, D.G.; Moons, K.G. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD statement. BMC Med. 2015, 13, 1. [Google Scholar] [CrossRef]
  30. Pencina, M.J.; D’Agostino, R.B. Evaluating Discrimination of Risk Prediction Models: The C Statistic; Livingston, E.H., Lewis, R.J., Eds.; JAMA Guide to Statistics and Methods; McGraw-Hill Education: New York, NY, USA, 2019. [Google Scholar]
  31. Van Calster, B.; McLernon, D.J.; van Smeden, M.; Wynants, L.; Steyerberg, E.W.; Bossuyt, P.; Bossuyt, P.; Collins, G.S.; Macaskill, P.; McLernon, D.J.; et al. Calibration: The Achilles heel of predictive analytics. BMC Med. 2019, 17, 230. [Google Scholar] [CrossRef]
  32. Veritas Health Innovation. Covidence Systematic Review Software 2021. Available online: https://www.covidence.org/ (accessed on 1 August 2022).
  33. Hayden, J.A.; van der Windt, D.A.; Cartwright, J.L.; Côté, P.; Bombardier, C. Assessing bias in studies of prognostic factors. Ann. Intern. Med. 2013, 158, 280–286. [Google Scholar] [CrossRef]
  34. Klonsky, E.D.; May, A.M.; Saffer, B.Y. Suicide, Suicide Attempts, and Suicidal Ideation. Annu. Rev. Clin. Psychol. 2016, 12, 307–330. [Google Scholar] [CrossRef]
  35. O’Connor, E.; Gaynes, B.; Burda, B.U.; Williams, C.; Whitlock, E.P. Screening for Suicide Risk in Primary Care: A Systematic Evidence Review for the U.S. Preventive Services Task Force; Evidence Synthesis No. 103. AHRQ Publication No. 13-05188-EF-1; Agency for Healthcare Research and Quality: Rockville, MD, USA, 2013.
  36. Saito, T.; Rehmsmeier, M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 2015, 10, e0118432. [Google Scholar] [CrossRef]
  37. MacKinnon, D.R.; Farberow, N.L. An assessment of the utility of suicide prediction. Suicide Life Threat. Behav. 1976, 6, 86–91. [Google Scholar]
  38. Akechi, T.; Okamura, H.; Nishiwaki, Y.; Uchitomi, Y. Predictive factors for suicidal ideation in patients with unresectable lung carcinoma: A 6-month follow-up study. Cancer 2002, 95, 1085–1093. [Google Scholar] [CrossRef]
  39. Akechi, T.; Okuyama, T.; Uchida, M.; Kubota, Y.; Hasegawa, T.; Suzuki, N.; Komatsu, H.; Kusumoto, S.; Iida, S. Factors associated with suicidal ideation in patients with multiple myeloma. Jpn. J. Clin. Oncol. 2020, 50, 1475–1478. [Google Scholar] [CrossRef] [PubMed]
  40. Johnson, C.C.; Phillips, K.M.; Miller, S.N. Suicidal Ideation among Veterans Living with Cancer Referred to Mental Health. Clin. Gerontol. 2020, 43, 24–36. [Google Scholar] [CrossRef] [PubMed]
  41. Mitchell, R.; Draper, B.; Harvey, L.; Brodaty, H.; Close, J. The association of physical illness and self-harm resulting in hospitalisation among older people in a population-based study. Aging Ment. Health 2017, 21, 279–288. [Google Scholar] [CrossRef]
  42. Ozdemiroglu, F.; Memis, C.O.; Meydan, N.; Dogan, B.; Kilic, S.M.; Sevincok, L.; Karakus, K. Self-Esteem, Pain and Suicidal Thoughts in a Sample of Cancer Patients. J. Mood Disord. 2017, 7, 156–162. [Google Scholar] [CrossRef]
  43. Park, S.A.; Chung, S.H.; Lee, Y. Factors associated with suicide risk in advanced cancer patients: A cross-sectional study. Asian Pac. J. Cancer Prev. 2016, 17, 4831–4836. [Google Scholar]
  44. Walker, J.; Waters, R.A.; Murray, G.; Swanson, H.; Hibberd, C.J.; Rush, R.W.; Storey, D.J.; Strong, V.A.; Fallon, M.T.; Wall, L.R.; et al. Better off dead: Suicidal thoughts in cancer patients. J. Clin. Oncol. 2008, 26, 4725–4730. [Google Scholar] [CrossRef]
  45. Webb, R.T.; Kontopantelis, E.; Doran, T.; Qin, P.; Creed, F.; Kapur, N. Risk of self-harm in physically ill patients in UK primary care. J. Psychosom. Res. 2012, 73, 92–97. [Google Scholar] [CrossRef]
  46. Xu, Q.; Jia, S.; Fukasawa, M.; Lin, L.; Na, J.; Mu, Z.; Li, B.; Li, N.; Zhao, T.; Ju, Z.; et al. A cross-sectional study on associations of physical symptoms, health self-efficacy, and suicidal ideation among Chinese hospitalized cancer patients. BMC Psychiatry 2020, 20, 1–10. [Google Scholar] [CrossRef]
  47. Zhang, Y.; Li, W.; Zhang, Z.; Sun, H.; Garg, S.; Yang, Y.; Wang, H. Suicidal Ideation in Newly-Diagnosed Chinese Cancer Patients. Front. Psychiatry 2020, 11, 708. [Google Scholar] [CrossRef]
  48. Zhong, B.L.; Li, S.H.; Lv, S.Y.; Tian, S.L.; Liu, Z.D.; Li, X.B.; Zhuang, H.-Q.; Tao, R.; Zhang, W.; Zhuo, C.-J. Suicidal ideation among Chinese cancer inpatients of general hospitals: Prevalence and correlates. Oncotarget 2017, 8, 25141–25150. [Google Scholar] [CrossRef]
  49. Alacreu-Crespo, A.; Cazals, A.; Courtet, P.; Olié, E. Brief assessment of psychological pain to predict suicidal events at one year in depressed patients. Psychother. Psychosom. 2020, 89, 320–323. [Google Scholar] [CrossRef]
  50. Gensichen, J.; Teising, A.; Konig, J.; Gerlach, F.M.; Petersen, J.J. Predictors of suicidal ideation in depressive primary care patients. J. Affect. Disord. 2010, 125, 124–127. [Google Scholar] [CrossRef]
  51. Li, H.; Xie, W.; Luo, X.; Fu, R.; Shi, C.; Ying, X.; Wang, N.; Yin, Q.; Wang, X. Clarifying the role of psychological pain in the risks of suicidal ideation and suicidal acts among patients with major depressive episodes. Suicide Life-Threat. Behav. 2014, 44, 78–88. [Google Scholar] [CrossRef]
  52. Caceda, R.; Durand, D.; Cortes, E.; Prendes-Alvarez, S.; Moskovciak, T.; Harvey, P.D.; Nemeroff, C.B. Impulsive Choice and Psychological Pain in Acutely Suicidal Depressed Patients. Psychosom Med. 2014, 76, 445–451. [Google Scholar] [CrossRef]
  53. Fishbain, D.A.; Bruns, D.; Lewis, J.E.; Disorbio, J.M.; Gao, J.; Meyer, L.J. Predictors of Homicide-Suicide Affirmation in Acute and Chronic Pain Patients. Pain Med. 2011, 12, 127–137. [Google Scholar] [CrossRef]
  54. Fishbain, D.A.; Bruns, D.; Meyer, L.J.; Lewis, J.E.; Gao, J.; Disorbio, J.M. Exploration of the relationship between disability perception, preference for death over disability, and suicidality in patients with acute and chronic pain. Pain Med. 2012, 13, 552–561. [Google Scholar] [CrossRef]
  55. Fishbain, D.A.; Lewis, J.E.; Bruns, D.; Gao, J.; Disorbio, J.M.; Meyer, L. Patient predictor variables for six forms of suicidality. Eur. J. Pain 2012, 16, 706–717. [Google Scholar] [CrossRef]
  56. Fishbain, D.A.; Lewis, J.E.; Gao, J.; Cole, B.; Steele Rosomoff, R. Are chronic low back pain patients who smoke at greater risk for suicide ideation? Pain Med. 2009, 10, 340–346. [Google Scholar] [CrossRef]
  57. McKernan, L.C.; Lenert, M.C.; Crofford, L.J.; Walsh, C.G. Outpatient Engagement and Predicted Risk of Suicide Attempts in Fibromyalgia. Arthritis Care Res. 2019, 71, 1255–1263. [Google Scholar] [CrossRef]
  58. Tektonidou, M.G.; Dasgupta, A.; Ward, M.M. Suicidal ideation among adults with arthritis: Prevalence and subgroups at highest risk. Data from the 2007-2008 National Health and Nutrition Examination Survey. Arthritis Care Res. 2011, 63, 1322–1333. [Google Scholar] [CrossRef]
  59. Tang, N.K.; Crane, C. Suicidality in chronic pain: A review of the prevalence, risk factors and psychological links. Psychol. Med. 2006, 36, 575–586. [Google Scholar] [CrossRef]
  60. Iams, J.D.; Newman, R.B.; Thom, E.A.; Goldenberg, R.L.; Mueller-Heubach, E.; Moawad, A.; Sibai, B.M.; Caritis, S.N.; Miodovnik, M.; Paul, R.H.; et al. Frequency of Uterine Contractions and the Risk of Spontaneous Preterm Delivery. N. Engl. J. Med. 2002, 346, 250–255. [Google Scholar] [CrossRef] [PubMed]
  61. Kong, H.J. Managing Unstructured Big Data in Healthcare System. Healthc Inf. Res. 2019, 25, 1–2. [Google Scholar] [CrossRef] [PubMed]
  62. Kreimeyer, K.; Foster, M.; Pandey, A.; Arya, N.; Halford, G.; Jones, S.F.; Forshee, R.; Walderhaug, M.; Botsis, T. Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review. J. Biomed. Inform. 2017, 73, 14–29. [Google Scholar] [CrossRef]
  63. Obeid, J.S.; Dahne, J.; Christensen, S.; Howard, S.; Crawford, T.; Frey, L.J.; Stecker, T.; Bunnell, B.E. Identifying and Predicting Intentional Self-Harm in Electronic Health Record Clinical Notes: Deep Learning Approach. JMIR Med. Inform. 2020, 8, e17784. [Google Scholar] [CrossRef]
  64. Romero-Brufau, S.; Huddleston, J.M.; Escobar, G.J.; Liebow, M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit. Care 2015, 19, 285. [Google Scholar] [CrossRef]
  65. Lo-Ciganic, W.-H.; Huang, J.L.; Zhang, H.H.; Weiss, J.C.; Wu, Y.; Kwoh, C.K.; Donohue, J.M.; Cochran, G.; Gordon, A.J.; Malone, D.C.; et al. Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions. JAMA Netw. Open 2019, 2, e190968. [Google Scholar] [CrossRef]
  66. Di Girolamo, N.; Mans, C. 11—Research Study Design; Miller, R.E., Lamberski, N., Calle, P.P., Eds.; Fowler’s Zoo and Wild Animal Medicine Current Therapy; W.B. Saunders: Philadelphia, PA, USA, 2019; Volume 9, pp. 59–62. [Google Scholar]
  67. Moons, K.G.; Altman, D.G.; Reitsma, J.B.; Ioannidis, J.P.; Macaskill, P.; Steyerberg, E.W.; Vickers, A.J.; Ransohoff, D.F.; Collins, G.S. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann. Intern. Med. 2015, 162, W1–W73. [Google Scholar] [CrossRef]
Jcm 11 04813 g001 550
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Flowchart of the Systematic Review.
Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Flowchart of the Systematic Review.
Jcm 11 04813 g001
Table
Table 1. Characteristics of Suicide Prediction Modeling Studies.
Table 1. Characteristics of Suicide Prediction Modeling Studies.
Author, YearCountryStudy DesignType of Data SourcesStudy Population a Total # Pts Outcome (s)Statistical MethodsValidationC-StatisticAccuracySensitivityPPV
Fishbain, 2009USACross-sectionalSingle site questionnaireChronic low back pain pts who smoke81SILogistic regressionNo validationN/A0.78N/AN/A
Fishbain, 2011USACross-sectionalCommunity questionnaire (multisite)Rehabilitation pain pts2264SILogistic regressionNo validationN/A0.96N/AN/A
Fishbain, 2012USACross-sectionalCommunity questionnaire (multisite)Rehabilitation pain pts2264SI bLogistic regressionNo validationN/A0.78–0.88N/AN/A
Fishbain, 2012USACross-sectionalCommunity questionnaire (multisite)Rehabilitation pain pts2264SBLogistic regressionNo validationN/A0.87–0.95N/AN/A
Lopez-Morinigo, 2018UKRetrospective cohortSingle site EMRPts seen in a comprehensive pain clinic13,758SDCox proportional hazards modelNo validation0.67N/A0.650.01
McKernan, 2018USACase-controlSingle site EMRPts with fibromyalgia8879SI & SABootstrapped L-1 penalized regressionIndependent sample to test the external validation of published SPMs0.82 (SA), 0.80 (SI)N/AN/A0.08 (SA), 0.14 (SI)
Sun, 2020ChinaCross-sectionalSingle site chart review, Single site questionnairePsychiatric outpatients with major depressive disorder137Past SI & SALogistic regressionNo validation0.84N/A0.910.43
Tektonidou, 2011USACross-sectionalNationwide questionnairePts aged ≥40 with arthritis, diabetes, or cancer2344SIRandom forest modelBootstrap, Cross-validationN/A1 cN/AN/A
Abbreviations: EMR: electronic medical record, N/A: not available, pts: patients, PPV: positive predictive value, SA: suicidal attempts, SD: suicide deaths, SI: suicidal ideation, UK: United Kingdom, USA: United States of America. a All patients are adult patients. b SI related item: prefer death over disability c The cross-validated test set misclassification error for each random forest was 0.
Table
Table 2. Summary of Individual Risk Factors Identified from more than 3 studies by Data Source for Identification a.
Table 2. Summary of Individual Risk Factors Identified from more than 3 studies by Data Source for Identification a.
Risk FactorsNumber of Studies% of the 87 StudiesData Source that can be Used to Identify Risk Factors b
Depression/depressive disorders and their severity2933%Structured/Unstructured/Collected data c
Any unspecified physical or somatic pain conditions1719%Structured
Anxiety disorders and their severity1214%Structured/Unstructured/Collected data
History of suicidal behavior/ideation/attempts/suicidality89%Structured/Unstructured/Collected data
Pain duration/severity/intensity89%Unstructured/Collected data
Sleep disorders including insomnia89%Structured
Age78%Structured
Psychache/mental pain78%Unstructured/Collected data
PTSD67%Structured
Fibromyalgia pain56%Structured
Gender56%Structured
Migraine/headaches and frequency56%Structured
Opioid use and dosage (e.g., >100 MME)56%Structured
Perceived burdensomeness55%Unstructured/Collected data
Antidepressant use and type45%Structured
Comorbidity or comorbidity index45%Structured
Perceived/feeling hopeless45%Unstructured/Collected data
Race/ethnicity45%Structured
AUD33%Structured
Anger issues33%Structured/Unstructured/Collected data
Any mental health illness33%Structured
Any unspecified physical health illness33%Structured
Back pain/low back pain33%Structured
Cancer pain33%Structured/Unstructured/Collected data
Drug use disorders33%Structured
History of sexual/physical abuse33%Structured/Unstructured/Collected data
Marital status (e.g., unmarried)33%Structured/Unstructured/Collected data
Mental quality of life33%Unstructured/Collected data
Pain catastrophizing33%Unstructured/Collected data
Perceived/feeling stressful33%Unstructured/Collected data
Respiratory diseases33%Structured
Unemployment33%Unstructured/Collected data
Abbreviations: AUD: alcohol use disorder, EMR: electronic medical records, MME: morphine milligram equivalents, PTSD: Posttraumatic stress disorder. a Risk factors reported from less than 3 studies are listed in Table S3 in the Supplementary Materials. b This is the authors’ view of where the majority of these data can be captured from. c We categorized the type of data sources that can be used to identify and measure each risk factor into: (1) “structured data” that naturally occur (e.g., as a result of clinical documentation or billing activities) and outside a research context such as structured EMR or administrative claims data; (2) “unstructured data” include unstructured clinical notes in EMR required efforts such as natural language processing to extract information; and (3) “collected data” that require additional design such as from a questionnaire or registry. Structured/Unstructured/Collected data refers to some of the risk factors (e.g., depression diagnosis) and may be identified from structured data, and some (e.g., depression severity) may be identified from unstructured data or questionnaires.
Table
Table 3. Quality In Prognosis Studies (QUIPS) Risk of Bias Assessment Results.
Table 3. Quality In Prognosis Studies (QUIPS) Risk of Bias Assessment Results.
Study ParticipationStudy AttritionPrognostic Factor MeasurementOutcome MeasurementStudy ConfoundingStatistical Analysis and Reporting
Fishbain, 2009ModerateModerateLowHighModerateModerate
Fishbain, 2011HighModerateLowModerateModerateModerate
Fishbain, 2012HighModerateLowModerateModerateModerate
Fishbain, 2012LowLowLowLowLowLow
Lopez-Morinigo, 2018LowLowLowLowLowLow
McKernan, 2018LowLowLowLowLowLow
Sun, 2020LowLowModerateLowLowLow
Tektonidou, 2011LowLowLowLowLowLow
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Huang, S.; Lewis, M.O.; Bao, Y.; Adekkanattu, P.; Adkins, L.E.; Banerjee, S.; Bian, J.; Gellad, W.F.; Goodin, A.J.; Luo, Y.; et al. Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review. J. Clin. Med. 2022, 11, 4813. https://doi.org/10.3390/jcm11164813

AMA Style

Huang S, Lewis MO, Bao Y, Adekkanattu P, Adkins LE, Banerjee S, Bian J, Gellad WF, Goodin AJ, Luo Y, et al. Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review. Journal of Clinical Medicine. 2022; 11(16):4813. https://doi.org/10.3390/jcm11164813

Chicago/Turabian Style

Huang, Shu, Motomori O. Lewis, Yuhua Bao, Prakash Adekkanattu, Lauren E. Adkins, Samprit Banerjee, Jiang Bian, Walid F. Gellad, Amie J. Goodin, Yuan Luo, and et al. 2022. "Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review" Journal of Clinical Medicine 11, no. 16: 4813. https://doi.org/10.3390/jcm11164813

APA Style

Huang, S., Lewis, M. O., Bao, Y., Adekkanattu, P., Adkins, L. E., Banerjee, S., Bian, J., Gellad, W. F., Goodin, A. J., Luo, Y., Fairless, J. A., Walunas, T. L., Wilson, D. L., Wu, Y., Yin, P., Oslin, D. W., Pathak, J., & Lo-Ciganic, W.-H. (2022). Predictive Modeling for Suicide-Related Outcomes and Risk Factors among Patients with Pain Conditions: A Systematic Review. Journal of Clinical Medicine, 11(16), 4813. https://doi.org/10.3390/jcm11164813

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop