Next Article in Journal
Induction of Apoptosis of Cancer Cells Using the Cisplatin Delivery Based Electrospray (CDES) System
Next Article in Special Issue
Unsupervised Cell Segmentation and Labelling in Neural Tissue Images
Previous Article in Journal
Integrated Approach for Detecting Convection Effects in Geothermal Environments Based on TIR Camera Measurements
Previous Article in Special Issue
Instance Segmentation Based on Deep Convolutional Neural Networks and Transfer Learning for Unconstrained Psoriasis Skin Images
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Machine Learning in Chronic Pain Research: A Scoping Review

by
Marit Dagny Kristine Jenssen
1,
Per Atle Bakkevoll
2,3,*,
Phuong Dinh Ngo
2,
Andrius Budrionis
2,
Asbjørn Johansen Fagerlund
2,
Maryam Tayefi
2,
Johan Gustav Bellika
2,3 and
Fred Godtliebsen
1,2
1
Department of Mathematics and Statistics, University of Tromsø—The Arctic University of Norway, NO-9019 Tromsø, Norway
2
Norwegian Centre for E-Health Research, P.O. Box 35, NO-9038 Tromsø, Norway
3
Department of Clinical Medicine, University of Tromsø—The Arctic University of Norway, NO-9019 Tromsø, Norway
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(7), 3205; https://doi.org/10.3390/app11073205
Submission received: 18 February 2021 / Revised: 26 March 2021 / Accepted: 29 March 2021 / Published: 2 April 2021
(This article belongs to the Special Issue Medical Artificial Intelligence)

Abstract

:
Given the high prevalence and associated cost of chronic pain, it has a significant impact on individuals and society. Improvements in the treatment and management of chronic pain may increase patients’ quality of life and reduce societal costs. In this paper, we evaluate state-of-the-art machine learning approaches in chronic pain research. A literature search was conducted using the PubMed, IEEE Xplore, and the Association of Computing Machinery (ACM) Digital Library databases. Relevant studies were identified by screening titles and abstracts for keywords related to chronic pain and machine learning, followed by analysing full texts. Two hundred and eighty-seven publications were identified in the literature search. In total, fifty-three papers on chronic pain research and machine learning were reviewed. The review showed that while many studies have emphasised machine learning-based classification for the diagnosis of chronic pain, far less attention has been paid to the treatment and management of chronic pain. More research is needed on machine learning approaches to the treatment, rehabilitation, and self-management of chronic pain. As with other chronic conditions, patient involvement and self-management are crucial. In order to achieve this, patients with chronic pain need digital tools that can help them make decisions about their own treatment and care.

1. Introduction

Chronic pain has a serious impact on both individuals and society. Prevalence estimates vary among studies, partly due to inconsistent definitions of chronic pain and differences in the methods used to estimate it [1]. Several studies conducted in Europe and the United States have estimated that prevalence is 15–50% [1,2,3,4,5]. In terms of individuals, chronic pain can severely affect quality of life [6,7]. It is also known that comorbidities are common among patients with chronic pain, e.g., anxiety, depression, sleep disturbance, fatigue, and decreased overall physical and mental functioning [1,8,9]. Mortality rates have been found to be higher in individuals affected by chronic pain. For instance, a large meta-analysis showed that for widespread chronic pain, mortality is 2.5-times higher than normal [10]. In terms of society, chronic pain has a major influence on the consumption of healthcare resources and related costs [11]. The reduced work capacity of patients with chronic pain results in major expenditures in sick leave compensation and disability benefit [12]. For individuals with chronic pain, a study found that the probability of being unemployed was twice as high and the probability of receiving disability benefits was four-times higher, compared to the general population [13].
The most common definition of pain is “an unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage” [14]. Chronic pain or persistent pain is pain that lasts more than three months (i.e., longer than the normal healing time of tissue). Chronic pain is different from acute pain. Acute pain is caused by a specific disease or injury. Acute pain is usually resolved as the injured tissue heals [15]. Chronic pain may arise from psychological states, does not serve any biological purpose, and does not have a predictable endpoint [16]. Despite recent advances in empirical pain research, effective methods for the treatment of chronic pain are still lacking. How individual patients with chronic pain respond to treatment and the adverse effects they experience are highly variable. In many cases, the cause of chronic pain is unknown, as there are no objective findings or measures that can be used to detect it. This makes treatment and management complex and time consuming. Due to the high variability among patients, successful management of chronic pain requires personalisation with respect to each patient’s pain intensity and duration, disease state, tolerance of adverse events, and risk of medication abuse [17].
Identifying the underlying mechanisms of disease and enabling personalisation of care are key to improving the treatment of chronic pain. These are areas where machine learning (ML) is expected to be useful because ML excels at detecting patterns, rules, and causal dependencies in large datasets. ML algorithms learn from data without being explicitly programmed and can be used to obtain insights and make predictions and decisions from large and complex datasets. In healthcare, these analyses can help to better predict, identify, and treat diseases. To create reliable ML models, large volumes of relevant training data are needed. Increased adoption of information systems in healthcare creates vast amounts of health data and facilitates research in ML, which has the potential to realise person-centred healthcare. Simultaneously, citizens are increasingly using health apps and wearables for self-management, thus generating large amounts of health-related data outside of the formal healthcare sector. This wide range of health data, generated both in and out of clinical settings, can potentially be used by ML algorithms to support chronic pain research. This paper intends to determine the state-of-the-art knowledge in this field, identify directions for future studies using health data from numerous sources, and indicate which decision-support tools and recommendation systems for both patients and clinicians may be useful. Such tools can be built either by using existing chronic pain models or data-driven methods.
ML is used both for classification (predicting discrete values such as male/female, pain/not chronic pain, etc.) and regression (predicting continuous values, such as height, salary, etc.). A ML algorithm can be supervised (labelled training data available) or unsupervised (no labelled training data available). Brief explanations of the most common ML algorithms follow below. For more information about ML algorithms, the book by Theodoridis [18] can be consulted.
A naive Bayes classifier calculates the probability that a data point belongs to a certain class using the Bayes rule. The data point is assigned to the class to which it most likely belongs. This method is naive in the sense that it assumes the data points are independent of each other. Support vector machine (SVM) is a supervised method where the task is to create a hyperplane separating two classes. The hyperplane is found by optimising a cost function. Decision trees (DTs) are multistage decision systems where classes are sequentially rejected until an accepted class is reached. DTs can be used both for classification and regression. A random forest (RF) is an ensemble of DTs. The output of a RF is the mode of the classes (classification) or the average prediction of the trees (regression). K-nearest neighbours (kNN) is a supervised learning method where for each data point, the distance to the training points is considered. If most of the nearest training points to the data point belong to a certain class, the data point is assigned to that class. Clustering is the task of dividing an unlabelled dataset into groups. k-means clustering is one of the most popular clustering methods. In k-means clustering, the task is to partition n data points into k clusters. Each observation is assigned to the cluster with the nearest mean. AdaBoost is the abbreviation for adaptive boosting. Boosting is a technique used for creating a strong classifier from a number of weak classifiers. Deep learning is a family of ML methods based on neural networks (NNs). The algorithms are inspired by the human brain. NNs consist of neurons arranged in different layers, similar to neurons in the brain.
Previous review papers have focused on how ML and statistical methods [19,20,21,22] can be utilised in the treatment of patients with chronic pain. However, these reviews had a restricted scope or were outdated in that they did not consider recent developments in ML research. An up-to-date analysis of ML research in chronic pain care is currently lacking. This is the intended contribution of the present paper.
The aim of this scoping review is to summarize and disseminate research findings on the use of ML approaches in chronic pain research, to identify research gaps, and to make recommendations for future research [23]. The literature search was guided by a set of predefined search queries to major research databases. The identified publications were screened by several independent reviewers following pre-defined inclusion and exclusion rules. This rigid process ensured broad coverage of published evidence and facilitated the objective validity and relevance assessment of every publication included in the review [23]. The following research questions (RQ) will be addressed in the present review:
  • RQ1: Which ML methods have been used?
  • RQ2: Which data have been used?
  • RQ3: Were patient-generated data used?
  • RQ4: Which chronic pain conditions were applied with ML algorithms?
  • RQ5: Which stages (e.g., assessment/diagnosis or treatment) of chronic pain care were addressed?
By answering the research questions, we plan to contribute to the body of knowledge by providing insight into how ML approaches may be used to improve the treatment and management of patients with chronic pain.

2. Methods

2.1. Search Strategy

The literature search of three databases, PubMed, IEEE Xplore, and the ACM Digital Library, was performed 27 February 2019. The search included the keywords “machine learning”, “pain”, and “fibromyalgia”, to be present in the title or abstract. Specifically, the following query was used:
(pain OR fibromyalgia) AND machine learning.
This query returned 299 records. After removing duplicates, two-hundred eighty-seven records were included in the screening process (Figure 1). PubMed searches required keyword presence in the title or abstract. This option was not available in IEEE Xplore and the ACM Digital Library, which limited keyword searches to abstracts only.

2.2. Inclusion Criteria

Before screening titles and abstracts, the paper inclusion criteria were defined as follows:
  • The paper must be written in English.
  • The reported trials must be done on humans.
  • The trials must include patients with chronic pain.
  • The study must have been completed.
  • This review focused on chronic pain studies where the source of the pain was not known, such as fibromyalgia (FM) and chronic low back pain (CLBP).

2.3. Exclusion Criteria

To limit the scope of the review, the following exclusion criteria were applied to the identified publications:
  • Review papers were excluded.
  • Studies on acute pain were excluded.
  • Headache, migraine, and chest pain were excluded.
  • Postoperative pain and cancer-related pain were excluded.
  • Studies with only healthy research participants were excluded.
We wanted to review studies on chronic pain patients. Review papers occurring in our literature search were therefore excluded. Our focus was on chronic pain; hence, we exclude all studies on acute pain. Chest pain is often acute or related to heart disease, which is why this was excluded. Headache and migraine could be chronic, but we decided to exclude these conditions. For postoperative and cancer-related pain, the reason for the pain is known, while we focused on pain with an unknown reason. Pain studies on healthy participants were excluded because they were not related to chronic pain.

2.4. Screening and Full-Text Reading

Before full-text reading, we read through the abstracts of the 287 records (Figure 1). Based on screening of the abstracts, one-hundred ninety-nine papers were excluded. There was a conflict on 51 papers. Of these 51 papers, twenty-eight were excluded after resolving the conflict. In total, two-hundred twenty-seven papers were excluded based on the screening. After screening the abstracts, sixty papers remained for full-text reading. There was a conflict on 12 papers. As in the screening process, we resolved the conflict and ended up with 39 papers included in the review. At this stage, twenty-one articles were excluded by the following reasons:
  • Wrong population (only healthy participants included in trials, no patients with chronic pain): twelve papers (all from the original search)
  • Wrong publication type: nine papers (five from the original search and four from the references)
  • Wrong study design: twelve papers (seven from the original search and five from the references)
  • Content overlap [24,25] (from original search, excluding [24])
  • Studies not using ML: ten papers (all from the papers found in the reference lists)
Some of the papers were excluded for more than one of the reasons above. While reading papers in full text, we discovered some papers in the reference lists that seemed relevant for our review, but that were not identified in the original literature search because the term “machine learning” was not present in their titles or abstracts. Hence, to identify additional papers for inclusion in the review, we checked the reference lists of the papers that had already been included. Titles containing words such as “classification” and “clustering” were considered. Thirty-two papers from the reference lists were selected for further review. Following a full-text reading of these papers, nine of them were included, and eighteen of them are excluded. In total, fifty-three studies were included in the review (Table 1).

3. Results

3.1. Applications of Machine Learning in Chronic Pain Research

The results of the review are summarised in Table 1. In this table, the publications are grouped into six different categories with respect to the objectives of the studies and which stages of chronic pain care were addressed (RQ5):
  • Classification/diagnosis of patients with chronic pain using structured health data
  • Classification/diagnosis of patients with chronic pain using text and images
  • Genomics approaches and pain biomarker identification
  • Treatment
  • Self-management
  • Measurement of pain intensity
For each publication, information was extracted and is represented as columns in Table 1 for answering the research questions (RQ1–RQ5) as follows: chronic pain condition (RQ4), data size and data sources (RQ2 and RQ3), and ML methods (RQ1).
The results clearly showed an obvious problem of interest would be to find an appropriate way of distinguishing patients with chronic pain from healthy controls. Many of the included papers had such a focus [32,35,36,46,49,50,52,53,54]. The identification of useful means of distinguishing patients with chronic pain from healthy controls would make it easier to characterise a patient with a certain set of symptoms and thereby make it easier to suggest a successful treatment [26]. A specific line of research was provided in [51], where the goal was to distinguish FM from rheumatoid arthritis (RA) patients. Although these patients have many similar symptoms, they usually require different treatment. Hence, a successful classifier in this setting could lead to improved treatment outcomes for these patients. A good first step in identifying the best treatment for the individual patient is to cluster patients with the same disease, as done in [27,31].
Identifying genes relevant to chronic pain is useful in determining patients with chronic pain conditions and can provide links to the pathophysiology of chronic pain. However, the number of papers related to genomics approaches and biomarker identification is small. We found two studies that aimed to identify biomarkers for bladder pain and pelvic pain using extended random forest [56,58]. In other studies, Ultsch et al. [57] identified 535 genes related to pain. This set of genes can be considered a reflection of the knowledge concerning the genetic architecture of pain acquired in any context of pain research available at the time of publication. Lukkahatai et al. [55] developed an algorithm to identify genes that discriminate FM patients from healthy controls.
Efficient ML methods for classifying and clustering patients with chronic pain have proven to be useful in pain assessment and treatment decision-making. These methods have been used both for making a correct diagnosis and to identify subgroups among patients with the same diagnosis who potentially react differently to a certain treatment, such as two of the studies that aimed at identifying subgroups of FM patients by using clustering techniques [27,31]. Examples of research more directly related to treatment include outcome prediction in acupuncture [25], suicide prediction among FM patients [62], and disc localisation in CLBP patients [67]. Jiang et al. [63] used SVM-based classifiers of surface electromyography (SEMG) data to identify low back pain (LBP) patients who would respond to a 12 week rehabilitation program. The best performing classifier achieved a classification accuracy of 97%. Thus, the authors claimed that their study could be helpful for physicians or therapists in recommending the best treatment for individual LBP patients. In another study [61], a clinical decision support tool was built for treatment of LBP using three ML models (DT, RF, and boosted tree).
Self-management has also been shown to be a key application of machine learning in empowering the patients to manage their condition. Six of the included papers focused on self-management of chronic pain [69,71,72,73,74,76]. Two of these were authored by the same group of researchers, with both studies using the same training dataset [74,76]. They aimed to support patient self-management by analysing self-reported data from patients with chronic pain pre-treatment, post-treatment, and at three month follow-up. The main topics of the studies were prediction of patients’ health level [76] and feature selection to optimise questionnaires by omitting irrelevant questions [74]. Three studies were related to fear-avoidance behaviour and anticipation of pain [69,71,73]. Regular physical activity is an important factor for successful management of chronic musculoskeletal pain [73]. However, many patients avoid physical activity due to a strong fear of pain. Jamison et al. [69] aimed to identify individuals who are prone to catastrophise their pain, by analysing daily assessment data from a mobile application for patients with chronic pain. Meier et al. [71] analysed self-reported questionnaire data and functional magnetic resonance imaging (fMRI) of patients with CLBP to estimate the level of pain-related fear in individuals. Rabbi et al. [73] examined the feasibility of a mobile app to promote physical activity among patients suffering from chronic back pain. The application uses ML to analyse sensor data and self-reported physical activity data to generate personalised physical activity recommendations based on the patient’s behaviour. One study focused on measuring and predicting pain volatility by employing ML methods to analyse data from users of a pain management app [72].
Valid and reliable assessment of pain is essential in diagnostics of chronic pain and in guiding treatment decisions [82]. This review includes five studies focusing on automatic estimation of pain intensity. Of these, four studies used facial expression videos from a publicly available shoulder pain research database as the training data [78,79,80,81]. The training data were annotated, and all four studies used algorithms for supervised learning. In two of the studies, a SVM was applied to classify discrete levels of pain intensity [79,81]. Kaltwang et al. [80] applied the relevance vector regression algorithm for continuous estimation of pain intensity. Jaiswal et al. [78] used a convolutional neural network (CNN) to estimate pain intensity by regression. Lee et al. [77] used ML to predict clinical pain based on brain imaging data (fMRI) and physiological data from patients with CLBP. This study used support vector regression (SVR) for the prediction of pain intensity on a continuous scale. In research settings, brain imaging approaches can help to better understand the underlying mechanisms of pain.

3.2. Machine Learning Algorithms

A wide variety of ML algorithms were used in the reviewed papers. To provide a structured overview, these publications were grouped according to a hierarchy of ML algorithms.

3.2.1. Supervised Algorithms

The vast majority of included publications used supervised approaches for addressing problems in chronic pain research. These models rely on labelled data available for training. Most of the algorithms solved classification tasks by assigning a predefined class label to an observation. Common examples included SVMs [77,79] and DT-based models (DT and RF) [51,61]. The predictions made by these algorithms are relatively easy to interpret, making them attractive choices for addressing clinical problems.
Deep learning-based models have already demonstrated their value, especially in analysing unstructured data and images. Despite their superior performance, they are still only employed to a limited extent in medical research. This was confirmed by the papers included in this review, as only four out of 53 included publications used neural net-based models [26,36,68,78].
Seven of the included publications used various forms of regression algorithms [37,41,52,59,62,72,80]. Instead of assigning a predefined class label, these algorithms output continuous numerical values. Such outputs may be preferred when predicting risk scores or probabilities or solving other problems requiring a continuous value rather than a class label.

3.2.2. Unsupervised Algorithms

Unsupervised ML algorithms do not require labelled data for training. These algorithms are used for identifying patterns or reducing dimensionality in high-dimensional data. Four out of the 53 included publications used unsupervised method clustering (k-means) [26,27,28,31,72]. Typical problems solved by clustering algorithms were partitioning patients with chronic pain according to pain intensity/volatility, physical and mental state, and social support [31,72]. Clustering was also used for identifying granular characteristics of FM patient subgroups [27] and grouping of symptoms reported in patient survey data [26]. Clustering results, in most cases, were used in further analysis and served as the input for task-specific data-processing pipelines.

3.3. Data

The amount of health-related data collected and stored is increasing at a rapid pace. Due in particular to advances in medical imaging and genomics, substantial volumes of health data are stored in clinical systems. In addition, the increased availability and use of consumer health technologies, such as smartphones, mobile apps, and wearable devices, have led to the availability of patient-generated health data. Hence, a wide range of health data exists, both in and out of clinical settings, which can potentially be used by ML algorithms in chronic pain research.
Health data are either structured or unstructured. Structured data are easier to store and access and can be used in the development of ML algorithms. Examples of structured data that have been used for chronic pain diagnosis and treatment include electronic health record (EHR) data [25,32,62], claims data [27], International Classification of Diseases (ICD) codes, and patient-generated data (from questionnaires or mobile phone applications) [26,56,72]. Feature selection in structured data can be systematically done with the help of statistical analysis, similar analysis studies, and guidelines [27].
Unstructured data are data that do not have a data model or are not organised in a pre-defined manner. The most common unstructured data in the studies included in this review are magnetic resonance imaging (MRI) data. MRI involves the use of strong magnetic fields and radio waves to generate images of the organs in the body. Both fMRI and structural magnetic resonance imaging (sMRI) images are widely used in classifying patients with chronic pain versus healthy controls. Other imaging data include facial expressions used to measure pain intensity [78,79,80,81]. Medical texts are important parts of unstructured data in the EHRs that can provide useful information. Natural language processing has been used to obtain pain-related information from medical texts [39,50]. Sensor data such as gait information and electromyography (EMG) are also useful in examining the kinematic and kinetic parameters of the gait of patients [35,36] or measuring muscle activity.
Patient-generated data are also emerging as important for chronic pain research. Wearable devices such as wristbands may provide biometric and physical activity data [60]. Sensor data and patient self-reported data can complement EHR data to provide a more comprehensive picture of a patient’s health status. Questionnaires or self-reported data can be either structured or unstructured, depending on whether the data are in the form of multiple choice responses or free-text, such as patients who can indicate the frequency of pain symptoms they experienced on a point scale. ML methods that are used to analyse structured questionnaires data include adaptive networks [26].
The availability of training data is often an obstacle for ML research in health settings, due to strict privacy and confidentiality regulations. Hence, most of the datasets used in the included studies are not publicly available. However, a few are available for academic use, and these are shown in Table 2.

3.4. Types of Chronic Pain That May Benefit from ML Research

While many cases of chronic pain originate from primary underlying medical conditions or injury, the majority of studies included in this paper focused on pain conditions with unclear pathophysiology and relatively high prevalence. This subsection describes how ML has been used for the diagnosis, treatment, and management of these conditions.
CLBP is the most prevalent chronic pain condition, with estimates falling in the range of 5% to 20% [84]. Prevalence increases linearly with age between the ages of 20 and 60 years and is higher among women than men [84]. In some cases, CLBP can be caused by injury or disease, but ninety percent of patients with CLBP have non-specific low back pain [85]. CLBP is the condition most frequently studied in the current review, and more aspects of the care pathway are covered than is the case for other conditions, as can be seen in Table 3.
FM has also been examined in several studies. FM is characterised by widespread musculoskeletal pain [92], and its prevalence is estimated as being between 2% and 5% [93,94]. The most common application of ML in FM is to apply classification and clustering techniques to aid in diagnostics or to discover previously unknown subgroups of FM patients [26,27,31,32,33,46,51,55]. Identifying subgroups might be particularly useful in the treatment and management of FM since these patients are highly heterogeneous and may respond differently to the same therapy. Yim et al. [31] conducted a study where FM patients were divided into four subgroups using a cluster analysis method. Data on pain, physical, social, and psychological function were used to discover the clusters. Other applications of machine learning in FM included predicting quality of sleep, level of fatigue [60], and suicide risk [62].
ML applications for other functional disorders, such as myofascial pain syndrome (MPS) [28], irritable bowel syndrome (IBS) [26,49], and chronic fatigue syndrome (CFS) [26,37,45], could also be useful for FM patients. Similar to the FM studies, many of these studies focused on the classification of patients. Lin et al. [28] classified MPS patients by SEMG patterns using template matching and K-means clustering. Melidis et al. [26] used symptoms data from patients with a functional disorder, FM, IBS, or CFS, to discover symptom clusters. Labus et al. [49] used sMRI data to classify IBS patients versus healthy controls.
Two studies focused on the diagnosis of chronic pelvic pain (CPP) (pain in the lower abdomen and/or in the pelvis) [29,53]. Oliveiro and Poli-Neto [29] compared different classification algorithms on the multi-label problem of diagnosing which diseases (e.g., interstitital cystitis/bladder pain syndrome (IC) and IBS) are causing the pelvic pain. Bagarinao et al. [53] used sMRI data and a multivariate classification approach to distinguish participants with CPP from healthy controls. Two studies applied ML to identify diagnostic biomarkers for IC [56,58]. As with many other chronic pain conditions, the underlying cause of the disease and diagnostic biomarkers remain unknown. Chancellor et al. [56] applied supervised ML on urine samples and questionnaire data to identify biomarkers that can discriminate two sub-types of IC patients and healthy controls. To discover stool-based biomarkers for IC, Braundmeier-Fleming et al. [58] used ML to classify DNA from stool samples of IC patients and healthy controls.
Ankylosing spondylitis (AS), also known as Bekhterev’s disease, is a chronic inflammatory autoimmune disease that primarily affects spine joints, causing severe chronic pain [95]. The exact cause of AS remains unclear, and no effective treatment exists [95]. AS was the focus of two studies in the current review, both of which applied ML approaches to the same fMRI dataset to classify AS and healthy controls [41,42].
For each of the chronic pain conditions complex regional pain syndrome (CRPS), chronic neck pain, and trigeminal neuralgia (TN), we identified a single study. Yang et al. [36] proposed to apply multilayer perceptron neural networks to detect abnormal gait patterns for CRPS. CRPS affects the limbs and can be induced by surgery or trauma [96]. Zhang et al. [25] used similarity-based learning for outcome prediction of acupuncture therapy for chronic neck pain. To distinguish TN patients from healthy controls, Zhong et al. [40] applied multivariate pattern classification to neuroimaging data. TN is a condition characterised by severe facial pain and can be associated with systemic diseases such as multiple sclerosis or cardiovascular diseases [89].
Four studies focused on shoulder pain; all used the same dataset of facial expression videos to automatically assess pain intensity [78,79,80,81]. Five studies investigated chronic pain in general. Four of these focused on self-management [69,72,74,76], while one study aimed to identify genes relevant to pain [57] to aid diagnostics.

4. Discussion

4.1. Important Findings

The considerable number of papers that appeared in the literature search indicates that the use of ML methods in chronic pain research has drawn significant attention and is considered potentially useful. A wide variety of ML methods were employed in the considered studies. Most of these methods involve supervised classification or regression algorithms. Classical ML algorithms, such as SVM and DT, were commonly used in the selected studies. Deep learning was applied to a limited extent, indicating that the uptake of these algorithms may be more problematic for the research community. The limited interpretability and explainability of deep learning-based models may be important reasons why these methods, the use of which has become the de facto standard in modern image and video analysis, are less visible in the scientific publications in the medical field.
A limited number of the studies focused on the treatment of chronic pain. The majority of the studies contributed to treatment indirectly through various types of classification. The task of finding the best treatment for a specific patient is still performed by the clinician, without the assistance of clinical decision support tools. Hence, there is a need for tools that help clinicians make decisions concerning safe, effective, and efficient treatment plans based on either contemporary chronic pain models or directly on the data. Useful tools for these tasks could include DTs and reinforcement learning (RL).
Various types of data have been utilised in chronic pain studies. However, methods that integrate multiple data sources (e.g., both EHR data and patient-generated data) are underdeveloped. Consequently, existing work typically depends on a limited number of data types. The issue of data quality in clinical data and how it affects the results of studies have still not been fully explored. For genomics data, translation of findings from animal to humans is still limited and remains a challenging topic [26]. Furthermore, the sample size used in studies is typically small [63]. Moreover, long-term monitoring with large datasets is required in future work to test whether these techniques can be applied to assess rehabilitation or treatment outcomes for patients [36].
As shown in Table 3, previous research has concentrated on the early stages of the chronic pain care pathway. However, few studies have focused on the rehabilitation and self-management phases. CLBP is the pain condition that has attracted the most research interest, as it was the focus of 20 of the included studies. This is also the only condition in which research focused on all phases of the care pathway. The significant amount of research that has focused on CLBP reflects the high prevalence of the condition. FM was the second most frequently investigated condition, with 11 studies covering the diagnosis and clinical decision support phases. While the majority of the research focused on CLBP and FM, several chronic pain conditions were addressed by only one or two studies.

4.2. Recent Research

Several papers relevant for this review have been published after February 2019. A few papers are mentioned below. Lamichhane et al. 2021 [97] used SVM to classify LBP patients versus healthy controls. Brain images (MRI and fMRI) were used in the classification process. The study fits into “Classifying patients using text and images” in Table 1. Santana et al. 2020 [98] tested different classifiers (logistic regression, support vector classifier, kNN, DT, RF, and multilayer perceptron (MLP)) on chronic pain patients (FM and CLBP) and healthy controls. Questionnaires associated with depression and anxiety and quantitative sensory tests were used as input data to the classifiers. This study fits into the group “Classifying patients using structured health data” in Table 1. Santana et al. 2019 [99] used deep learning to classify chronic pain patients versus healthy controls. Resting state fMRI data were used; hence, the study can be put into the group “Classifying patients using text and images” in Table 1.

4.3. Potential Limitations of This Review

The aim of the review presented in this paper was to survey a wide range of studies to summarize the state-of-the-art with regard to ML methods in chronic pain research. The literature search was performed using three major research databases (Pubmed, IEEE Xplore, and the ACM Digital Library). To reduce the risk of overlooking important studies, we used a generic query. However, no query search can identify all relevant publications on a topic. To identify additional studies for the review, the reference lists of the included papers were checked. Publications not indexed in these major databases (typically some domain-specific conferences and recently established journals) were left out of this review. The use of strict criteria for determining the inclusion of papers can lead to more attention being paid to formalities than on the contents of a specific publication. Inclusion criteria are defined beforehand and sometimes lead to highly relevant and even top-rated publications in the field being overlooked due to minor discrepancies. In this study, the risk of excluding relevant publications was minimised by involving five independent reviewers in the publication screening process and by relying on majority votes when it came to deciding which papers should be included or excluded. Moreover, in the initial screening stage (title and abstract), only publications that were clearly irrelevant were discarded. Questionable papers were passed to the next step and considered after full-text reading.

4.4. Future Research Directions

As noted previously, few of the included studies focused on the use of ML in the treatment of chronic pain. This observation, combined with the fact that learning from data can facilitate the treatment of patients with chronic pain, indicates that decision support tools and recommendation systems for both patients and clinicians may be useful. Tools can be built either by using existing chronic pain models or based on data-driven methods.
At present, no satisfactory treatment exists for patients with chronic pain. There are at least two important reasons for this: First, chronic pain is not a life-threatening disease. Hence, it attracts less attention and is considered to be less of a priority in healthcare service [100]. Second, the population of patients with chronic pain is large and heterogeneous. Identifying the best treatment for an individual patient is challenging. ML may be key to remedy this problem. The underlying concept is that patients and clinicians can learn about the best treatment in a specific case based on real-world patient data. Existing analytical predictive models and ML techniques can be utilised to provide a personalised model of the risk associated with the diseases. Based on this model, personalised treatment plans and recommendations can be derived by maximising the treatment outcome while simultaneously minimising patient risk.
By collecting and analysing large volumes of patient data, it may be possible to identify underlying patterns that could prove instrumental to improving the quality of life for a large group of people. To achieve the best possible decision support system, access to both EHR data and patient-generated data is important. Both data sources have the potential to support the search for a successful decision support tool. Typically, over the short term, it is easier and more realistic to obtain access to patient-generated data because we assume they contain fewer sensitive data. Information like heart rate, physical activity, sleep, medication, and patient-reported pain level may offer valuable insights into the patients’ health under given circumstances. Such data may be used to determine whether the observed data can be used to quantify a patient’s level of pain. If so, using these data to build a ML model that can provide tailored recommendations patients may reduce the number of painful episodes.
RL and DTs may be utilised to build a data-driven decision support tool for the treatment and management of chronic pain. RL is an area of ML concerned with how to take optimal actions in an environment to maximise the notion of cumulative reward. Novel RL research needs to be conducted to make the algorithm safer and more robust and to reduce the risk for patients. DT is another machine learning technique that can be effective to visually and explicitly derive decisions and decision-making policies based on data.
To enable the development and usability of personalised decision support systems, the availability of patient-generated data is crucial. By utilising new self-management technologies, personal data can be collected in a straightforward and cost-effective manner. ML algorithms can learn from these large amounts of personal data to customise treatment and management to the needs and preferences of each individual. However, a platform to collect and process these data in a privacy-preserving manner needs to be developed, to enable ML algorithms to exploit the vast potential of such data sources.

5. Conclusions

Publications on ML in chronic pain research were reviewed. The reviewed papers were grouped into six categories. The largest group consisted of papers presenting approaches to classifying patients with chronic pain using structured data. While many studies have emphasised classification tasks, less attention has been paid to the treatment and management of chronic pain. FM and CLBP were the most frequently researched conditions. Several conditions attracted less research attention, as they were covered by one or two studies. More research is needed on the treatment, rehabilitation, and self-management of chronic pain. As with other chronic diseases, encouraging patient involvement and self-management is crucial. To become more involved in the management of their conditions, patients require tools that can help them make healthy choices. Increased use of advanced data-analytics methods alongside more traditional approaches has the potential to make valuable contributions to chronic pain treatment and management.

Funding

This work was supported by Tromsø Research Foundation, the Northern Norway Regional Health Authority (Grant Numbers HNF1463-19 and HNF1445-19), and the Norwegian Directorate of Health through the national project-Report for AI Implementation in Healthcare.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Acknowledgments

We would like to thank Line Linstad, Norwegian Centre for E-health Research, for her comments, which greatly improved the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

ANOVAanalysis of variance
ASankylosing spondylitis
CFSchronic fatigue syndrome
CLBPchronic low back pain
CNNconvolutional neural network
CPPchronic pelvic pain
CRPScomplex regional pain syndrome
DTdecision tree
EHRelectronic health record
EMGelectromyography
FMfibromyalgia
fMRIfunctional magnetic resonance imaging
IBSirritable bowel syndrome
ICinterstitial cystitis/bladder pain syndrome
kNNk-nearest neighbours
LBPlow back pain
LDAlinear discriminant analysis
LUCCKlearning using concave and convex kernels
MLmachine learning
MLPmultilayer perceptron
MPSmyofascial pain syndrome
MRImagnetic resonance imaging
MVPAmultivariate pattern analysis
PROMpatient reported outcome measures
PRoNTopattern recognition for neuroimaging toolbox
RArheumatoid arthritis
RFrandom forest
RLreinforcement learning
RQresearch question
SEMGsurface electromyography
sMRIstructural magnetic resonance imaging
SVMsupport vector machine
SVRsupport vector regression
TNtrigeminal neuralgia
VASvisual analogue scale

References

  1. Mills, S.; Torrance, N.; Smith, B.H. Identification and Management of Chronic Pain in Primary Care: A Review. Curr. Psychiatry Rep. 2016, 18, 22. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Breivik, H.; Collett, B.; Ventafridda, V.; Cohen, R.; Gallacher, D. Survey of chronic pain in Europe: Prevalence, impact on daily life, and treatment. Eur. J. Pain 2006, 10, 287–333. [Google Scholar] [CrossRef]
  3. Landmark, T.; Romundstad, P.; Dale, O.; Borchgrevink, P.C.; Kaasa, S. Estimating the prevalence of chronic pain: Validation of recall against longitudinal reporting (the HUNT pain study). Pain 2012, 153, 1368–1373. [Google Scholar] [CrossRef]
  4. Fayaz, A.; Croft, P.; Langford, R.M.; Donaldson, L.J.; Jones, G.T. Prevalence of chronic pain in the UK: A systematic review and meta-analysis of population studies. BMJ Open 2016, 6, e010364. [Google Scholar] [CrossRef]
  5. Dahlhamer, J.; Lucas, J.; Zelaya, C.; Nahin, R.; Mackey, S.; DeBar, L.; Kerns, R.; Von Korff, M.; Porter, L.; Helmick, C. Prevalence of Chronic Pain and High-Impact Chronic Pain Among Adults—United States, 2016. MMWR Morb. Mortal. Wkly. Rep. 2018, 67, 1001–1006. [Google Scholar] [CrossRef]
  6. Dueñas, M.; Ojeda, B.; Salazar, A.; Mico, J.A.; Failde, I. A review of chronic pain impact on patients, their social environment and the health care system. J. Pain Res. 2016, 9, 457–467. [Google Scholar] [CrossRef] [Green Version]
  7. Hadi, M.A.; McHugh, G.A.; Closs, S.J. Impact of Chronic Pain on Patients’ Quality of Life: A Comparative Mixed-Methods Study. J. Patient Exp. 2019, 6, 133–141. [Google Scholar] [CrossRef] [PubMed]
  8. Bair, M.J.; Robinson, R.L.; Katon, W.; Kroenke, K. Depression and pain comorbidity: A literature review. Arch. Intern. Med. 2003, 163, 2433–2445. [Google Scholar] [CrossRef] [Green Version]
  9. Girardi, A.; Curran, M.S. The comorbidity of chronic pain and depression. Nurs. Made Incred. Easy 2019, 17, 50–54. [Google Scholar] [CrossRef]
  10. Macfarlane, G.J.; Barnish, M.S.; Jones, G.T. Persons with chronic widespread pain experience excess mortality: Longitudinal results from UK Biobank and meta-analysis. Ann. Rheum. Dis. 2017, 76, 1815–1822. [Google Scholar] [CrossRef]
  11. Institute of Medicine (US). Pain as a public health challenge. In Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research; National Academies Press: Washington, DC, USA, 2011. [Google Scholar] [CrossRef]
  12. Nachemson, A. Chronic pain–the end of the welfare state? Qual. Life Res. 1994, 3, S11–S17. [Google Scholar] [CrossRef]
  13. Landmark, T.; Romundstad, P.; Dale, O.; Borchgrevink, P.C.; Vatten, L.; Kaasa, S. Chronic pain: One year prevalence and associated characteristics (the HUNT pain study). Scand. J. Pain 2013, 4, 182–187. [Google Scholar] [CrossRef]
  14. Merskey, H.; Bogduk, N. (Eds.) Classification of Chronic Pain: Task Force on Taxonomy of the International Association for the Study of Pain (IASP), 2nd ed.; IASP Press: Seattle, WA, USA, 1994. [Google Scholar]
  15. Ashburn, M.A.; Staats, P.S. Management of chronic pain. Lancet 1999, 353, 1865–1869. [Google Scholar] [CrossRef]
  16. Grichnik, K.P.; Ferrante, F.M. The difference between acute and chronic pain. Mt. Sinai J. Med. 1991, 58, 217–220. [Google Scholar] [PubMed]
  17. Paster, R.Z. Chronic Pain Management Issues in the Primary Care Setting and the Utility of Long-Acting Opioids. Expert Opin. Pharmacother. 2010, 11, 1823–1833. [Google Scholar] [CrossRef] [PubMed]
  18. Theodoridis, S.; Koutroumbas, K. Pattern Recognition, Fourth Edition, 4th ed.; Academic Press, Inc.: Cambridge, MA, USA, 2008. [Google Scholar]
  19. Smith, A.; López-Solà, M.; McMahon, K.; Pedler, A.; Sterling, M. Multivariate pattern analysis utilizing structural or functional MRI-In individuals with musculoskeletal pain and healthy controls: A systematic review. Semin. Arthritis Rheum. 2017, 47, 418–431. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  20. Pombo, N.; Araújo, P.; Viana, J. Knowledge discovery in clinical decision support systems for pain management: A systematic review. Artif. Intell. Med. 2014, 60, 1–11. [Google Scholar] [CrossRef] [PubMed]
  21. Liu, D.; Cheng, D.; Houle, T.T.; Chen, L.; Zhang, W.; Deng, H. Machine learning methods for automatic pain assessment using facial expression information: Protocol for a systematic review and meta-analysis. Medicine 2018, 97, e13421. [Google Scholar] [CrossRef]
  22. Lötsch, J.; Ultsch, A. Machine learning in pain research. Pain 2018, 159, 623. [Google Scholar] [CrossRef]
  23. Arksey, H.; O’Malley, L. Scoping studies: Towards a methodological framework. Int. J. Soc. Res. Methodol. 2005. [Google Scholar] [CrossRef] [Green Version]
  24. Liang, Z.; Zhang, G.; Xu, S.; Ou, A.; Fang, J.; Xu, N.; Fu, W. A kernel-decision tree based algorithm for outcome prediction on acupuncture for neck pain: A new method for interim analysis. In Proceedings of the 2011 IEEE International Conference on Bioinformatics and Biomedicine Workshops (BIBMW), Atlanta, GA, USA, 12–15 November 2011; pp. 760–764. [Google Scholar] [CrossRef]
  25. Zhang, G.; Liang, Z.; Yin, J.; Fu, W.; Li, G.Z. A similarity based learning framework for interim analysis of outcome prediction of acupuncture for neck pain. Int. J. Data Min. Bioinform. 2013, 8, 381–395. [Google Scholar] [CrossRef]
  26. Melidis, C.; Denham, S.L.; Hyland, M.E. A test of the adaptive network explanation of functional disorders using a machine learning analysis of symptoms. Biosystems 2018, 165, 22–30. [Google Scholar] [CrossRef]
  27. Davis, F.; Gostine, M.; Roberts, B.; Risko, R.; Cappelleri, J.C.; Sadosky, A. Characterizing classes of fibromyalgia within the continuum of central sensitization syndrome. J. Pain Res. 2018, 11, 2551–2560. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Lin, Y.C.; Yu, N.Y.; Jiang, C.F.; Chang, S.H. Characterizing the SEMG patterns with myofascial pain using a multi-scale wavelet model through machine learning approaches. J. Electromyogr. Kinesiol. 2018, 41, 147–153. [Google Scholar] [CrossRef]
  29. Oliverio, V.; Poli-Neto, O.B. Case study: Classification algorithms comparison for the multi-label problem of chronic pelvic pain diagnosing. In Proceedings of the 2017 IEEE 33rd International Conference on Data Engineering (ICDE), San Diego, CA, USA, 19–22 April 2017; pp. 1507–1509. [Google Scholar] [CrossRef]
  30. Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I.H. The WEKA data mining software: An update. ACM SIGKDD Explor. Newsl. 2009, 11, 10–18. [Google Scholar] [CrossRef]
  31. Yim, Y.R.; Lee, K.E.; Park, D.J.; Kim, S.H.; Nah, S.S.; Lee, J.; Kim, S.K.; Lee, Y.A.; Hong, S.J.; Kim, H.S.; et al. Identifying fibromyalgia subgroups using cluster analysis: Relationships with clinical variables. Eur. J. Pain 2017, 21, 374–384. [Google Scholar] [CrossRef]
  32. Emir, B.; Masters, E.T.; Mardekian, J.; Clair, A.; Kuhn, M.; Silverman, S.L. Identification of a Potential Fibromyalgia Diagnosis Using Random Forest Modeling Applied to Electronic Medical Records. J. Pain Res. 2015, 8, 277–288. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Garcia-Zapirain, B.; Garcia-Chimeno, Y.; Rogers, H. Machine learning techniques for automatic classification of patients with fibromyalgia and arthritis. Int. J. Comput. Trends Technol. (IJCTT) 2015, 25, 149–152. [Google Scholar] [CrossRef]
  34. Olugbade, T.A.; Aung, M.H.; Bianchi-Berthouze, N.; Marquardt, N.; Williams, A.C. Bi-modal detection of painful reaching for chronic pain rehabilitation systems. In Proceedings of the ICMI ’14 16th International Conference on Multimodal Interaction, Istanbul, Turkey, 12–16 November 2014; ACM: New York, NY, USA, 2014; pp. 455–458. [Google Scholar] [CrossRef] [Green Version]
  35. Chan, H.; Zheng, H.; Wang, H.; Sterritt, R.; Newell, D. Smart mobile phone based gait assessment of patients with low back pain. In Proceedings of the 2013 Ninth International Conference on Natural Computation (ICNC), Shenyang, China, 23–25 July 2013; pp. 1062–1066. [Google Scholar] [CrossRef]
  36. Yang, M.; Zheng, H.; Wang, H.; McClean, S.; Hall, J.; Harris, N. A machine learning approach to assessing gait patterns for Complex Regional Pain Syndrome. Med. Eng. Phys. 2012, 34, 740–746. [Google Scholar] [CrossRef]
  37. Goertzel, B.N.; Pennachin, C.; de Souza Coelho, L.; Maloney, E.M.; Jones, J.F.; Gurbaxani, B. Allostatic load is associated with symptoms in chronic fatigue syndrome patients. Pharmacogenomics 2006, 7, 485–494. [Google Scholar] [CrossRef] [Green Version]
  38. Fodeh, S.J.; Finch, D.; Bouayad, L.; Luther, S.L.; Ling, H.; Kerns, R.D.; Brandt, C. Classifying clinical notes with pain assessment using machine learning. Med. Biol. Eng. Comput. 2018, 56, 1285–1292. [Google Scholar] [CrossRef]
  39. Tan, W.K.; Hassanpour, S.; Heagerty, P.J.; Rundell, S.D.; Suri, P.; Huhdanpaa, H.T.; James, K.; Carrell, D.S.; Langlotz, C.P.; Organ, N.L.; et al. Comparison of Natural Language Processing Rules-based and Machine-learning Systems to Identify Lumbar Spine Imaging Findings Related to Low Back Pain. Acad. Radiol. 2018, 25, 1422–1432. [Google Scholar] [CrossRef]
  40. Zhong, J.; Chen, D.Q.; Hung, P.S.P.; Hayes, D.J.; Liang, K.E.; Davis, K.D.; Hodaie, M. Multivariate pattern classification of brain white matter connectivity predicts classic trigeminal neuralgia. Pain 2018, 159, 2076–2087. [Google Scholar] [CrossRef] [PubMed]
  41. Cheng, J.C.; Rogachov, A.; Hemington, K.S.; Kucyi, A.; Bosma, R.L.; Lindquist, M.A.; Inman, R.D.; Davis, K.D. Multivariate machine learning distinguishes cross-network dynamic functional connectivity patterns in state and trait neuropathic pain. Pain 2018, 159, 1764–1776. [Google Scholar] [CrossRef]
  42. Rogachov, A.; Cheng, J.C.; Hemington, K.S.; Bosma, R.L.; Kim, J.A.; Osborne, N.R.; Inman, R.D.; Davis, K.D. Abnormal Low-Frequency Oscillations Reflect Trait-Like Pain Ratings in Chronic Pain Patients Revealed through a Machine Learning Approach. J. Neurosci. 2018, 38, 7293–7302. [Google Scholar] [CrossRef] [PubMed]
  43. Schrouff, J.; Rosa, M.J.; Rondina, J.; Marquand, A.F.; Chu, C.; Richiardi, J.; Phillips, C.; Mourão-Miranda, J. Pattern Recognition for Neuroimaging Toolbox. Neuroinformatics 2013, 11, 319–337. [Google Scholar] [CrossRef] [Green Version]
  44. Mano, H.; Kotecha, G.; Leibnitz, K.; Matsubara, T.; Sprenger, C.; Nakae, A.; Shenker, N.; Shibata, M.; Voon, V.; Yoshida, W.; et al. Classification and characterisation of brain network changes in chronic back pain: A multicenter study. Wellcome Open Res. 2018, 3. [Google Scholar] [CrossRef] [Green Version]
  45. Sevel, L.S.; Boissoneault, J.; Letzen, J.E.; Robinson, M.E.; Staud, R. Structural brain changes versus self-report: Machine-learning classification of chronic fatigue syndrome patients. Exp. Brain Res. 2018, 236, 2245–2253. [Google Scholar] [CrossRef] [PubMed]
  46. López-Solà, M.; Woo, C.W.; Pujol, J.; Deus, J.; Harrison, B.J.; Monfort, J.; Wager, T.D. Towards a neurophysiological signature for fibromyalgia. Pain 2017, 158, 34–47. [Google Scholar] [CrossRef]
  47. Qin, R.; Meng, H.; Li, M. Continuous pain related behavior recognition from muscle activity and body movements. In Proceedings of the 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Changsha, China, 13–15 August 2016; pp. 1710–1715. [Google Scholar] [CrossRef] [Green Version]
  48. Olugbade, T.A.; Bianchi-Berthouze, N.; Marquardt, N.; Williams, A.C. Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain. In Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), Xi’an, China, 21–24 September 2015; pp. 243–249. [Google Scholar] [CrossRef] [Green Version]
  49. Labus, J.S.; Van Horn, J.D.; Gupta, A.; Alaverdyan, M.; Torgerson, C.; Ashe-McNalley, C.; Irimia, A.; Hong, J.Y.; Naliboff, B.; Tillisch, K.; et al. Multivariate morphological brain signatures predict chronic abdominal pain patients from healthy control subjects. Pain 2015, 156, 1545–1554. [Google Scholar] [CrossRef] [PubMed]
  50. Robinson, M.E.; O’Shea, A.M.; Craggs, J.G.; Price, D.D.; Letzen, J.E.; Staud, R. Comparison of machine classification algorithms for fibromyalgia: Neuroimages versus self-report. J. Pain 2015, 16, 472–477. [Google Scholar] [CrossRef] [Green Version]
  51. Sundermann, B.; Burgmer, M.; Pogatzki-Zahn, E.; Gaubitz, M.; Stüber, C.; Wessolleck, E.; Heuft, G.; Pfleiderer, B. Diagnostic classification based on functional connectivity in chronic pain: Model optimization in fibromyalgia and rheumatoid arthritis. Acad. Radiol. 2014, 21, 369–377. [Google Scholar] [CrossRef]
  52. Callan, D.; Mills, L.; Nott, C.; England, R.; England, S. A tool for classifying individuals with chronic back pain: Using multivariate pattern analysis with functional magnetic resonance imaging data. PLoS ONE 2014, 9, e98007. [Google Scholar] [CrossRef] [PubMed]
  53. Bagarinao, E.; Johnson, K.A.; Martucci, K.T.; Ichesco, E.; Farmer, M.A.; Labus, J.; Ness, T.J.; Harris, R.; Deutsch, G.; Apkarian, A.V.; et al. Preliminary structural MRI based brain classification of chronic pelvic pain: A MAPP network study. Pain 2014, 155, 2502–2509. [Google Scholar] [CrossRef] [Green Version]
  54. Ung, H.; Brown, J.E.; Johnson, K.A.; Younger, J.; Hush, J.; Mackey, S. Multivariate classification of structural MRI data detects chronic low back pain. Cereb. Cortex 2012, 24, 1037–1044. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Lukkahatai, N.; Walitt, B.; Deandrés-Galiana, E.J.; Fernández-Martínez, J.L.; Saligan, L.N. A predictive algorithm to identify genes that discriminate individuals with fibromyalgia syndrome diagnosis from healthy controls. J. Pain Res. 2018, 11, 2981–2990. [Google Scholar] [CrossRef] [Green Version]
  56. Chancellor, M.B.; Bartolone, S.N.; Veerecke, A.; Lamb, L.E. Crowdsourcing Disease Biomarker Discovery Research: The IP4IC Study. J. Urol. 2018, 199, 1344–1350. [Google Scholar] [CrossRef]
  57. Ultsch, A.; Kringel, D.; Kalso, E.; Mogil, J.S.; Lötsch, J. A data science approach to candidate gene selection of pain regarded as a process of learning and neural plasticity. Pain 2016, 157, 2747–2757. [Google Scholar] [CrossRef] [PubMed]
  58. Braundmeier-Fleming, A.; Russell, N.T.; Yang, W.; Nas, M.Y.; Yaggie, R.E.; Berry, M.; Bachrach, L.; Flury, S.C.; Marko, D.S.; Bushell, C.B.; et al. Stool-based biomarkers of interstitial cystitis/bladder pain syndrome. Sci. Rep. 2016, 6, 1–10. [Google Scholar] [CrossRef] [Green Version]
  59. Golabchi, F.N.; Sapienza, S.; Severini, G.; Reaston, P.; Tomecek, F.; Demarchi, D.; Reaston, M.; Bonato, P. Assessing aberrant muscle activity patterns via the analysis of surface EMG data collected during a functional evaluation. BMC Musculoskelet. Disord. 2019, 20, 13. [Google Scholar] [CrossRef] [PubMed]
  60. Sabeti, E.; Gryak, J.; Derksen, H.; Biwer, C.; Ansari, S.; Isenstein, H.; Kratz, A.; Najarian, K. Learning Using Concave and Convex Kernels: Applications in Predicting Quality of Sleep and Level of Fatigue in Fibromyalgia. Entropy 2019, 21, 442. [Google Scholar] [CrossRef] [Green Version]
  61. Nijeweme-d’Hollosy, W.O.; van Velsen, L.; Poel, M.; Groothuis-Oudshoorn, C.G.M.; Soer, R.; Hermens, H. Evaluation of three machine learning models for self-referral decision support on low back pain in primary care. Int. J. Med. Inform. 2018, 110, 31–41. [Google Scholar] [CrossRef] [PubMed]
  62. McKernan, L.C.; Lenert, M.C.; Crofford, L.J.; Walsh, C.G. Outpatient Engagement Lowers Predicted Risk of Suicide Attempts in Fibromyalgia. Arthritis Care Res. 2019, 71, 1255–1263. [Google Scholar] [CrossRef]
  63. Jiang, N.; Luk, K.D.K.; Hu, Y. A Machine Learning-based Surface Electromyography Topography Evaluation for Prognostic Prediction of Functional Restoration Rehabilitation in Chronic Low Back Pain. Spine 2017, 42, 1635–1642. [Google Scholar] [CrossRef]
  64. Aung, M.S.; Kaltwang, S.; Romera-Paredes, B.; Martinez, B.; Singh, A.; Cella, M.; Valstar, M.; Meng, H.; Kemp, A.; Shafizadeh, M.; et al. The automatic detection of chronic pain-related expression: Requirements, challenges and the multimodal EmoPain dataset. IEEE Trans. Affective Comput. 2015, 7, 435–451. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Aung, M.H.; Bianchi-Berthouze, N.; Watson, P.; Williams, A.C.D.C. Automatic recognition of fear-avoidance behavior in chronic pain physical rehabilitation. In Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany, 20–23 May 2014; pp. 158–161. [Google Scholar] [CrossRef] [Green Version]
  66. Aung, M.S.; Singh, A.; Lim, S.; Williams, A.C.d.C.; Watson, P.; Bianchi-Berthouze, N. Automatic recognition of protective behaviour in chronic pain rehabilitation. In Proceedings of the Workshop on Ubiquitous Games and Gamification for Promoting Behavior Change and Wellbeing, Trento, Italy, 16 September 2013; ACM: Trento, Italy, 2013; pp. 25–28. [Google Scholar]
  67. Ghosh, S.; Malgireddy, M.R.; Chaudhary, V.; Dhillon, G. A new approach to automatic disc localization in clinical lumbar MRI: Combining machine learning with heuristics. In Proceedings of the 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), IEEE, Barcelona, Spain, 2–5 May 2012; pp. 114–117. [Google Scholar] [CrossRef]
  68. Dickey, J.P.; Pierrynowski, M.R.; Bednar, D.A.; Yang, S.X. Relationship between pain and vertebral motion in chronic low-back pain subjects. Clin. Biomech. 2002, 17, 345–352. [Google Scholar] [CrossRef]
  69. Jamison, R.N.; Xu, X.; Wan, L.; Edwards, R.R.; Ross, E.L. Determining Pain Catastrophizing From Daily Pain App Assessment Data: Role of Computer-Based Classification. J. Pain 2019, 20, 278–287. [Google Scholar] [CrossRef]
  70. Jones, B.L.; Nagin, D.S.; Roeder, K. A SAS procedure based on mixture models for estimating developmental trajectories. Sociol. Methods Res. 2001, 29, 374–393. [Google Scholar] [CrossRef]
  71. Meier, M.L.; Vrana, A.; Humphreys, B.K.; Seifritz, E.; Stämpfli, P.; Schweinhardt, P. Pain-Related Fear-Dissociable Neural Sources of Different Fear Constructs. eNeuro 2018, 5. [Google Scholar] [CrossRef] [Green Version]
  72. Rahman, Q.A.; Janmohamed, T.; Pirbaglou, M.; Clarke, H.; Ritvo, P.; Heffernan, J.M.; Katz, J. Defining and Predicting Pain Volatility in Users of the Manage My Pain App: Analysis Using Data Mining and Machine Learning Methods. J. Med. Internet Res. 2018, 20, e12001. [Google Scholar] [CrossRef]
  73. Rabbi, M.; Aung, M.S.; Gay, G.; Reid, M.C.; Choudhury, T. Feasibility and Acceptability of Mobile Phone-Based Auto-Personalized Physical Activity Recommendations for Chronic Pain Self-Management: Pilot Study on Adults. J. Med. Internet Res. 2018, 20, e10147. [Google Scholar] [CrossRef]
  74. Huang, Y.; Zheng, H.; Nugent, C.; McCullagh, P.; Black, N.; Vowles, K.E.; McCracken, L. Feature selection and classification in supporting report-based self-management for people with chronic pain. IEEE Trans. Inf. Technol. Biomed. 2011, 15, 54–61. [Google Scholar] [CrossRef] [PubMed]
  75. Vowles, K.E.; McCracken, L.M. Acceptance and values-based action in chronic pain: A study of treatment effectiveness and process. J. Consult. Clin. Psychol. 2008, 76, 397–407. [Google Scholar] [CrossRef]
  76. Huang, Y.; Zheng, H.; Nugent, C.; McCullagh, P.; Black, N.; Vowles, K.; McCracken, L. Classification of health level from chronic pain self reporting. In Proceedings of the IADIS International Conference e-Health 2009, IADIS, Algarve, Portugal, 21–23 June 2009; pp. 43–50. [Google Scholar]
  77. Lee, J.; Mawla, I.; Kim, J.; Loggia, M.L.; Ortiz, A.; Jung, C.; Chan, S.T.; Gerber, J.; Schmithorst, V.J.; Edwards, R.R.; et al. Machine learning-based prediction of clinical pain using multimodal neuroimaging and autonomic metrics. Pain 2019, 160, 550–560. [Google Scholar] [CrossRef]
  78. Jaiswal, S.; Egede, J.; Valstar, M. Deep learned cumulative attribute regression. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, China, 15–19 May 2018; pp. 715–722. [Google Scholar] [CrossRef]
  79. Hammal, Z.; Cohn, J.F. Automatic detection of pain intensity. In Proceedings of the 14th ACM International Conference on Multimodal Interaction, ACM, Santa Monica, CA, USA, 22–26 October 2012; pp. 47–52. [Google Scholar] [CrossRef] [Green Version]
  80. Kaltwang, S.; Rudovic, O.; Pantic, M. Continuous pain intensity estimation from facial expressions. In Proceedings of the International Symposium on Visual Computing, Crete, Greece, 16–18 July 2012; Springer: Berlin/Heidelberg, Germany, 2012; pp. 368–377. [Google Scholar] [CrossRef] [Green Version]
  81. Lucey, P.; Cohn, J.F.; Prkachin, K.M.; Solomon, P.E.; Matthews, I. Painful data: The UNBC-McMaster shoulder pain expression archive database. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011), IEEE, Santa Barbara, CA, USA, 21–25 March 2011; pp. 57–64. [Google Scholar] [CrossRef]
  82. Fillingim, R.B.; Loeser, J.D.; Baron, R.; Edwards, R.R. Assessment of Chronic Pain: Domains, Methods, and Mechanisms. J. Pain Off. J. Am. Pain Soc. 2016, 17, T10–T20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  83. Wallace, P.J.; Shah, N.D.; Dennen, T.; Bleicher, P.A.; Bleicher, P.D.; Crown, W.H. Optum Labs: Building a novel node in the learning health care system. Health Aff. 2014, 33, 1187–1194. [Google Scholar] [CrossRef] [PubMed]
  84. Meucci, R.D.; Fassa, A.G.; Faria, N.M.X. Prevalence of chronic low back pain: Systematic review. Rev. Saúde Pública 2015, 49, 73. [Google Scholar] [CrossRef]
  85. Koes, B.W.; van Tulder, M.W.; Thomas, S. Diagnosis and treatment of low back pain. BMJ 2006, 332, 1430–1434. [Google Scholar] [CrossRef] [Green Version]
  86. Cimmino, M.A.; Ferrone, C.; Cutolo, M. Epidemiology of chronic musculoskeletal pain. Best Pract. Res. Clin. Rheumatol. 2011, 25, 173–183. [Google Scholar] [CrossRef]
  87. Hunter, T.M.; Boytsov, N.N.; Zhang, X.; Schroeder, K.; Michaud, K.; Araujo, A.B. Prevalence of rheumatoid arthritis in the United States adult population in healthcare claims databases, 2004–2014. Rheumatol. Int. 2017, 37, 1551–1557. [Google Scholar] [CrossRef]
  88. Pribicevic, M. The epidemiology of shoulder pain: A narrative review of the literature. In Pain in Perspective; Ghosh, S., Ed.; InTechOpen: London, UK, 2012; pp. 147–186. [Google Scholar] [CrossRef] [Green Version]
  89. De Toledo, I.P.; Réus, J.C.; Fernandes, M.; Porporatti, A.L.; Peres, M.A.; Takaschima, A.; Linhares, M.N.; Guerra, E.; Canto, G.D.L. Prevalence of trigeminal neuralgia. J. Am. Dent. Assoc. 2016, 147. [Google Scholar] [CrossRef] [PubMed]
  90. Goode, A.P.; Freburger, J.; Carey, T. Prevalence, practice patterns, and evidence for chronic neck pain. Arthritis Care Res. 2010, 62, 1594–1601. [Google Scholar] [CrossRef] [Green Version]
  91. Gerwin, R. Myofascial Pain Syndrome. In Pain Medicine: An Interdisciplinary Case-Based Approach; Hayek, S.M., Shah, B.J., Desai, M.J., Chelimsky, T.C., Eds.; Oxford University Press: New York, NY, USA, 2015; pp. 69–88. ISBN 978-0-19-939082-3. Available online: https://oxfordmedicine.com/view/10.1093/med/9780199931484.001.0001/med-9780199931484 (accessed on 2 April 2021). [CrossRef]
  92. Clauw, D.J.; Arnold, L.M.; McCarberg, B.H. The science of fibromyalgia. Mayo Clin. Proc. 2011, 86, 907–911. [Google Scholar] [CrossRef] [Green Version]
  93. Wolfe, F.; Brähler, E.; Hinz, A.; Häuser, W. Fibromyalgia Prevalence, Somatic Symptom Reporting, and the Dimensionality of Polysymptomatic Distress: Results From a Survey of the General Population. Arthritis Care Res. 2013, 65, 777–785. [Google Scholar] [CrossRef] [PubMed]
  94. Neumann, L.; Buskila, D. Epidemiology of fibromyalgia. Curr. Pain Headache Rep. 2003, 7, 362–368. [Google Scholar] [CrossRef]
  95. Zhu, W.; He, X.; Cheng, K.; Zhang, L.; Chen, D.; Wang, X.; Qiu, G.; Cao, X.; Weng, X. Ankylosing spondylitis: Etiology, pathogenesis, and treatments. Bone Res. 2019, 7, 1–16. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  96. Goh, E.L.; Chidambaram, S.; Ma, D. Complex regional pain syndrome: A recent update. Burns Trauma 2017, 5. [Google Scholar] [CrossRef] [Green Version]
  97. Lamichhane, B.; Jayasekera, D.; Jakes, R.; Glasser, M.F.; Zhang, J.; Yang, C.; Grimes, D.; Frank, T.L.; Ray, W.Z.; Leuthardt, E.C.; et al. Multi-modal biomarkers of low back pain: A machine learning approach. NeuroImage Clin. 2021, 29, 102530. [Google Scholar] [CrossRef]
  98. Santana, A.N.; de Santana, C.N.; Montoya, P. Chronic Pain Diagnosis Using Machine Learning, Questionnaires, and QST: A Sensitivity Experiment. Diagnostics 2020, 10, 958. [Google Scholar] [CrossRef]
  99. Santana, A.N.; Cifre, I.; De Santana, C.N.; Montoya, P. Using Deep Learning and Resting-State fMRI to Classify Chronic Pain Conditions. Front. Neurosci. 2019, 13, 1313. [Google Scholar] [CrossRef]
  100. Goldberg, D.S.; McGee, S.J. Pain as a global public health priority. BMC Public Health 2011, 11, 770. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Flow diagram of the review process. Of 287 identified research papers, thirty-nine were included in the review (PubMed n = 31, IEEE Xplore n = 6, ACM Digital Library n = 2). In addition, fourteen papers from the reference lists were included in the review.
Figure 1. Flow diagram of the review process. Of 287 identified research papers, thirty-nine were included in the review (PubMed n = 31, IEEE Xplore n = 6, ACM Digital Library n = 2). In addition, fourteen papers from the reference lists were included in the review.
Applsci 11 03205 g001
Table 1. Decisions: Classifying using health data, classifying using text and images, genomics approaches and biomarker identification, treatment, self-management, and measuring pain intensity.
Table 1. Decisions: Classifying using health data, classifying using text and images, genomics approaches and biomarker identification, treatment, self-management, and measuring pain intensity.
StudyYearConditionData SizeData SourcesML Methods
Classifying patients
using structured health data
Melidis [26]2018IBS, FM, CFS1751Symptom frequency questionnaireAdaptive network model, k-means clustering, NN
Davis et al. [27]2018FM2529De-identified clinic-level dataK-means clustering, principal component analysis, discriminant analysis
Lin et al. [28]2018MPS56SEMGTemplate matching, k-means clustering
Oliviero and Poli-Neto [29]2017CPP346A database of diagnosed cases containing the full history of the patient. Seven different conditions related to pelvic pain were included in the data.Naive Bayes, C4.5, SVM, AdaBoost, kNN, Weka software [30]
Yim et al. [31]2017FM313Demographic characteristicsClustering
Emir et al. [32]2015FM587,960Structured EHR data (Humedica database)RF
Garcia-Zapirain et al. [33]2015FM127Psychopathological and medical-social features, VASAdaBoost
Olugbade et al. [34]2014CLBP31Movements, muscle activity characteristics, SEMG (EmoPaindataset)One-way ANOVA, SVM
Chan [35]2013CLBP40Gait datakStar, SVM, MLP, DT
Yang et al. [36]2012CRPS20Gait dataMLP, SVM, RF, LDA, KStar
Goertzel et al. [37]2006CFS103Clinical dataNonlinear regression
Classifying patients using
text and images
Fodeh [38]2018Musculoskeletal pain92Clinical notes from EHRkNN, DT, SVM, RF
Tan [39]2018CLBP871Lumbar spine imaging, radiology reportsNatural language processing
Zhong [40]2018TN46sMRISVM
Cheng [41]2018AS133Questionnaire, fMRIMultiple linear regression
Rogachov et al. [42]2018AS133fMRIPRoNTo [43]
Mano [44]2018CLBP97fMRI (+test data from OpenPain project)SVM, conditional variational autoencoder
Sevel [45]2018CFS33sMRI and PROMSVM
Lopez-Sola [46]2017FM72fMRISVM
Qin et al. [47]2016CLBP22Body movement and EMG (EmoPain dataset)RF, kNN, hidden Markov model
Olugbade et al. [48]2015CLBP53Motion capture, EMG, video, audio, and Hospital Anxiety and Depression Scale (EmoPain dataset)SVM, RF
Labus [49]2015IBS212sMRIMVPA
Robinson et al. [50]2015FM26sMRI, PROM (mood and pain intensity)Naive Bayes, logistic regression, kNN, MLP, SVM, DT
Sundermann [51]2014FM, RA50fMRI, VASMVPA, SVM, DT, RF, naive Bayes, LDA, MLP
Callan et al. [52]2014CLBP26fMRISparse logistic regression
Bagarinao [53]2014CPP66sMRISVM
Ung [54]2012CLBP94sMRISVM
Genomics approaches and biomarker identification
Lukkahatai et al. [55]2018FM47Gene expression dataFisher’s ratio
Chancellor et al. [56]2018IC448Urine samples, questionnairesSupervised learning
Ultsch et al. [57]2016Chronic pain535 genesPain genesctional genomics analysis
Braundmeier-Fleming et al. [58]2016IC34DNA from stool and vaginal samplesextended random forest
Treatment
Golabchi [59]2019CLBP62SEMG, VASRF, linear regression
Sabeti [60]2019FM20PROM and patient-generated data (wristband sensors)kNN, AdaBoost, SVM, RF, LDA, LUCCK
Nijeweme-d’Hollosy [61]2018CLBP1326Age, well-being, pain descriptionDT, RF, and boosted tree
McKernan [62]2018FM15,945EHRPenalized regression
Jiang [63]2017CLBP78SEMGSVM
Aung et al. [64]2015CLBP22The EmoPain dataset (anthropometric measurements, facial expressions, movement behaviours)SVM, RF
Aung et al. [65]2014CLBP21Sit-to-stand and one-leg standRF
Aung et al. [66]2013CLBP21The EmoPain datasetRF
Zhang [25]2013Chronic neck pain794Patients recordsDT, non-dominated sort
Ghosh [67]2012CLBP53MRISVM
Dickey [68]2002CLBP93D vertebral motionLDA and NN
Self-management
Jamison et al. [69]2019Chronic pain144Pain Catastrophizing Scale, Pain Disability Index, Hospital Anxiety and Depression Scale, VASMultilevel AR (1) modelling, SAS procedure [70], χ 2 -test, logistic regression
Meier [71]2018CLBP20MRI, PROM (pain-related fear)MVPA, PRoNTo [43]
Rahman [72]2018Chronic pain28,952Patient-generated data (mobile app: pain data)Logistic regression, RF, SVM, k-means clustering
Rabbi et al. [73]2018CLBP10Patient-generated data (mobile app: sensor-based and self-reported physical activity, and self-reported painRL
Huang et al. [74]2011Chronic pain187 [75]PROMSVM
Huang [76]2009Chronic pain187 [75]PROMDT, naive Bayes, SVM, MLP
Measuring pain intensity
Lee et al. [77]2019CLBP71Exacerbation manoeuvres, fMRISVM, SVR
Jaiswal et al. [78]2018Shoulder pain25Facial expression videosCNN
Hammal and Cohn [79]2012Shoulder pain129Facial expression videos, PROMSVM, active appearance model
Kaltwang et al. [80]2012Shoulder pain25Facial expression videosRelevance vector regression, active appearance model
Lucey [81]2011Shoulder pain129Facial expression videosSVM
Table 2. Datasets available for academic use.
Table 2. Datasets available for academic use.
TitleParticipantsData Types
UNBC-McMaster Shoulder Pain Expression Archive Database [81]129 patients with shoulder pain (66 female and 63 male)Video of facial expressions while performing range-of-motion tests. Frames are coded with self-reported and observer-reported ratings of pain.
EmoPain dataset [64]22 patients with CLBP (15 female, 7 male) and 28 healthy controlsA fully labelled multimodal dataset containing face videos, 3D motion capture, and EMG signals from back muscles. Pain-related facial expressions and body movement behaviours were elicited from people with chronic pain performing physical exercises.
OpenPain dataset [44] fMRI data of patients with CLBP
Optum Humedica database [83]>150 million peopleDe-identified EHR data and claims data, including diagnoses, treatments, pharmacy and over-the-counter prescriptions, text notes, etc.
Table 3. Applications of machine learning studies on different types of chronic pain (prevalence sources: [86,87,88,89,90,91]).
Table 3. Applications of machine learning studies on different types of chronic pain (prevalence sources: [86,87,88,89,90,91]).
ConditionPrevalenceDiagnosisClinical Decision SupportRehabilitationSelf-Management
Myofascial Pain Syndrome (MPS)9%–20%[28]
Chronic Musculoskeletal Pain11.4%–24%[38]
Chronic Pelvic Pain (CPP)4%–24%[29,53]
Chronic Low Back Pain (CLBP)5%–20%[39,44,52,54,77][59,61,67,68][34,35,48,63,64,65,66][47,71,73]
Chronic Shoulder Pain15.1%[78,79,80,81]
Irritable Bowel Syndrome (IBS)11%[26,49]
Interstitial Cystitis/Bladder Pain Syndrome (IC)2.7%–6.53%[56,58]
Fibromyalgia (FM)2%–5%[26,27,31,32,33,46,50,51,55][27,60,62]
Chronic Fatigue Syndrome (CFS)0.76%–3.28 %[26,37,45]
Chronic Neck Pain2.2% [25]
Rheumatoid Arthritis (RA)0.41%–0.54%[51]
Ankylosing Spondylitis (AS)0.07%–0.32%[41,42]
Trigeminal Neuralgia (TN)0.03%–0.3%[40]
Complex Regional Pain Syndrome (CRPS)0.025% [36]
Unspecified Chronic Pain [57] [69,72,74,76]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Jenssen, M.D.K.; Bakkevoll, P.A.; Ngo, P.D.; Budrionis, A.; Fagerlund, A.J.; Tayefi, M.; Bellika, J.G.; Godtliebsen, F. Machine Learning in Chronic Pain Research: A Scoping Review. Appl. Sci. 2021, 11, 3205. https://doi.org/10.3390/app11073205

AMA Style

Jenssen MDK, Bakkevoll PA, Ngo PD, Budrionis A, Fagerlund AJ, Tayefi M, Bellika JG, Godtliebsen F. Machine Learning in Chronic Pain Research: A Scoping Review. Applied Sciences. 2021; 11(7):3205. https://doi.org/10.3390/app11073205

Chicago/Turabian Style

Jenssen, Marit Dagny Kristine, Per Atle Bakkevoll, Phuong Dinh Ngo, Andrius Budrionis, Asbjørn Johansen Fagerlund, Maryam Tayefi, Johan Gustav Bellika, and Fred Godtliebsen. 2021. "Machine Learning in Chronic Pain Research: A Scoping Review" Applied Sciences 11, no. 7: 3205. https://doi.org/10.3390/app11073205

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop