Next Article in Journal
Tentacle Mesh for Fixation-Free Spigelian Hernia Repair: Mini-Invasive Approach Granting Broad Defect Overlap
Previous Article in Journal
Left Atrial Appendage Occlusion in Patients with Failure of Antithrombotic Therapy: Good Vibes from Early Studies
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 12
Issue 11
10.3390/jcm12113865
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Consideration of Psychosocial Factors in Acute Low Back Pain by Physical Therapists

by
Emilia Otero-Ketterer
1,2,
Cecilia Peñacoba-Puente
3,
Ricardo Ortega-Santiago
4,5,*,
Fernando Galán-del-Río
5 and
Juan Antonio Valera-Calero
6,7
1
Escuela Internacional de Doctorado, Universidad Rey Juan Carlos, 28922 Alcorcón, Spain
2
Physiotherapy Department, Mutua Universal Mugenat, 28801 Alcalá de Henares, Spain
3
Department of Psychology, Universidad Rey Juan Carlos, 28922 Alcorcón, Spain
4
Cátedra Institucional en Docencia, Clínica e Investigación en Fisioterapia: Terapia Manual, Punción Seca y Ejercicio Terapéutico, Universidad Rey Juan Carlos, 28922 Alcorcón, Spain
5
Department of Physical Therapy, Occupational Therapy, Physical Medicine and Rehabilitation, Universidad Rey Juan Carlos, 28922 Alcorcón, Spain
6
Department of Radiology, Rehabilitation and Physiotherapy, Faculty of Nursery, Physiotherapy and Podiatry, Universidad Complutense de Madrid, 28040 Madrid, Spain
7
Grupo InPhysio, Instituto de Investigación Sanitaria del Hospital Clínico San Carlos (IdISSC), 28040 Madrid, Spain
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2023, 12(11), 3865; https://doi.org/10.3390/jcm12113865
Submission received: 10 March 2023 / Revised: 24 May 2023 / Accepted: 2 June 2023 / Published: 5 June 2023
(This article belongs to the Section Clinical Rehabilitation)
Versions Notes

Abstract

:
Clinical guidelines consistently recommend screening psychosocial (PS) factors in patients with low back pain (LBP), regardless of its mechanical nature, as recognized contributors to pain chronicity. However, the ability of physiotherapists (PTs) in identifying these factors remains controversial. This study aimed to assess the current identification of psychosocial risk factors by physical therapists (PTs) and which characteristics of PTs are associated with the identification of the main risk for chronicity (physical or psychosocial). A cross-sectional descriptive study surveying Spanish PTs in public and private health services was conducted, including questions on PT characteristics and three low back pain (LBP) patient vignettes with different biopsychosocial (BPS) clinical presentations. From 484 respondents, the majority of PTs agreed regarding the main risk for chronicity for each vignette (PS 95.7% for vignette A, PS and physical 83.5% for vignette B and PS 66% for vignette C). Female PTs were more likely to rate psychosocial compared with males (p < 0.05). PTs with higher levels of social and emotional intelligence (both, p < 0.05) were more likely to identify the main risk for chronicity. However, only gender and social information processing for vignette A (p = 0.024) and emotional clarity for vignette B (p = 0.006) were able to predict the identification of psychosocial and physical risk, respectively. The main risk for chronicity was correctly identified by a large majority of PTs through patient vignettes. Gender, social and emotional intelligence played a relevant role in the recognition of psychosocial risk and biopsychosocial factors.

1. Introduction

The paradoxical increase in low back pain (LBP) disability during the application of a tissue-centered biomedical approach generated the proposal of the biopsychosocial (BPS) model [1,2,3,4,5]. This model was based on the recognition of the biological, psychological and social interactions within the clinical evaluation and treatment of the disease [6], being adopted by the World Health Organization in the International Classification of Functioning, Disability and Health in 2000 [7] up to the present. Since its conception, the BPS model distinguished low back pain from low back disability according to the presence of patients with similar findings in imaging tests of the lumbar spine and with disparate levels of disability due to pain [6].
Although approximately 90% of cases of LBP are not associated with a precise pathoanatomical diagnosis [8], its inexistence cannot be stated [9]. This condition has been commonly referred to as non-specific LBP [10] and currently termed nociplastic, referring to the fact that there may be altered nociception [11,12]. Therefore, the differentiation made by the BPS model between the source of pain and the disability it generates is still relevant. Biological, psychological and social aspects, context, and classical and operant behavioral conditioning [13,14] shape, to a greater or lesser extent, the individual experience of pain [15,16]. Thus, psychosocial (PS) factors play a role in the clinical presentation of patients and are known predictors of the development and maintenance of chronic pain [17,18,19,20], being even more determinant aspects than somatosensory and motor alterations in LBP individuals [21,22].
Therefore, early identification of patients at risk of poor prognosis for recovery is critical to target modifiable factors and minimize the risk of chronicity of symptoms [23]. Clinical guidelines consistently recommend screening for PS factors in all patients with LBP, regardless of its mechanical nature [24,25,26], using validated patient-reported outcome measures in combination with clinical judgment [27,28]. Research reflects contradictory findings between the poor ability of physical therapists (PTs) to identify individual constructs such as fear avoidance, kinesiophobia, or emotional distress [29,30,31] as well as to allocate LBP patients into risk stratification groups [32] and those suggesting their adequate intuition in this regard [33,34,35].
Despite the clinical relevance of PT ability to detect PS factors in LBP patients, studies on this issue related to the sociodemographic or professional characteristics of PTs are scarce, and to the best of our knowledge, there are no studies carried out in Spain. Furthermore, although emotional and social intelligence are valuable assets for health care professionals [36,37] as they deal with people who are under emotional stress [38], being increasingly important for good physical therapist–patient interaction [39], there are no known studies assessing their influence on the detection of PS factors in LBP. A better understanding in this regard can lead to its inclusion in the development of educational training aimed at the identification and management of the psychosocial status of patients.
Therefore, the objective of this study is to assess the current identification of psychosocial risk factors by PTs. In addition, we aim to explore whether sociodemographic (age and gender), professional (years of clinical experience, work setting and postgraduate training in PS factors) and attitudinal aspects related to social and emotional intelligence of PTs are associated with the identification of the main risk for chronicity (physical or psychosocial).

2. Materials and Methods

This is a descriptive, cross-sectional study using an online survey. The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) has been followed [40] and the study protocol was revised and approved by the Clinical Ethics Committee of Hospital Clínico San Carlos (ID: 21/257). An informed consent form was drafted containing information about the expected time of the survey (approximately 20 min), data storage, researchers’ information and the study aim. After reading and signing the informed consent form, as well as meeting the inclusion criteria (e.g., being a regulated PT and having treated at least one case of acute LBP in the last year), participants were eligible. No incentives were offered to provide the survey results.

2.1. Study Population

The survey was based on questions about the characteristics of PTs in terms of their identification of the main risk for chronicity (physical or psychosocial) in three patient vignettes (case scenarios) with acute LBP. Thus, PTs in Public Health Service, Mutual Health Insurance Companies collaborating with the Public Health Service, and private practice from multiple regions of Spain were invited to participate through online announcements during 2020. To meet sample size requirements and above the recommendation of collecting 10 events (cases) for each predictor variable [41] (i.e., 70 respondents considering the 7 predictor variables in our study), we aimed to obtain at least 100 cases for each physical therapy work setting (a total of 300 cases) in order to provide the most adequate descriptive information on the identification of PS factors.

2.2. Study Development

The open survey instrument was approached in three phases. In the first stage, the patient vignettes were constructed reflecting different BPS clinical presentations: vignette A (high psychosocial risk); vignette B (high physical risk); and vignette C (high physical and psychosocial risk). Information on clinical history, symptoms, relevant physical findings and PS status was included. These case scenarios were validated by a group of national experts, selected by the authors on the basis of their clinical and research trajectory. Two expert psychologists in pain, as well as two rehabilitation physicians and three PTs with expertise in the treatment of musculoskeletal pain rated the main risk for chronicity in each vignette. There was agreement from all experts on the rating except two experts who commented on a higher physical risk in vignette A, so minor corrections were made to that vignette to aid differentiation with the main PS risk. Then, the principal investigator developed a draft released on Google Forms (https://docs.google.com/forms/) (accessed on 12 September 2022) that included the questions (one per page) on PT characteristics and patient vignettes, which was tested with a small sample of 15 participants for potential comprehension, usability, and technical functionality issues prior to initiating the research. Participants had not enabled the option for review and change their answers. As no problems were found, the final version was published openly to initiate data collection.

2.3. Outcome Variables

2.3.1. Screening Questionnaire for Psychosocial Factors in LBP

Following the procedure described above, the ad hoc instrument relating to the detection of PS factors in LBP patients consisted of three items (patient vignettes) with the distribution: vignette A (high psychosocial risk); vignette B (high physical risk); and vignette C (high physical and psychosocial risk). In each of these, the physical therapist was asked to answer the next question: “Of the following types of factors, which one contributes most to this patient’s risk of chronicity? Physical or Psychosocial”.

2.3.2. Sociodemographic and Professional Variables

Sociodemographic factors (age and gender) and professional factors (years of clinical experience treating musculoskeletal disorders, work setting and postgraduate training in PS factors) were assessed.

2.3.3. Emotional and Social Intelligence

To assess social intelligence, understood as the ability to communicate and form relationships with empathy and assertiveness, we used the Social Intelligence Questionnaire [42] which is composed of 34 items, with a five-point Likert scale. The scale allows the assessment of five dimensions of social intelligence: self-monitoring, social sensitivity, social information processing, social skills and social awareness (Cronbach’s alpha = 0.78–0.82) [42].
The Spanish version [43] of the Trait Meta Mood Scale (TMMS-24) was used for assessing emotional intelligence, which has shown high internal consistency and satisfactory test–retest reliability [44]. This scale is composed of 24 items, with a five-point Likert scale and allows the assessment of three dimensions of emotional intelligence: Emotional attention, clarity and repair (Cronbach’s alpha = 0.80–0.86) (see Supplementary File S1 for checking the survey questionnaire form) [44].

2.4. Statistical Analysis

Data from Google Forms were transferred to the SPSS statistical package (version 27.0) for Mac OS to be analyzed. First, a Kolmogorov–Smirnov test was used to determine the normal distribution of the data. Later, all PT characteristics were analyzed using descriptive statistics and crosstabulations. First, bivariate analyzes were carried out. Specifically, differences in the rating of the main chronicity risk were explored using the unpaired Student t-test for the quantitative variables and Pearson chi-square test for the qualitative variables. Considering the variables that had been significant in the bivariate analyzes, a stepwise multiple regression was calculated to assess which variable (sociodemographic, professional and social and emotional intelligence) best predicts the identification of the main risk factor for chronicity [45]. All statistical analyses were conducted setting a 95% confidence level, and a significance level of p < 0.05 to be considered statistically significant.

3. Results

Four hundred and eighty-four PTs completed the survey (67.1% female; age: 35 ± 8.2 years). Only 4 of 17 Official Associations of Physiotherapy did not publish the survey. Given that an unknown number of PTs viewed the announcements on the web pages and social media of their Official Associations, it was not possible to calculate the response rate obtained. Respondents’ sociodemographic, professional, and attitudinal characteristics are described in Table 1, Table 2 and Table 3, respectively, for the total study population and for the number of PTs who responded correctly to the main risk factor for chronicity in each vignette.
Table 4 reports agreement in the majority of PTs, with the main risk for chronicity stated for each of the patient vignettes—95.7% (n = 464) and 83.5% (n = 405) stated the main risk as psychosocial and physical for vignettes A and B, respectively, and 66% (n = 320) stating psychosocial risk for the vignette C.
The bivariate analyzes showed that in none of the three case scenarios were ratings of the main risk for chronicity associated with work setting or postgraduate training in PS factors (Table 2 and Table 3). However, there was a significant association on the gender variable for vignettes A and C, in which female PTs were more likely to rate the main risk of chronicity as psychosocial (ꭓ2 = 14.7, p < 0.001 and ꭓ2 = 3.8, p = 0.049, respectively), with no statistical difference found for vignette B (ꭓ2 = 0.78, p = 0.38). In addition, age and years of clinical experience in treating musculoskeletal disorders was associated with the rating for vignette C, in which older and more experienced PTs were more likely to state physical risk over psychosocial risk as the main risk factor (mean 36.6 ± 9, p = 0.016 and mean 13.1 ± 7.9, p = 0.004, respectively). Furthermore, there were significant associations on certain aspects of emotional and social intelligence for vignettes A and B, in which PTs with higher emotional attention, social sensitivity and social information processing were more likely to rate the main risk in vignette A as psychosocial (mean 27.1, SD ± 6, p = 0.012; mean 21.4, SD ± 3.6, p < 0.01 and mean 25.1, SD ± 4, p = 0.023, respectively), as well as PTs with higher emotional clarity and social information processing were more likely to rate the main risk in vignette B as physical (mean 30.2, SD ± 5.6, p = 0.006 and mean 25.2, SD ± 4, p = 0.018, respectively) (Table 2 and Table 3).
The results of the logistic regression analysis showed that only some variables, social information processing (B = −0.117, p = 0.024, Exp (B) = 0.89) and gender (B = −1.717, p = 0.001, Exp (B) = 0.180) for vignette A (R2 Cox y Snell = 0.37, p = 0.024) as well as emotional clarity (B = 0.058, p = 0.006, Exp (B) = 1.059) for vignette B (R2 Cox y Snell = 0.17, p = 0.006), remained significantly associated with the identification of the main risk for chronicity, psychosocial and physical, respectively, after the inclusion of gender as a covariate.

4. Discussion

This study is the first nationwide survey in Spain exploring the identification of PS risk factors by PTs working in the musculoskeletal field, framed within the adoption of the biopsychosocial model in patients with LBP. In addition, we analyzed whether different sociodemographic, professional and attitudinal characteristics of these health care professionals were associated with the detection of the main risk for chronicity, and which variables best predicted their identification, providing insights into the aspects that influence their recognition. Overall, the identification of the main risk for chronicity was properly carried out by a large majority of PTs through LBP patient vignettes.
The ability of PTs to rate chronicity risk according to PS aspects has been reported to be adequate for high and low risk cases, but not for moderate risk cases in a survey study also based on LBP patient vignettes [46]. Correct estimates of BPS risk of chronicity by PTs were also reported by comparing clinical assessments of actual LBP patients, with respect to patient responses on the Orebro Musculoskeletal Pain Questionnaire (OMPQ) [33,34], as well as from PTs who followed a BPS training program (35). However, Wassinger et al. recently reported slight agreement between the PTs’ ratings and OMPQ during the evaluation of musculoskeletal conditions [47]. Other studies have described difficulties of PTs in identifying individual constructs such as depression, fear avoidance and pain catastrophizing in patients with LBP [27,29,31,32]. Moreover, Beales et al. found inconsistent correlations for PT estimates of each of the OMPQ domains [33], although this inconsistency may have been influenced by the fact that such constructs were not measured using the established assessment tools and their cut-off points [31]. All these conflicts in findings may be partly explained by the methodology, either in relation to the use of actual patients versus patient vignettes or the use of different assessment tools. However, underreporting by some of these authors of PT characteristics that could have influenced their results has also been detected, such as those characteristics mentioned above [29,32,34,35].
Among the characteristics of PTs associated with the identification of PS factors, in our sample, female PTs better identified the main PS risk in patient vignette A than male PTs. In addition, female PTs were more likely to rate the main risk for chronicity in patient vignette C as psychosocial versus physical than male PTs and estimated the main physical risk of patient vignette B similarly to male PTs. These results suggest that the ability of female PTs was not biased toward the PS component, being better at identifying PS chronicity risk than male PTs. These findings support the results presented by a meta-analytic review showing that female physicians address psychosocial and emotional information about their patients during medical visits to a greater extent than male physicians, having found no differences in the handling of biomedical information [48].
Several aspects of the PTs’ social and emotional intelligence influenced the correct discrimination between the main physical and psychosocial risk for vignettes A and B. However, stepwise multiple regression analysis showed that only social information processing (component of social intelligence relating to how people develop and handle relationship information) and emotional clarity (component of emotional intelligence that allows us to integrate what we feel into our thinking and to be able to consider the complexity of emotional changes) remained significantly associated with their recognition. Emotional and social intelligence is the ability to understand and manage emotions in oneself and others [36]. Although the importance of these skills in meeting new communication challenges, transfer of knowledge available for clinical practice and patients’ education, among others, has recently been described [37], studies specifically analyzing how social and emotional intelligence influence the implementation of the BPS approach are lacking. PTs have traditionally focused on structural/biomechanical care, but the need for interventions directed at the BPS components of the individual experience of pain is becoming increasingly relevant [49] as they are ideally situated to identify barriers to recovery and reduce pain experiences by educating and addressing patients’ dysfunctional beliefs about pain [50,51,52]. Since there are ways to develop the social and emotional intelligence skills of health care professionals [53], incorporating these aspects in the development of interventions and trainings aimed at the BPS approach of PTs may also be useful to overcome the challenge of the traditional roles involved [36,54].
Formal education [55,56] as well as clinical experience and on-the-job learning [57] make up expertise in physiotherapy. We asked whether postgraduate training in PS aspects, older and more experienced PTs and work setting influenced recognition of PS factors, finding no statistically significant differences except for older and more experienced PTs, that were more likely to report main physical risk in patient vignette C. This significant finding could reflect some biomedical bias for more experienced PTs, although this trend did not imply a worse identification of PS risk in patient vignette A, similar to that reported by other authors [31]. In addition, although only one-third of PTs in our sample reported having received such PS training (29.5%, n = 143), our results could reflect the difficulty of changing behaviors in clinical practice after BPS training, reported by other studies [58,59,60]. On the other hand, it has been reported that PTs working in community orthopedic settings held attitudes toward LBP (HC-Pairs questionnaire) better aligned with best practice evidence than non-community orthopedic PTs [61]. The difference with our findings can be explained by the fact that the work settings considered in our study were all clinical environments.

Limitations

This study has several limitations. First, the vignettes tend not to represent a real clinical situation and also, our findings depended on participants’ self-reports, so we cannot infer our results to the identification of the main risk for chronicity by PTs in their daily clinical practice. However, some authors have noted the tendency of studies employing “real patients” to show favorable results for clinicians’ assessment of chronicity risk and/or identification of psychosocial factors compared to assessment instruments versus those employing videos or vignettes [33]. Variations in health care delivery by PTs toward LBP patients with PS risk factors are widespread. Such variations are driven by the pain-related attitudes and beliefs of these professionals compatible with a biomedical approach, sometimes stigmatizing these patients and feeling unprepared to treat them, resulting in negative and unhelpful consequences in the therapeutic encounter [62,63]. Obtaining useful information on the identification of PS chronicity risk for PTs within health care systems is a challenge, even more so when the situations to be assessed are complex or confront the traditional roles involved in the profession. Several validation studies have found that the use of vignettes reflects the stated intentions and behaviors of health professionals more closely than data extracted from recordings of actual consultations [64,65,66]. While both methods have strengths and weaknesses, well-designed vignette studies may have advantages in certain scenarios [67]. To this end, several steps were taken in the present study to address the quality criteria, including a description of the steps followed for vignette development, validation by an expert panel, and pretest, to explore participants’ likely judgments by presenting a series of hypothetical but realistic clinical scenarios, and to generate inferences about the relationships between the nature of each vignette and the PTs’ subsequent responses to them [68]. Moreover, although the case scenarios shown in this study do not represent the wide variety of PS aspects that may be present in patients with LBP, and focused more on the psychological emotional and social components of pain, recent evidence has highlighted less attention to such aspects when adopting the biopsychosocial model in physical therapy settings compared to cognitive and behavioral ones [69], so an analysis focused on how their identification by PTs is performed is currently relevant. Additionally, in the present study we explicitly asked the participants to select between the main risk of chronicity, physical or psychosocial. Although this could have been oriented towards one of these two response options, all the scenarios of LBP cases shown presented a physical component to a greater or lesser degree, which allowed us to analyze the PTs’ evaluation of its relevance with respect to chronicity and the existence of a possible bias towards the physical aspects of pain, even though the PS component was present (vignette A). In addition, a third vignette (vignette C) was presented, with a high risk of both physical and psychosocial chronicity, which allowed us to contrast the possible existence of biases in the identification of the physical and psychosocial aspects in the responses shown to vignettes A and B. Second, given that in this study it was not possible to calculate the response rate and that respondents could be interested in PS factors, PTs’ abilities to identify PS risk may be biased. However, even if non-response was not random, in survey studies, non-response does not appear to generate significant response bias [70]. Finally, the participants in our sample had on average a high level of clinical experience, so we cannot generalize the results to more recently qualified PTs.

5. Conclusions

The main risk for chronicity was correctly identified by most of PTs. Our findings suggest that female PTs were better at identifying PS chronicity risk than male PTs and several aspects of the PTs’ social and emotional intelligence also influenced the correct distinction between the main physical and psychosocial risk. However, only gender and social information processing as well as emotional clarity were statistically significant in predicting accurate identification of the main risk of psychosocial and physical chronicity, respectively. Future research may analyze whether incorporating these attitudinal aspects into training strategies improves PT recognition in clinical practice and patient outcomes.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm12113865/s1, Supplementary File S1 contains the survey used in this study.

Author Contributions

Conceptualization, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; methodology, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; software, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; validation, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; investigation, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; resources, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; data curation, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; writing—original draft preparation, E.O.-K. writing—review and editing, J.A.V.-C.; visualization, E.O.-K., C.P.-P., R.O.-S., F.G.-d.-R. and J.A.V.-C.; supervision, C.P.-P.; project administration, R.O.-S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Clinical Ethics Committee of Hospital Clínico San Carlos (ID: 21/257).

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study.

Data Availability Statement

All data derived from this study are presented in the text.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018, 392, 1789–1858, Erratum in Lancet 2019, 393, e44.. [Google Scholar] [CrossRef] [Green Version]
  2. Da Menezes Costa, L.C.; Maher, C.G.; Hancock, M.J.; McAuley, J.H.; Herbert, R.D.; Costa, L.O. The prognosis of acute and persistent low-back pain: A meta-analysis. Can. Med. Assoc. J. 2012, 184, E613–E624. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Mills, K.; Brown, B.T.; Herbert, R.D.; Maher, C.G.; Hancock, M.J. Risk of Recurrence of Low Back Pain: A Systematic Review. J. Orthop. Sports Phys. Ther. 2017, 47, 305–313. [Google Scholar] [CrossRef]
  4. Hartvigsen, J.; Hancock, M.J.; Kongsted, A.; Louw, Q.; Ferreira, M.L.; Genevay, S.; Hoy, D.; Karppinen, J.; Pransky, G.; Sieper, J.; et al. What low back pain is and why we need to pay attention. Lancet 2018, 391, 2356–2367. [Google Scholar] [CrossRef] [Green Version]
  5. Alonso-García, M.M.; Sarría-Santamera, A.M. The Economic and Social Burden of Low Back Pain in Spain: A National Assessment of the Economic and Social Impact of Low Back Pain in Spain. Spine 2020, 45, E1026–E1032. [Google Scholar] [CrossRef]
  6. Waddell, G. 1987 Volvo Award in Clinical Sciences: A New Clinical Model for the Treatment of Low-Back Pain. Spine 1987, 12, 632–644. [Google Scholar] [CrossRef]
  7. World Health Organization. The International Classification Functioning, Disability and Health; World Health Organization: Geneva, Switzerland, 2000; Available online: https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health (accessed on 20 November 2022).
  8. Maher, C.; Underwood, M.; Buchbinder, R. Non-specific low back pain. Lancet 2017, 389, 736–747. [Google Scholar] [CrossRef] [Green Version]
  9. DePalma, M.J.; Ketchum, J.M.; Saullo, T. What Is the Source of Chronic Low Back Pain and Does Age Play a Role? Pain Med. 2011, 12, 224–233. [Google Scholar] [CrossRef] [Green Version]
  10. Airaksinen, O.; Brox, J.I.; Cedraschi, C.; Hildebrandt, J.; Klaber-Moffett, J.; Kovacs, F.; Mannion, A.F.; Reis, S.; Staal, J.B.; Ursin, H.; et al. Chapter 4 European guidelines for the management of chronic nonspecific low back pain. Eur. Spine J. 2006, 15, s192–s300. [Google Scholar] [CrossRef] [Green Version]
  11. Knezevic, N.N.; Candido, K.D.; Vlaeyen, J.W.S.; Van Zundert, J.; Cohen, S.P. Low back pain. Lancet 2021, 398, 78–92. [Google Scholar] [CrossRef]
  12. Treede, R.-D.; Rief, W.; Barke, A.; Aziz, Q.; Bennett, M.I.; Benoliel, R.; Cohen, M.; Evers, S.; Finnerup, N.B.; First, M.B.; et al. Chronic pain as a symptom or a disease: The IASP Classification of Chronic Pain for the International Classification of Diseases (ICD-11). Pain 2019, 160, 19–27. [Google Scholar] [CrossRef] [Green Version]
  13. Meulders, A. Fear in the context of pain: Lessons learned from 100 years of fear conditioning research. Behav. Res. Ther. 2020, 131, 103635. [Google Scholar] [CrossRef]
  14. Vlaeyen, J.W.; Crombez, G.; Linton, S.J. The fear-avoidance model of pain. Pain 2016, 157, 1588–1589. [Google Scholar] [CrossRef] [Green Version]
  15. De Ridder, D.; Adhia, D.; Vanneste, S. The anatomy of pain and suffering in the brain and its clinical implications. Neurosci. Biobehav. Rev. 2021, 130, 125–146. [Google Scholar] [CrossRef]
  16. Raja, S.N.; Carr, D.B.; Cohen, M.; Finnerup, N.B.; Flor, H.; Gibson, S.; Keefe, F.J.; Mogil, J.S.; Ringkamp, M.; Sluka, K.A.; et al. The revised International Association for the Study of Pain definition of pain: Concepts, challenges, and compromises. Pain 2020, 161, 1976–1982. [Google Scholar] [CrossRef]
  17. Edwards, R.R.; Dworkin, R.H.; Sullivan, M.D.; Turk, D.C.; Wasan, A.D. The Role of Psychosocial Processes in the Development and Maintenance of Chronic Pain. J. Pain 2016, 17, T70–T92. [Google Scholar] [CrossRef] [Green Version]
  18. Hruschak, V.; Cochran, G. Psychosocial predictors in the transition from acute to chronic pain: A systematic review. Psychol. Health Med. 2018, 23, 1151–1167. [Google Scholar] [CrossRef]
  19. Martinez-Calderon, J.; Flores-Cortes, M.; Morales-Asencio, J.M.; Luque-Suarez, A. Which Psychological Factors Are Involved in the Onset and/or Persistence of Musculoskeletal Pain? An Umbrella Review of Systematic Reviews and Meta-Analyses of Prospective Cohort Studies. Clin. J. Pain 2020, 36, 626–637. [Google Scholar] [CrossRef]
  20. Otero-Ketterer, E.; Peñacoba-Puente, C.; Pinheiro-Araujo, C.F.; Valera-Calero, J.A.; Ortega-Santiago, R. Biopsychosocial Factors for Chronicity in Individuals with Non-Specific Low Back Pain: An Umbrella Review. Int. J. Environ. Res. Public Health 2022, 19, 10145. [Google Scholar] [CrossRef]
  21. Klyne, D.M.; Moseley, G.L.; Sterling, M.; Barbe, M.F.; Hodges, P.W. Are Signs of Central Sensitization in Acute Low Back Pain a Precursor to Poor Outcome? J. Pain 2019, 20, 994–1009. [Google Scholar] [CrossRef] [Green Version]
  22. Nieto-García, J.; Suso-Martí, L.; La Touche, R.; Grande-Alonso, M. Somatosensory and Motor Differences between Physically Active Patients with Chronic Low Back Pain and Asymptomatic Individuals. Medicina 2019, 55, 524. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Lewis, J.; O’Sullivan, P. Is it time to reframe how we care for people with non-traumatic musculoskeletal pain? Br. J. Sports Med. 2018, 52, 1543–1544. [Google Scholar] [CrossRef] [PubMed]
  24. Chou, R.; Deyo, R.; Friedly, J.; Skelly, A.; Hashimoto, R.; Weimer, M.; Fu, R.; Dana, T.; Kraegel, P.; Griffin, J.; et al. Nonpharmacologic Therapies for Low Back Pain: A Systematic Review for an American College of Physicians Clinical Practice Guideline. Ann. Intern. Med. 2017, 166, 493–505. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Nijs, J.; Apeldoorn, A.; Hallegraeff, H.; Clark, J.; Smeets, R.; Malfliet, A.; Girbes, E.L.; De Kooning, M.; Ickmans, K. Low back pain: Guidelines for the clinical classification of predominant neuropathic, nociceptive, or central sensitization pain. Pain Physician 2015, 18, E333–E346. [Google Scholar] [CrossRef]
  26. Van Tulder, M.; Becker, A.; Bekkering, T.; Breen, A.; Gil del Real, M.T.; Hutchinson, A.; Koes, B.; Laerum, E.; Malmivaara, A.; On behalf of the COST B13 Working Group on Guidelines for the Management of Acute Low Back Pain in Primary Care. Chapter 3. European guidelines for the management of acute nonspecific low back pain in primary care. Eur. Spine J. 2006, 15, s169–s191. [Google Scholar] [CrossRef] [Green Version]
  27. Karran, E.L.; McAuley, J.H.; Traeger, A.C.; Hillier, S.L.; Grabherr, L.; Russek, L.N.; Moseley, G.L. Can screening instruments accurately determine poor outcome risk in adults with recent onset low back pain? A systematic review and meta-analysis. BMC Med. 2017, 15, 13, Erratum in BMC Med. 2017, 15, 44. [Google Scholar] [CrossRef] [Green Version]
  28. Pauli, J.; Starkweather, A.; Robins, J.L. Screening Tools to Predict the Development of Chronic Low Back Pain: An Integrative Review of the Literature. Pain Med. 2019, 20, 1651–1677. [Google Scholar] [CrossRef]
  29. Calley, D.Q.; Jackson, S.; Collins, H.; George, S.Z. Identifying Patient Fear-Avoidance Beliefs by Physical Therapists Managing Patients with Low Back Pain. J. Orthop. Sport. Phys. Ther. 2010, 40, 774–783. [Google Scholar] [CrossRef] [Green Version]
  30. Haggman, S.; Maher, C.G.; Refshauge, K.M. Screening for Symptoms of Depression by Physical Therapists Managing Low Back Pain. Phys. Ther. 2004, 84, 1157–1166. [Google Scholar] [CrossRef]
  31. Miki, T.; Kondo, Y.; Takebayashi, T.; Takasaki, H. Difference between physical therapist estimation and psychological patient-reported outcome measures in patients with low back pain. PLoS ONE 2020, 15, e0227999. [Google Scholar] [CrossRef]
  32. Brunner, E.; Dankaerts, W.; Meichtry, A.; O’sullivan, K.; Probst, M. Physical Therapists’ Ability to Identify Psychological Factors and Their Self-Reported Competence to Manage Chronic Low Back Pain. Phys. Ther. 2018, 98, 471–479. [Google Scholar] [CrossRef] [Green Version]
  33. Beales, D.; Kendell, M.; Chang, R.; Håmsø, M.; Gregory, L.; Richardson, K.; O’Sullivan, P. Association between the 10 item Örebro Musculoskeletal Pain Screening Questionnaire and physiotherapists’ perception of the contribution of biopsychosocial factors in patients with musculoskeletal pain. Man. Ther. 2016, 23, 48–55. [Google Scholar] [CrossRef]
  34. Dagfinrud, H.; Storheim, K.; Magnussen, L.; Ødegaard, T.; Hoftaniska, I.; Larsen, L.; Ringstad, P.; Hatlebrekke, F.; Grotle, M. The predictive validity of the Örebro Musculoskeletal Pain Questionnaire and the clinicians’ prognostic assessment following manual therapy treatment of patients with LBP and neck pain. Man. Ther. 2013, 18, 124–129. [Google Scholar] [CrossRef]
  35. Fersum, K.V.; O’Sullivan, P.; Kvåle, A.; Skouen, J. Inter-examiner reliability of a classification system for patients with non-specific low back pain. Man. Ther. 2009, 14, 555–561. [Google Scholar] [CrossRef]
  36. Grunberg, N.E.; McManigle, J.E.; Barry, E.S. Using Social Psychology Principles to Develop Emotionally Intelligent Healthcare Leaders. Front. Psychol. 2020, 11, 1917. [Google Scholar] [CrossRef]
  37. Pérez-Fuentes, M.D.C.; Molero Jurado, M.D.M.; Gázquez Linares, J.J.; Oropesa Ruiz, N.F. The Role of Emotional Intelligence in Engagement in Nurses. Int. J. Environ. Res. Public Health 2018, 15, 1915. [Google Scholar] [CrossRef]
  38. Andal, S. Emotional Intelligence among Health Care Professionals. IOSR J. Nurs. Health Sci. 2021, 10, 35–39. [Google Scholar]
  39. Gard, G.; Gyllensten, A.L. The Importance of Emotions in Physiotherapeutic Practice. Phys. Ther. Rev. 2000, 5, 155–160. [Google Scholar] [CrossRef]
  40. Eysenbach, G. Improving the quality of Web surveys: The Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J. Med. Internet Res. 2004, 6, e34, Erratum in J. Med. Internet Res. 2012, 14, e8. [Google Scholar] [CrossRef]
  41. Vittinghoff, E.; McCulloch, C.E. Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression. Am. J. Epidemiol. 2007, 165, 710–718. [Google Scholar] [CrossRef] [Green Version]
  42. Zautra, E.K.; Zautra, A.J.; Gallardo, C.E.; Velasco, L. Can We Learn to Treat One Another Better? A Test of a Social Intelligence Curriculum. PLoS ONE 2015, 10, e0128638. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Fernandez-Berrocal, P.; Extremera, N.; Ramos, N. Validity and Reliability of the Spanish Modified Version of the Trait Meta-Mood Scale. Psychol. Rep. 2004, 94, 751–755. [Google Scholar] [CrossRef]
  44. Salovey, P.; Mayer, J.D. Emotional Intelligence. Imagin. Cogn. Personal. 1990, 10, 185–211. [Google Scholar] [CrossRef]
  45. Stoltzfus, J.C. Logistic Regression: A Brief Primer. Acad. Emerg. Med. 2011, 18, 1099–1104. [Google Scholar] [CrossRef]
  46. Bishop, A.; Foster, N.E. Do Physical Therapists in the United Kingdom Recognize Psychosocial Factors in Patients with Acute Low Back Pain? Spine 2005, 30, 1316–1322. [Google Scholar] [CrossRef] [PubMed]
  47. Wassinger, C.A.; Sole, G. Agreement and screening accuracy between physical therapists ratings and the Örebro Musculoskeletal Pain Questionnaire in screening for risk of chronic pain during Musculoskeletal evaluation. Physiother. Theory Pract. 2022, 38, 2949–2955. [Google Scholar] [CrossRef] [PubMed]
  48. Roter, D.L.; Hall, J.A.; Aoki, Y. Physician gender effects in medical communication: A meta-analytic review. JAMA 2002, 288, 756–764. [Google Scholar] [CrossRef] [Green Version]
  49. Gatchel, R.J.; Peng, Y.B.; Peters, M.L.; Fuchs, P.N.; Turk, D.C. The biopsychosocial approach to chronic pain: Scientific advances and future directions. Psychol. Bull. 2007, 133, 581–624. [Google Scholar] [CrossRef]
  50. Jeffels, K.; Foster, N. Can Aspects of Physiotherapist Communication Influence Patients’ Pain Experiences? A Systematic Review. Phys. Ther. Rev. 2003, 8, 197–210. [Google Scholar] [CrossRef]
  51. Main, C.J.; George, S.Z. Psychologically Informed Practice for Management of Low Back Pain: Future Directions in Practice and Research. Phys. Ther. 2011, 91, 820–824. [Google Scholar] [CrossRef] [Green Version]
  52. Wijma, A.J.; van Wilgen, C.P.; Meeus, M.; Nijs, J. Clinical biopsychosocial physiotherapy assessment of patients with chronic pain: The first step in pain neuroscience education. Physiother. Theory Pract. 2016, 32, 368–384. [Google Scholar] [CrossRef]
  53. Al, U.B.A. Is Emotional Intelligence a Need Skill for Health Care Provider? J. Fam. Med. Dis. Prev. 2018, 4, 71. [Google Scholar] [CrossRef] [Green Version]
  54. Holopainen, R.; Simpson, P.; Piirainen, A.; Karppinen, J.; Schütze, R.; Smith, A.; O’Sullivan, P.; Kent, P. Physiotherapists’ perceptions of learning and implementing a biopsychosocial intervention to treat musculoskeletal pain conditions: A systematic review and metasynthesis of qualitative studies. Pain 2020, 161, 1150–1168. [Google Scholar] [CrossRef]
  55. King, C.A.; Bithell, C. Expertise in diagnostic reasoning: A comparative study. Br. J. Ther. Rehabil. 1998, 5, 78–87. [Google Scholar] [CrossRef]
  56. Rushton, A.; Lindsay, G. Developing clinical expertise for healthcare professionals through masters courses. Int. J. Ther. Rehabil. 2007, 14, 156–161. [Google Scholar] [CrossRef]
  57. Richardson, B. Professional Development: 2. Professional Knowledge and Situated Learning in the Workplace. Physiotherapy 1999, 85, 467–474. [Google Scholar] [CrossRef]
  58. Fritz, J.; Wallin, L.; Söderlund, A.; Almqvist, L.; Sandborgh, M. Implementation of a behavioral medicine approach in physiotherapy: Impact and sustainability. Disabil. Rehabil. 2020, 42, 3467–3474. [Google Scholar] [CrossRef] [Green Version]
  59. Overmeer, T.; Boersma, K.; Denison, E.; Linton, S.J. Does Teaching Physical Therapists to Deliver a Biopsychosocial Treatment Program Result in Better Patient Outcomes? A Randomized Controlled Trial. Phys. Ther. 2011, 91, 804–819. [Google Scholar] [CrossRef] [Green Version]
  60. Stevenson, K.; Lewis, M.; Hay, E. Does physiotherapy management of low back pain change as a result of an evidence-based educational programme? J. Eval. Clin. Pract. 2006, 12, 365–375. [Google Scholar] [CrossRef]
  61. Roitenberg, N. Physiotherapists’ attitudes toward low back pain treatment: Do work setting and clinical experience with low back patients matter. J. Eval. Clin. Pract. 2019, 25, 224–229. [Google Scholar] [CrossRef]
  62. Daykin, A.R.; Richardson, B. Physiotherapists’ Pain Beliefs and Their Influence on the Management of Patients with Chronic Low Back Pain. Spine 2004, 29, 783–795. [Google Scholar] [CrossRef] [PubMed]
  63. Synnott, A.; O’keeffe, M.; Bunzli, S.; Dankaerts, W.; O’Sullivan, P.; O’Sullivan, K. Physiotherapists may stigmatise or feel unprepared to treat people with low back pain and psychosocial factors that influence recovery: A systematic review. J. Physiother. 2015, 61, 68–76. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Peabody, J.W.; Luck, J.; Glassman, P.; Dresselhaus, T.R.; Lee, M. Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of 3 methods for measuring quality. JAMA 2000, 283, 1715–1722. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Peabody, J.W.; Luck, J.; Glassman, P.; Jain, S.; Hansen, J.; Spell, M.; Lee, M. Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study. Ann. Intern. Med. 2004, 141, 771–780. [Google Scholar] [CrossRef] [Green Version]
  66. Shah, R.; Edgar, D.F.; Evans, B.J.W. A comparison of standardised patients, record abstraction and clinical vignettes for the purpose of measuring clinical practice. Ophthalmic Physiol. Opt. 2010, 30, 209–224. [Google Scholar] [CrossRef]
  67. Sheringham, J.; Kuhn, I.; Burt, J. The use of experimental vignette studies to identify drivers of variations in the delivery of health care: A scoping review. BMC Med. Res. Methodol. 2021, 21, 81. [Google Scholar] [CrossRef]
  68. Tremblay, D.; Turcotte, A.; Touati, N.; Poder, T.G.; Kilpatrick, K.; Bilodeau, K.; Roy, M.; Richard, P.O.; Lessard, S.; Giordano, É. Development and use of research vignettes to collect qualitative data from healthcare professionals: A scoping review. BMJ Open 2022, 12, e057095. [Google Scholar] [CrossRef]
  69. Mescouto, K.; Olson, R.E.; Hodges, P.W.; Setchell, J. A critical review of the biopsychosocial model of low back pain care: Time for a new approach? Disabil. Rehabil. 2022, 44, 3270–3284. [Google Scholar] [CrossRef]
  70. Wåhlberg, A.E.A.; Poom, L. An Empirical Test of Nonresponse Bias in Internet Surveys. Basic Appl. Soc. Psychol. 2015, 37, 336–347. [Google Scholar] [CrossRef]
Table
Table 1. Sociodemographic characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Table 1. Sociodemographic characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Identification of the Main Risk Factor for Chronicity
Total Study
Population (n = 484)		Vignette Ap
Value		Vignette Bp ValueVignette Cp
Value
Psychosocial (n = 464)Physical (n = 405)Psychosocial (n = 320)Physical (n = 165)
Age (years), mean (SD)35 (8.2)35.3 (8.3)0.4235.2 (8.2)0.6834.6 (7.8)36.6 (9)0.016
Gender, n (%) <0.01 0.38 0.049
Females325 (67.1)319 (68.8) 268 (66.2) 224 (70)101 (61.2)
Males159 (32.9)145 (31.3) 137 (33.8) 96 (30)64 (38.8)
Table
Table 2. Professional characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Table 2. Professional characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Identification of the Main Risk Factor for Chronicity
Total Study Population (n = 484)Vignette Ap
Value		Vignette Bp ValueVignette Cp
Value
Psychosocial (n = 464)Physical (n = 405)Psychosocial (n = 320)Physical (n = 165)
Experience (years), mean (SD)12 (7.4)11.8 (7.5)0.2111.8 (7.3)0.611.1 (7.1)13.1 (7.9)0.004
Work setting, n (%) 0.65 0.3 0.71
Private practice272 (56.2)259 (55.8) 222 (54.8) 178 (55.6)94 (57)
Public Health Service109 (22.5)106 (22.8) 96 (23.7) 70 (21.9)39 (23.6)
Insurance Companies103 (21.3)99 (21.3) 87 (21.5) 72 (22.5)32 (19.4)
Postgraduate training in PS factors, n (%) 0.07 0.64 0.2
Yes143 (29.5)134 (28.9) 122 (30.1) 89 (27.8)55 (33.3)
No341 (70.5)330 (71.1) 283 (69.9) 231 (72.2)110 (66.7)
Table
Table 3. Attitudinal characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Table 3. Attitudinal characteristics for the total study population of PTs (n = 484) and their responses on the main risk factor for chronicity of each vignette.
Total Study Population (n = 484)Vignette Ap
Value		Vignette Bp
Value		Vignette Cp
Value
Psychosocial (n = 464)Physical (n = 405)Psychosocial (n = 320)Physical (n = 165)
Emotional Intelligence87 (12.6)87.7 (12.1)0.1187.9 (11.7)0.1387.5 (12.1)87.6 (12.9)0.92
     Emotional Attention27.1 (6.1)27.1 (6)0.0126.9 (6)0.6527.1 (6)26.6 (6.2)0.34
     Emotional Clarity29.5 (5.8)29.9 (5.5)0.7830.2 (5.6)<0.0129.8 (5.8)30 (5.5)0.60
     Emotional Repair30.4 (5.7)30.7 (5.5)0.6330.7 (5.4)0.3030.5 (5.5)30.9 (5.6)0.47
Social Intelligence123.7 (15.1)124.4 (14.8)0.04124.6 (14.8)0.11124.2 (14.9)124 (15.3)0.84
     Self Monitoring25.8 (4)25.9 (3.9)0.0726 (4)0.2426 (3.8)25.7 (4.2)0.33
     Social Sensitivity21.3 (3.7)21.4 (3.6)0.0121.4 (3.6)0.2321.3 (3.5)21.2 (4)0.86
     Social Information Processing25 (4.2)25.1 (4)0.0225.2 (4)0.0125 (4.1)25 (4.2)0.89
     Social Skills25.8 (4.8)26.1 (4.7)0.2826.1 (4.7)0.2426 (4.9)25.9 (4.5)0.77
     Social Awareness25.8 (4.3)25.9 (4.1)0.7825.9 (4.2)0.9225.8 (4.2)26.1 (4.1)0.41
Table
Table 4. Identification of the main risk factor for chronicity (n = 484).
Table 4. Identification of the main risk factor for chronicity (n = 484).
Vignette AVignette BVignette C
Psychosocial, n(%)464 (95.7)80 (16.5)320 (66)
Physical, n(%)21 (4.3)405 (83.5)165 (34)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Otero-Ketterer, E.; Peñacoba-Puente, C.; Ortega-Santiago, R.; Galán-del-Río, F.; Valera-Calero, J.A. Consideration of Psychosocial Factors in Acute Low Back Pain by Physical Therapists. J. Clin. Med. 2023, 12, 3865. https://doi.org/10.3390/jcm12113865

AMA Style

Otero-Ketterer E, Peñacoba-Puente C, Ortega-Santiago R, Galán-del-Río F, Valera-Calero JA. Consideration of Psychosocial Factors in Acute Low Back Pain by Physical Therapists. Journal of Clinical Medicine. 2023; 12(11):3865. https://doi.org/10.3390/jcm12113865

Chicago/Turabian Style

Otero-Ketterer, Emilia, Cecilia Peñacoba-Puente, Ricardo Ortega-Santiago, Fernando Galán-del-Río, and Juan Antonio Valera-Calero. 2023. "Consideration of Psychosocial Factors in Acute Low Back Pain by Physical Therapists" Journal of Clinical Medicine 12, no. 11: 3865. https://doi.org/10.3390/jcm12113865

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop