Next Article in Journal
Sex-Related Disparities in the Prevalence of Depression among Patients Hospitalized with Type 2 Diabetes Mellitus in Spain, 2011–2020
Previous Article in Journal
Prognostic Utility of the Modified Glasgow Prognostic Score in Urothelial Carcinoma: Outcomes from a Pooled Analysis
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 11
Issue 21
10.3390/jcm11216262
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Doll Therapy Intervention Reduces Challenging Behaviours of Women with Dementia Living in Nursing Homes: Results from a Randomized Single-Blind Controlled Trial

by
Valentina Molteni
1,2,
Roberta Vaccaro
2,3,
Roberta Ballabio
1,2,
Laura Ceppi
2,4,
Marco Cantù
5,
Rita B. Ardito
6,*,
Mauro Adenzato
6,
Barbara Poletti
7,
Antonio Guaita
3 and
Rita Pezzati
1,2,4
1
Dipartimento di Economia Aziendale e Socio Sanitaria (SUPSI), Centro Competenza Anziani, 6928 Manno, Switzerland
2
GINCO Ticino Association, 6802 Monteceneri, Switzerland
3
Golgi Cenci Foundation, 20081 Abbiategrasso, Italy
4
School of Cognitive Therapy, 22100 Como, Italy
5
Ente Ospedaliero Cantonale, Istituto di Medicina di Laboratorio (EOLAB), 6500 Bellinzona, Switzerland
6
Department of Psychology, University of Turin, 10124 Turin, Italy
7
Laboratory of Neuroscience, Department of Neurology, Istituto Auxologico Italiano, IRCCS, 20149 Milan, Italy
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2022, 11(21), 6262; https://doi.org/10.3390/jcm11216262
Submission received: 14 September 2022 / Revised: 17 October 2022 / Accepted: 20 October 2022 / Published: 24 October 2022
(This article belongs to the Section Clinical Neurology)
Browse Figure
Versions Notes

Abstract

:
Background: Doll therapy (DT) is a non-pharmacological intervention for the treatment of the behavioural and psychological symptoms of dementia (BPSD). We designed a single-blind randomized controlled trial of the 30-day efficacy of DT in reducing the BPSD, professional caregivers’ distress and patients’ biomarkers of stress, and in improving the exploration and caregiving behaviours. Methods: We randomly assigned 134 women with moderate-to-severe dementia living in nursing homes (NHs) to a DT intervention (DTI, 67) or a sham intervention with a cube (SI, 67). Results: From the first to the 30th session, the DTI group showed a significant decrease in the Neuropsychiatric Inventory-NH (NPI-NH) total score and in the NPI-NH-Distress score compared to the SI group (both p < 0.001). We observed a greater interest in the doll than in the cube, a greater acceptance of a separation from the nurse among DTI participants, and caregiving and exploratory behaviours towards the doll. There were no differences between the groups in the stress biomarkers. Conclusions: Consistent with attachment theory, our findings support the 30-day efficacy of DT, as this non-pharmacological intervention promotes perceptions of security by creating a situation in which patients feel confident and engaged in a caregiving relationship with the doll and reduces the challenging behaviours that are stressful for professional caregivers.

1. Introduction

Non-pharmacological therapies (NPTs) to support people with dementia (PWD) are recommended as first-line treatments for the behavioural and psychological symptoms of dementia (BPSD) [1]. More than half of people with moderate to severe dementia have at least one BPSD, with apathy, anxiety, irritability, agitation, and depression being the most common [2,3]. Such challenging behaviours are associated with an increased caregiver burden [4] and a poor quality of life for both the patients and caregivers [5]. Doll therapy (DT) is an NPT for people in the advanced stages of dementia, with the aim of reducing challenging behaviours. Promising results have already been shown in the literature on the effectiveness of DT in promoting and maintaining the affective-relational dimension of attachment caregiving [6,7]. DT appears to improve verbal [8,9] and non-verbal communication, including eye contact and touch [10], exploration and caregiving behaviours [7], and a patient’s interaction with others [11,12]. Promising results have also been shown in relation to professional caregivers’ distress [13,14]. However, the available studies are mainly pilot or exploratory studies [7,13,15,16]. Randomized controlled trials with rigorous study designs and larger samples are critical for determining the effectiveness of DT implemented in long-term settings and for identifying the best practice for interventions.
Another area of interest is the explanatory model of the DT process. The concept of attachment, theorized by John Bowlby [17], has been applied to the relationship with the doll [7,18]. Attachment theory states that the human tendency to seek closeness and protection when feeling vulnerable or insecure is an expression of an innate motivational system. This theory is particularly relevant for PWD, as dementia often exposes patients to feelings of personal vulnerability and loss, as well as experiences of separation [19,20]. Miesen suggested that PWD in the advanced stages of the disease experience a constant loss of control and security, a situation that constitutes a constant trauma that the patient tries to cope with by seeking reassurance and protection from the persons present, thereby activating attachment behaviours: “this is a normal reaction to an abnormal situation” [21]. Due to the cognitive and psychological impairments caused by the disease, attachment is therefore often considered as an essential psychological need [22]. From this perspective, BPSD such as wandering, dysphoria, anxiety, agitation, and aggressiveness could be interpreted as attachment requests. A clinical observation of PWD interacting with the doll has shown that they treat the doll like a real baby and thus replace the requests for care and protection (with typical attachment behaviours such as vocalizations, gestures, and tears) with caregiving behaviours, such as reassuring and cradling the doll. Following this theory, we assume that DT restores a sense of calm and peace which leads to the cessation of requests for a proximity to caregivers expressed by BPSD [7,22]. In nursing home (NH) settings, PWD are often exposed to feelings of insecurity in situations of separation from professional caregivers. Understanding the attachment behaviours of PWD can shift caregivers’ attention from challenging behaviours to the relationship needs of PWD. In our study, we specifically addressed the training of professional caregivers to improve their knowledge about DT and their awareness of their relationship with PWD.
Blood pressure and heart rate are involved in the response to acute and chronic psychological stressors. Secretion of salivary cortisol is the final product of the activation of stress-response mechanisms, i.e., the hypothalamic-pituitary-adrenal axis [23]. Changes in the rate of its secretion have been associated with acute stress responses related to disease and cognitive impairment [24,25]. To our knowledge, there is limited data on the effectiveness of NPTs on biomarkers of stress in PWD, although encouraging results have recently been reported for mindfulness interventions for cognitive impairment [26]. To date, no studies have examined the effect of DT on biomarkers of stress in PWD.
Our primary aims were to add scientific data on the effectiveness of DT on the challenging behaviours of PWD and professional caregivers’ distress in nursing home settings on a large population sample. We intended to add data on the efficacy of DT on biomarkers of stress in PWD, an area for which scientific evidence is not yet available. Finally, our aim was to show that DT promotes feelings of security by creating a situation in which PWD feel confident and engaged in a caregiving relationship with the object in the absence of the nurse. We also expected an increase in the exploration and caregiving behaviours of the patients during the interaction with the doll. The secondary aim was to confirm the hypothesized stability of the PWDs previous attachment style (determined by the Adult Attachment Interview administered to the patients’ family caregivers), even in the advanced stages of dementia. For the purposes of the present work, we have not presented the results of the secondary aim.

2. Materials and Methods

2.1. Design and Participants

This study is a randomized, single-blind, controlled trial with two parallel arms designed to evaluate the 30-day effectiveness of the DT intervention (DTI) compared to the sham intervention (SI) in women with dementia living in a NH. An independent statistician performed the computer-based block randomization [27]. The participants were assigned using a concealed 1:1 randomization. The statistician generated the randomization sequence and forwarded it to the study coordinator who received the randomization to assign patients to either group A or B (block size: 2 × 2 = 4).
To ensure blinding, the psychologist who administered the primary outcome instruments did not know which arm the participants belonged to. Conversely, the enrolled participants and the nurses were not blinded with respect of the objects provided (doll or cube). The participants were female with moderate to severe dementia (corresponding to stages 4–7 of the Global Deterioration Scale) [28] who lived in 26 NHs of the Canton Ticino area (Switzerland), had at least one BPSD in addition to depression or apathy, had been admitted to a NH at least 3 months previously, and had no previous exposure to DT. Participants who met the inclusion criteria were considered for the present study regardless of the type of dementia and the presence of speech disorders. The exclusion criteria were: the male gender, presence of comorbid mental disorders (i.e., major depression, bipolar disorders, and schizophrenia), inability to sit comfortably in a chair or limitations in arm mobility, and presence of acute clinical conditions that interfered with participation in the study.
The study completion was on 9 December 2019.

2.2. Ethical Consideration and Consent to Participate

The study received approval from the Swiss Ethics Committees on Research Involving Humans (n. CE3140 BASEC2016–01992) before starting. The patients or their legal representatives (families) were asked to sign a written informed consent form for the use of their personal data and were given concise and comprehensive information about the research objectives and the methods of data processing.

2.3. Intervention

The intervention procedure has been described in detail elsewhere [29]. Participants attended daily DTI or SI sessions for a maximum of one hour, led by a trained professional caregiver. DTI encompasses the presentation of an empathy doll recreating the sensation of touching, looking at, and holding a child in one’s arms. This kind of dolls elicits emotional reactions and provides opportunities for pleasurable sensory experiences [30]. The SI group participated in similar daily sessions, but a non-anthropomorphic object was presented (i.e., a soft foam rubber cube covered with a coloured and velvety textile). The DTI or SI was discontinued after seven consecutive refusals.
During the 18 months that the NHs were involved in the research project, training was provided for professional caregivers. It consisted of two classroom lessons, each lasting for four hours, on DT procedures and ten monthly clinical supervisions, each lasting half an hour, on single-case discussions aimed at improving the quality of the caregiving relationship as a source of security according to attachment theory. The maximum number of participants among professional caregivers was 25 per group. Each training group was conducted by trained psychologists and nurses. According to the Guidelines for the use of dolls as a therapeutic tool (www.fightdementia.org.au), dolls were proposed to PWD after considering their personal history, any traumatic events, and their parenting style. During their training, professional caregivers were made aware of the importance of paying attention to how the PWD interact with the doll (i.e., whether or not they treat the doll like a real baby) and validating the meaning it has for the patient. PWD are free to leave or refuse the doll at any time if they do not enjoy holding it.

2.4. Primary Outcomes’ Measurements

2.4.1. Patient’s BPSD

The difference in the patient’s BPSD was measured as the net change in the Neuropsychiatric Inventory-Nursing Home (NPI-NH) total score, pre- to post-intervention. We expected a significant reduction in the NPI-NH total score in the experimental group compared to the active control group. The NPI-NH was validated for interviewing the professional caregivers on twelve behavioural areas, including depression [31,32]. The frequency [from one (rarely, less than once a week) to four (very often, once, or more per day)] and the severity [from one (mild) to three (severe)] of each BPSD are reported. The domain scores (frequency × severity) are added into the total score, ranging from zero to 144, with higher scores indicating more severe symptoms.

2.4.2. Professional Caregivers’ Distress

The difference in the professional caregivers’ distress related to the patient’s BPSD was measured as the net change in the NPI-NH Distress score, pre- to post-intervention [31,32]. We expected a significant reduction in the NPI-NH Distress score in the experimental group compared to the active control group. When a BPSD is reported as present, the associated distress is rated on a scale from zero (no distress) to five points (very severe distress). The domain scores are added into the Distress score, ranging from zero to 60, with higher scores indicating more caregivers’ distress.

2.4.3. Patient’s Biomarkers of Stress

The decrease in patient stress was measured as a net change in the blood pressure (systolic and diastolic), heart rate, and salivary cortisol level [23,24,25,33]. We expected a significant difference in the biomarkers of stress between the DTI and SI groups. The saliva samples, blood pressure, and heart rate were collected immediately before and 15 min after the end of the intervention at the first and 30th session. The saliva samples were stored at an ambient temperature for 24 h and then processed immediately. A ratio between ‘immediately before’ versus ‘after 15 min’ was calculated for the first and 30th session: a ratio of one meant that the cortisol concentration, and thus the stress level (both low and high), had not changed (coded as irrelevant treatment); if the ratio was greater than one, the cortisol concentration, and thus the stress level, had decreased (coded as an effective treatment); and if the ratio was less than one, the cortisol concentration, and thus the stress level, had increased (coded as negative treatment).

2.4.4. Patient’s Interaction with the Object

We expected an increase in the interaction with the object (doll), as the exploration and caregiving behaviours during the object presentation were recorded at the first and 30th session by a trained psychologist who filled an observational grid developed for the aims of the present study [29]. It includes four kinds of behavioural responses: i.e., i. object presentation; ii. separation from the nurse; iii. interaction with the object; and iv. separation from the object. We expected significant differences between the DTI and SI groups from the first to the 30th session.

2.5. Statistical Analysis

2.5.1. Sample Size

A sample size of 64 subjects per group, for an analysis of covariance (ANCOVA) that includes the pre-intervention NPI-NH total score as a covariate, was computed using an estimated medium effect size (f = 0.25), an alpha level of 0.05, and a power of 0.8. Thereafter, a 10% attrition rate due to possible acute clinical conditions interfering with the participation in the study, or death, was considered. The power calculation was performed with the programme G*Power 3.1 [34].

2.5.2. Analysis

The quantitative variables were reported as the means with standard deviations (SD); the qualitative variables as were reported as percentages. Independent sample t-tests (or Mann–Whitney tests for the variables not normally distributed) or Chi-square tests according to the variable characteristics (continuous or binary, respectively) were performed to compare the experimental and active control groups in terms of sociodemographic, clinical characteristics, NPI-NH total, and NPI-NH Distress net change. The primary outcomes were reported as the adjusted mean and standard error (SE). An analysis of covariance (ANCOVA) was applied to compare the effects of the two experimental conditions (the DTI or SI group) on the post-intervention NPI-NH total score and NPI-NH Distress score. The pre-intervention NPI-NH total score and psychotropic medications were used as covariates in these analyses. The effect sizes of the between-groups changes according to Cohen (1992) [35] were also reported. A p-value of less than 0.05 was considered significant (two-sided). All analyses were conducted using SPSS Statistics 20.0 (IBM SPSS Statistics for Windows, version 20.0 (IBM Corp. Released 2011, Armonk, NY, USA). A per-protocol analysis was conducted.

3. Results

3.1. Participants

There were 134 eligible participants. Four of them (two in the DTI group and two in the SI group) dropped out of the study and one in the DTI group died during the 30 days of the intervention (Figure 1). One hundred and twenty-nine participants completed the study, 64 in the DTI group and 65 in the SI group. The attrition rate was of 3.73%.

3.2. Baseline Characteristics

The baseline sociodemographic and clinical characteristics of the sample are shown in Table 1. No significant differences were found between the two groups in age (z = −1.276, p = 0.202), pre-intervention NPI-NH total score (z = −1.05; p = 0.292), NPI-NH Distress score (z = −1.86, p = 0.063), psychotropic medication (z = −0.217, p = 0.828), Mini Mental State Examination score (z = −1.79, p = 0.074), Global Deterioration Scale score (z = −1.146, p = 0.252), and diagnosis [Fisher’s exact test = 3.163, p = 0.552), indicating the effectiveness of the randomization procedure.
Primary outcomes.

3.3. Patient’s BPSD and Caregivers’ Distress

The DTI group showed a greater decrease in the NPI-NH net change [mean (SD): −11.9 (9.7)] compared to the SI group [mean (SD): −4.01 (7.1); z = −5.65; p < 0.001]. In particular, the DTI group showed a greater reduction in agitation compared to the SI group [mean (SD): −1.52 (3.17) and mean (SD): −0.36 (2.56), respectively; z = −2.796, p < 0.005], anxiety [mean (SD): −1.95 (2.99) and mean (SD): −0.23 (1.91), respectively; z = −3.929, p < 0.001], apathy [mean (SD): −1.31 (2.64) and mean (SD): −0.1 (1.7), respectively; z = −3.418, p < 0.001], irritability [mean (SD): −0.18 (2.06) and mean (SD): −0.23 (1.91), respectively; z = −2.679, p < 0.05], and wandering [mean (SD): −1.53 (2.69) and mean (SD): −0.4 (3.06); z = −2.231, p < 0.05]. The DTI group reported a significant decrease in NPI-NH Distress net change [mean (SD): −3.9 (3.9)], which was greater than that of the SI group [mean (SD): −1.07 (3.1).; z = −4.42; p < 0.001].
The results of the one-way ANCOVAs on the post-intervention BPSD of the PWD and caregiver’s distress are shown in Table 2. Levene’s test for equality of variances and normality checks were performed and the assumptions were met. The DTI group reported a significant greater decrease in the post-intervention NPI-NH total score with respect to the SI group, with a medium effect size. The DTI group reported a significant decrease in the post-intervention NPI-NH Distress score, greater than that in the SI group, with a medium effect size.

3.4. Patient’s Biomarkers of Stress

The differences between the DTI and SI groups in the systolic and diastolic pressure ratio and heart rate ratio at the first and 30th session of the intervention can be seen in Table 3. At the first session, there were 64 subjects in the DTI group and 62 in the SI group, while at the 30th session, 62 subjects in the DTI group and 60 subjects in the SI group were retained for analysis. Based on the results of the Mann–Whitney test, there was a trend towards significant differences only in the heart rate ratio at the 30th session, with the SI group having a lower heart rate ratio (mean rank = 56.13, n = 61) compared to the DTI group [mean rank = 68.67, n = 63; z = −1.946; p = 0.052]. As shown in Table 4, the complete cortisol concentration data were obtained for 45 participants (DTI group = 19 and SI group = 26). No significant differences between the two groups emerged in the stress level from the first to the 30th session of intervention.

3.5. Patient’s Interaction with the Object

3.5.1. Object Presentation

The doll was accepted more often [mean (SD) = 23.09 (6.41)] than the cube [mean (SD) = 19.68 (7.93); (z = −2.339, p = 0.019); p < 0.005]. During the object presentation, in the DTI group (n = 62), among the 39 participants with a gaze direction “toward the nurse” in the first session, 15 directed their gaze “toward the object and the environment” in the last session (chi-square = 4.147, p < 0.05); whereas in the SI group (n = 57), among the 42 participants with a gaze direction “toward the nurse” in the first session, 37 subjects maintained their gaze on the nurse in the last session (chi-square = 17.094, p < 0.001).

3.5.2. Separation from the Nurse

During the separation from the nurse, after the presentation of the object, among 11 participants in the DTI group (n = 61) who expressed “complain and worry” in the first session, 7 accepted the separation, 2 showed a “lack of interest”, and 2 continued to complain in the last session (Fisher’s exact test = 20.431, p < 0.001). In the SI group (n = 60), among 12 participants who expressed “complain and worry” in the first session, 7 continued to complain in the last session (Fisher’s exact test = 27.718, p < 0.001).
During the last session, caregiving behaviours were showed by 34 out of 43 participants in the DTI group (n = 63) who “accepted separation from the nurse”, by 5 out of 9 participants who expressed “complain and worry”, and only by 2 out of the 11 participants with a “lack of interest” in the separation from the nurse (Fisher’s exact test = 14.094, p < 0.001). Similarly, during the last session, exploration behaviours were present among 40 out of 43 participants in the DTI group (n = 63) who “accepted separation from the nurse”, 8 out of 9 participants who expressed “complain and worry”, and 7 out of 11 participants with a “lack of interest” in the separation from the nurse (Fisher’s exact test = 5.831, p < 0.05).

3.5.3. Interaction with the Object

During the interaction with the object, the number of exploratory behaviours (e.g., touching, smelling, and looking at the object) did not differ in both groups during the first [DTI group: mean (SD) = 2.68 (0.8); SI group: mean (SD) = 2.72 (1); t = −0.218, p = 0.828] and the last session [DTI group: mean (SD) = 2.45 (0.93); SI group: mean (SD) = 2.47 (0.87); t = −0.091, p = 0.927]. Among 39 participants in the DTI group (n = 64) showing caregiving behaviours (e.g., caressing, cradling, talking, and smiling) in the first session, 6 did not show them in the last session, but among 25 participants not showing caregiving behaviours, 8 started to care for their dolls (chi-square = 18.319, p < 0.001). Of the participants in the SI group (n = 60), as expected, no one showed caregiving behaviours in the first session, but one participant started to care for her cube in the last session.

3.5.4. Separation from the Object

During the separation from the object, in the DTI group (n = 64), “putting the object aside” did not differ from the first to the last session (chi-square = 0.755, p > 0.05). Of the 20 participants in the SI group (n = 63) who put the cube aside in the first session, 10 kept the cube with them in the last session (chi-square = 4.506, p < 0.05).

4. Discussion

The present study demonstrated the 30-day efficacy of DTI compared to SI, resulting in a greater improvement in the BPSD and caregivers’ distress. DTI produced significant reductions in agitation, anxiety, wandering, apathy, and irritability. This is of pivotal importance as BPSD affect the presentation and progression of the disease, accounting the most difficulties for PWD, their families, and the professional caregivers. These improvements in BPSD are clinically important because BPSD are associated with a greater functional impairment, accelerated cognitive decline, poorer quality of life, increased caregivers’ burden, higher mortality, and more neuropathological markers of dementia [36]. Our observation of DTI has shown that PWD show fewer signs of emotional discomfort (i.e., frustration and agitation), are more stimulated, communicative, and friendly with others.
As previously reported, the evidence base for medications currently used to treat BPSD is limited or their efficacy is moderate, and there is a risk of adverse events, including mortality [37]. Our results are from a randomized controlled trial with a large sample size in a long-term setting, which support previous limited evidence of the therapeutic effect of DT [6,7,13,14,15,38] and confirms that DT can be considered as a first-line treatment for targeting challenging behaviours in women with dementia living in a NH. As Losada-Baltar and Jiménez-Gonzalo (2021) noted [39], the availability of evidence-based treatments in applied, real-world contexts helps us translate evidence into practice. In addition, DT significantly reduced the perceived professional caregivers’ distress, contributing to a higher quality of personalized care for the PWD. In line with a recent systematic review [40], this beneficial outcome could be related to the use of multiple implementation strategies, i.e., the provision of the intervention as part of staff’s usual duties, training of professional caregivers aimed at understanding the mechanisms of the intervention, and supervisions aimed at providing continuous support.
The risk of infantilization and deception has been addressed in DT [41,42]. In line with the previous literature, we considered training professional caregivers, improving their knowledge, and letting them experience the positive effects of DT as key points to overcome these risks [8,43]. In our view, the benefits of DT fulfil the bioethical concepts of beneficence (i.e., DT promotes well-being) and respect for autonomy (i.e., PWD can exercise their right to engage with dolls if they wish).
To our knowledge, DT has no potential negative effects in the case of grief or infertility experienced during one’s life, since the doll is not experienced as one’s own child, but as the child of a nurse. The DTI therefore does not reactivate past memories of grief, but responds to the PWDs current need for care, which can no longer be exercised in the context of the nursing home.
DTI offers relevant advantages in terms of costs-effectiveness: unlike other nonpharmacological treatments (e.g., pets, music, or art therapy), it does not necessarily require the presence of a skilled therapist but can be carried out within 24 h by nursing home staff (nurses, assistants) if they are properly trained and supervised. As previously described [7,29], PWD take care of the dolls even in the absence of a professional caregiver.
We did not find significant effects of DTI on the biomarkers of stress, i.e., the blood pressure, heart rate, and salivary cortisol levels. Since we considered dementia to be a condition of chronic stress, we expected a beneficial effect of DT due to improvement in BPSD. However, a recent study reported that the reduction in systolic BP is more related to a proximity to death and sex than dementia or age [44]. The null results on cortisol levels might be related to the insufficient salivation of the participants, which we found when collecting the saliva samples, so that only a small proportion of the samples were analysed. Furthermore, the data showed a high individual variability of the cortisol levels.
In addition to the reduction in BPSD, we found increased behaviours related to the relationship with the environment and professional caregivers. We observed a great interest in the doll, as the PWD directed their gaze more often to the doll than to the cube. We found a significant increase in the acceptance of or disinterest in being separated from the nurse among the DTI individuals who initially expressed worries and complains. Finally, we observed the presence of caring and exploratory behaviours towards the doll in the DTI individuals who accepted the separation from the nurse. Consistent with the theoretical framework of attachment [21,45,46], these findings confirmed the hypothesis that the doll facilitates perceptions of security by creating a situation in which PWD feel confident and engaged in a caregiving relationship with the object in the absence of the nurse.

Strengths and Limitations

To our knowledge, the present study is the first RCT on the effectiveness of DT with a large sample size in a long-term setting. Our findings thus enable us to identify the best practice for DTI with PWD and contribute to widespread DT. Another strength is the provision of training and supervision for professional caregivers to consolidate the procedures, and to have an ethical approach of the staff and to motivate them, as recommended by Cohen-Mansfield [47].
We acknowledge that this study has some limitations. First, the exclusion of men limits the generalizability of the results. As reported by the Department of Health and Sociality of the Canton Ticino (source: Statistica intra-muros, SOMED, Dipartimento della sanità e della socialità, Unità statistiche sanitarie, Bellinzona), 71.6% of PWD living in the NH of the Canton Ticino are women. Therefore, further research is needed to demonstrate the feasibility of DT in men. Secondly, the individual variability of salivary cortisol levels should be kept under control by estimating the individual levels at the baseline, i.e., by collecting the samples within 24 h before the start of the treatment. In addition, the scarce salivation of PWD made the collection of saliva samples much more difficult. Finally, we did not include a medium- to long-term follow-up on the effectiveness of the DT. However, professional caregivers administered DTI for as long as they evaluated it to be beneficial for PWD. Further research is needed to determine the medium- and long-term effects of DT in NHs.

5. Conclusions

The present study demonstrated the 30-day efficacy of DTI compared to SI and resulted in a greater improvement in the BPSD and caregivers’ distress. DT can be considered as a first-line treatment for challenging the behaviours of women with dementia living in NHs. However, further research is needed to determine the medium- and long-term effects of DT in NHs. Consistent with the theoretical framework of attachment, DT increases one’s perceptions of security by creating a situation in which PWD feel confident and engaged in a caregiving relationship with the doll and reduces the attachment behaviours that burden the professional caregiver. These benefits are not only a function of the tool used (the doll), but also of the relationship established between PWD and professional caregivers who are appropriately trained and aware of their role as an attachment figure.

Author Contributions

V.M. and R.V. contributed equally to the draft of the manuscript. Conceptualization, R.P. and V.M.; methodology, R.V., R.P., V.M., R.B.A., M.A. and A.G.; formal analysis, R.V. and M.C.; investigation, V.M., L.C. and R.B.; data curation, L.C. and R.V.; writing—original draft preparation, V.M. and R.V.; supervision, R.P.; Project Administration, R.P.; Funding Acquisition, Writing—review and editing R.P., R.B.A., M.A., B.P. and A.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the SUPSI (Manno, Switzerland), the School of Cognitive Psychotherapy ‘Centro Terapia Cognitiva’ (Como, Italy), the Alzheimer Switzerland Association (Bern, Switzerland), the Foundation for the Study of Neurodegenerative Diseases of Adults and the Elderly (Lugano, Switzerland), the Pica Alfieri Foundation (Lugano, Switzerland), the Alzheimer Switzerland Association, Section of Canton Ticino (Lugano, Switzerland), the Riziero Rezzonico Foundation (Lugano, Switzerland), and the Fidinam Foundation (Lugano, Switzerland). The Golgi Cenci Foundation (Abbiategrasso, Italy) has funded the design of the study, the statistical analysis, and the writing of the present manuscript. M.A. and R.B.A. were supported by University of Turin grants (‘Ricerca scientifica finanziata dall’Università’).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Swiss Ethics Committees on Research Involving Humans (n. CE3140 BASEC2016–01992).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study or their legal representative.

Data Availability Statement

The data presented in this study are available on request from the first author.

Acknowledgments

The authors thank the NH of Canton Ticino (Switzerland) that have taken part in the study with the support of the Office for Elderly and Home Care of this country.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

NPT = non-pharmacological therapies; PWD = people with dementia; BPSD = behavioural and psychological symptoms of dementia; NPI-NH = Neuropsychiatric Inventory-Nursing Home; DT = doll therapy; NH = nursing home; DTI = doll therapy intervention; SI = sham intervention; ANCOVA = analysis of covariance; SE = standard error.

References

  1. International Psychogeriatric Association IPA Complete Guides to Behavioral and Psychological Symptoms of Dementia (BPSD). Available online: https://www.ipa-online.org/publications/guides-to-bpsd (accessed on 2 March 2022).
  2. Van der Linde, R.M.; Dening, T.; Stephan, B.C.M.; Prina, A.M.; Evans, E.; Brayne, C. Longitudinal Course of Behavioural and Psychological Symptoms of Dementia: Systematic Review. Br. J. Psychiatry 2016, 209, 366–377. [Google Scholar] [CrossRef]
  3. Zhao, Q.; Tan, L.; Wang, H.; Jiang, T.; Tan, M.; Tan, L.; Xu, W.; Li, J.; Wang, J.; Lai, T.; et al. The Prevalence of Neuropsychiatric Symptoms in Alzheimer’s Disease: Systematic Review and Meta-Analysis. J. Affect. Disord. 2016, 190, 264–271. [Google Scholar] [CrossRef] [PubMed]
  4. Hazelhof, T.; Schoonhoven, L.; van Gaal, B.; Koopmans, R.; Gerritsen, D. Nursing Staff Stress from Challenging Behaviour of Residents with Dementia: A Concept Analysis. Int. Nurs. Rev. Rev. 2016, 63, 507–516. [Google Scholar] [CrossRef] [PubMed]
  5. Hongisto, K.; Hallikainen, I.; Selander, T.; Törmälehto, S.; Väätäinen, S.; Martikainen, J.; Välimäki, J.; Hartikainen, S.; Suhonen, J.; Koivisto, A. Quality of Life in Relation to Neuropsychiatric Symptoms in Alzheimer’s Disease: 5-Year Prospective ALSOVA Cohort Study. Int. J. Geriatr. Psychiatry 2018, 33, 47–57. [Google Scholar] [CrossRef] [PubMed]
  6. Ng, Q.X.; Ho, C.Y.X.; Koh, S.S.H.; Tan, W.C.; Chan, H.W. Doll Therapy for Dementia Sufferers: A Systematic Review. Complement. Ther. Clin. Pract. 2017, 26, 42–46. [Google Scholar] [CrossRef] [PubMed]
  7. Pezzati, R.; Molteni, V.; Bani, M.; Settanta, C.; Di Maggio, M.G.; Villa, I.; Poletti, B.; Ardito, R.B. Can Doll Therapy Preserve or Promote Attachment in People with Cognitive, Behavioral, and Emotional Problems? A Pilot Study in Institutionalized Patients with Dementia. Front. Psychol. 2014, 5, 342. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Mackenzie, L.; James, I.A.; Morse, R.; Mukaetova-Ladinska, E.; Reichelt, F.K. A Pilot Study on the Use of Dolls for People with Dementia. Age Ageing 2006, 35, 441–444. [Google Scholar] [CrossRef] [Green Version]
  9. Ellingford, L.; Mackenzie, L.; Marsland, L. Using Dolls to Alter Behavior in People with Dementia. Nurs. Times 2007, 103, 36–37. [Google Scholar]
  10. Fraser, F.; James, I. Why Does Doll Therapy Improve the Well-Being of Some Older Adults with Dementia? PSIGE Newsl. 2008, 105, 55–63. [Google Scholar]
  11. Braden, B.A.; Gaspar, P.M. Implementation of a Baby Doll Therapy Protocol for People with Dementia: Innovative Practice. Dementia 2015, 14, 696–706. [Google Scholar] [CrossRef]
  12. Mitchell, G.; McCormack, B.; McCance, T. Therapeutic Use of Dolls for People Living with Dementia: A Critical Review of the Literature. Dementia 2014, 15, 976–1001. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Cantarella, A.; Borella, E.; Faggian, S.; Navuzzi, A.; De Beni, R. Using Dolls for Therapeutic Purposes: A Study on Nursing Home Residents with Severe Dementia. Int. J. Geriatr. Psychiatry 2018, 33, 915–925. [Google Scholar] [CrossRef] [PubMed]
  14. Santagata, F.; Massaia, M.; D’Amelio, P. The Doll Therapy as a First Line Treatment for Behavioral and Psychologic Symptoms of Dementia in Nursing Homes Residents: A Randomized, Controlled Study. BMC Geriatr. 2021, 21, 545. [Google Scholar] [CrossRef] [PubMed]
  15. Balzotti, A.; Filograsso, M.; Altamura, C.; Fairfield, B.; Bellomo, A.; Daddato, F.; Vacca, R.A.; Altamura, M. Comparison of the Efficacy of Gesture-Verbal Treatment and Doll Therapy for Managing Neuropsychiatric Symptoms in Older Patients with Dementia. Int. J. Geriatr. Psychiatry 2019, 34, 1308–1315. [Google Scholar] [CrossRef] [PubMed]
  16. Xiang, Q.; Yih, C.; Ho, X.; Shao, S.; Koh, H.; Chuan, W.; Wuen, H. Complementary Therapies in Clinical Practice Doll Therapy for Dementia Sufferers: A Systematic Review. Complement. Ther. Clin. Pract. 2017, 26, 42–46. [Google Scholar] [CrossRef]
  17. Bowlby, J. The Making and Breaking of Affectional Bonds. I. Aetiology and Psychopathology in the Light of Attachment Theory. An Expanded Version of the Fiftieth Maudsley Lecture, Delivered before the Royal College of Psychiatrists, 19 November 1976. Br. J. Psychiatry 1977, 130, 201–210. [Google Scholar] [CrossRef] [Green Version]
  18. Bisiani, L.; Angus, J. Doll Therapy: A Therapeutic Means to Meet Past Attachment Needs and Diminish Behaviours of Concern in a Person Living with Dementia-a Case Study Approach. Dementia 2013, 12, 447–462. [Google Scholar] [CrossRef]
  19. Bradley, J.M.; Cafferty, T.P. Attachment among Older Adults: Current Issues and Directions for Future Research. Attach. Hum. Dev. 2001, 3, 200–221. [Google Scholar] [CrossRef]
  20. Consedine, N.S.; Magai, C. Attachment and Emotion Experience in Later Life: The View from Emotions Theory. Attach. Hum. Dev. 2003, 5, 1651–1687. [Google Scholar] [CrossRef]
  21. Miesen, B.; Jones, G. Care-Giving in Dementia: Contours of a Curriculum. Dementia 2010, 9, 473–489. [Google Scholar] [CrossRef]
  22. Alander, H.; Prescott, T.; James, I.A. Older Adults’ Views and Experiences of Doll Therapy in Residential Care Homes. Dementia 2015, 14, 574–588. [Google Scholar] [CrossRef] [PubMed]
  23. Tsigos, C.; Chrousos, G.P. Hypothalamic-Pituitary-Adrenal Axis, Neuroendocrine Factors and Stress. J. Psychosom. Res. 2002, 53, 865–871. [Google Scholar] [CrossRef]
  24. Lupien, S.J.; Lepage, M. Stress, Memory, and the Hippocampus: Can’t Live with It, Can’t Live without It. Behav. Brain Res. 2001, 127, 137–158. [Google Scholar] [CrossRef]
  25. Vreeburg, S.A.; Zitman, F.G.; van Pelt, J.; Derijk, R.H.; Verhagen, J.C.M.; van Dyck, R.; Hoogendijk, W.J.G.; Smit, J.H.; Penninx, B.W.J.H. Salivary Cortisol Levels in Persons with and without Different Anxiety Disorders. Psychosom. Med. 2010, 72, 340–347. [Google Scholar] [CrossRef]
  26. Ng, T.K.S.; Feng, L.; Fam, J.; Rawtaer, I.; Kumar, A.P.; Rane, G.; Cheah, I.K.M.; Mahendran, R.; Lee, Y.K.; Tan, E.C.; et al. Mindfulness Awareness Practice (MAP) to Prevent Dementia in Older Adults with Mild Cognitive Impairment: Protocol of a Randomized Controlled Trial and Implementation Outcomes. Int. J. Environ. Res. Public Health 2021, 18, 10205. [Google Scholar] [CrossRef]
  27. Randomizer Research Randomizer. Available online: https://www.randomizer.org/ (accessed on 1 February 2017).
  28. Reisberg, B.; Ferris, S.H.; de Leon, M.J.; Crook, T. The Global Deterioration Scale for assessment of primary degenerative dementia. Am. J. Psychiatry 1982, 139, 1136–1139. [Google Scholar]
  29. Vaccaro, R.; Ballabio, R.; Molteni, V.; Ceppi, L.; Ferrari, B.; Cantù, M.; Zaccaria, D.; Vandoni, C.; Ardito, R.B.; Adenzato, M.; et al. Doll Therapy Intervention for Women with Dementia Living in Nursing Homes: A Randomized Single-Blind Controlled Trial Protocol. Trials 2020, 21, 133. [Google Scholar] [CrossRef] [Green Version]
  30. Dusyma GmbH Joyk®-Empathy Dolls Made by Dusyma. Available online: https://www.dusyma.com/en/--cms-page.blog.joyk-empathiepuppen (accessed on 13 September 2022).
  31. Cummings, J.L.; Mega, M.; Gray, K.; Rosenberg-Thompson, S.; Carusi, D.A.; Gornbein, J. The Neuropsychiatric Inventory: Comprehensive Assessment of Psychopathology in Dementia. Neurology 1994, 44, 2308–2314. [Google Scholar] [CrossRef] [Green Version]
  32. Baranzini, F.; Grecchi, A.; Berto, E.; Costantini, C.; Ceccon, F.; Cazzamalli, S.; Callegari, C. Factor Analysis and Psychometric Properties of the Italian Version of the Neuropsychiatric Inventory-Nursing Home in a Population of Institutionalized Elderly in Psychiatric Comorbidity. Riv. Psichiatr. 2013, 48, 335–344. [Google Scholar]
  33. Lehrer, P.; Eddie, D. Dynamic Processes in Regulation and Some Implications for Biofeedback and Biobehavioral Interventions. Appl. Psychophysiol. Biofeedback 2013, 38, 143–155. [Google Scholar] [CrossRef] [Green Version]
  34. Faul, F.; Erdfelder, E.; Lang, A.-G.; Buchner, A. G*Power 3: A Flexible Statistical Power Analysis Program for the Social, Behavioral, and Biomedical Sciences. Behav. Res. Methods 2007, 39, 175–191. [Google Scholar] [CrossRef] [PubMed]
  35. Cohen, J. A Power Primer. Psychol. Bull. 1992, 112, 155–159. [Google Scholar] [CrossRef] [PubMed]
  36. Lanctôt, K.L.; Amatniek, J.; Ancoli-Israel, S.; Arnold, S.E.; Ballard, C.; Cohen-Mansfield, J.; Ismail, Z.; Lyketsos, C.; Miller, D.S.; Musiek, E.; et al. Neuropsychiatric signs and symptoms of Alzheimer’s disease: New treatment paradigms. Alzheimers Dement. 2017, 3, 440–449. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Kales, H.C.; Gitlin, L.N.; Lyketsos, C.G. Management of Neuropsychitric Symptoms of Dementia in Clinical Setting: Recommendations from a Multidisciplinary Expert Panel. J. Am. Geriatr. Soc. 2015, 62, 762–769. [Google Scholar] [CrossRef] [PubMed]
  38. Moyle, W.; Murfield, J.; Jones, C.; Beattie, E.; Draper, B.; Moyle, W.; Murfield, J.; Jones, C.; Beattie, E.; Draper, B. Can Lifelike Baby Dolls Reduce Symptoms of Anxiety, Agitation, or Aggression for People with Dementia in Long-Term Care? Findings from a Pilot Randomised Controlled Trial. Aging Ment. Health 2018, 23, 1442–1450. [Google Scholar] [CrossRef]
  39. Losada-Baltar, A.; Jiménez-Gonzalo, L. Translating Evidence into Practice: Nonpharmacological Interventions for BPSD. Int. Psychogeriatr. 2021, 33, 867–869. [Google Scholar] [CrossRef]
  40. Bennett, S.; Laver, K.; MacAndrew, M.; Beattie, E.; Clemson, L.; Runge, C.; Richardson, L. Implementation of Evidence-Based, Non-Pharmacological Interventions Addressing Behavior and Psychological Symptoms of Dementia: A Systematic Review Focused on Implementation Strategies. Int. Psychogeriatr. 2021, 33, 947–975. [Google Scholar] [CrossRef]
  41. Mitchell, G.; O’Donnell, H. The Therapeutic Use of Doll Therapy in Dementia. Br. J. Nurs. 2013, 22, 329–334. [Google Scholar] [CrossRef] [Green Version]
  42. Mitchell, G.; Templeton, M. Ethical Considerations of Doll Therapy for People with Dementia. Nurs. Ethics 2014, 21, 720–730. [Google Scholar] [CrossRef] [Green Version]
  43. Minshull, K. The Impact of Doll Therapy on Well-Being of People with Dementia. J. Dement. Care 2009, 17, 35–38. [Google Scholar]
  44. Hestad, K.A.; Engedal, K.; Selbæk, G.; Strand, B.H. Blood Pressure in Dementia, Mild Cognitive Impairment, and Subjective Cognitive Decline Related to Time of Death. Brain Behav. 2021, 11, e02166. [Google Scholar] [CrossRef] [PubMed]
  45. Ardito, R.B.; Adenzato, M.; Dell’Osbell, G.; Izard, E.; Veglia, F. Attachment Representations in Adults with Congenital Blindness: Association with Maternal Interactive Behaviors During Childhood. Psychol. Rep. 2004, 95, 263–274. [Google Scholar] [CrossRef] [PubMed]
  46. Adenzato, M.; Ardito, R.B.; Izard, E. Impact of Maternal Directiveness and Overprotectiveness on the Personality Development of a Sample of Individuals with Acquired Blindness. Soc. Behav. Pers. 2006, 34, 17–26. [Google Scholar] [CrossRef] [Green Version]
  47. Cohen-Mansfield, J. Nonpharmacological Treatment of Behavioral Disorders in Dementia. Curr. Treat. Options Neurol. 2013, 15, 765–785. [Google Scholar] [CrossRef]
Jcm 11 06262 g001 550
Figure 1. CONSORT flow diagram for individual randomized controlled trials of nonpharmacologic treatments.
Figure 1. CONSORT flow diagram for individual randomized controlled trials of nonpharmacologic treatments.
Jcm 11 06262 g001
Table
Table 1. Baseline sociodemographic and clinical characteristics by groups.
Table 1. Baseline sociodemographic and clinical characteristics by groups.
DTI (n = 64)SI (n = 65)
Age, mean (SD)86.9 (5.9)88.4 (5.5)
Pre-intervention # NPI-NH total score (range 0–144), mean (SD)33.84 (13.28)31.55 (12.1)
Pre-intervention ¥ NPI-NH Distress score (range 0–60), mean (SD)12.08 (5.4)10.6 (5.04)
Pre-intervention psychotropic medications, mean (SD)1.87 (1.11)1.81 (1.08)
Mini Mental State Examination (range 1–30), mean (SD)7.92 (4.95)10.27 (5.32)
Global Deterioration Scale (range 1–7), median (iqr)6 (5–7)6 (5–7)
Diagnosis, n (%)
Alzheimer23 (35.9)24 (36.9)
Vascular2 (3.1)4 (6.2)
Mixed7 (10.7)9 (13.8)
Other8 (12.5)3 (4.6)
Non specified dementia24 (37.5)25 (38.5)
Values denote mean with standard deviation (SD) in parenthesis or median with interquartile range (iqr) in parenthesis or frequency with percentages in parenthesis. # Neuropsychiatric Inventory-Nursing Home total score. ¥ Neuropsychiatric Inventory-Nursing Home–Distress.
Table
Table 2. Efficacy of DTI (experimental group) versus SI (control group) on post-intervention NPI-NH total and Distress score.
Table 2. Efficacy of DTI (experimental group) versus SI (control group) on post-intervention NPI-NH total and Distress score.
DTI (n = 64)SI (n = 65)
Post-InterventionAdjusted
Mean (SE)		95% CIAdjusted
Mean (SE)		95% CI Fp-Value§ Partial Ƞ2
NPI-NH total score21.067 (1.014)19.061–23.07328.508 (1.018)26.492–30.52343.433<0.0010.638
NPI-NH Distress score7.612 (0.435)6.751–8.47310.093 (0.432)9.237–10.94840.970<0.0010.625
F value derived from ANCOVA adjusting for pre-intervention NPI-NH total score and psychotropic medications. § Effect size relationship between the pre-intervention and post-intervention scores.
Table
Table 3. Comparison between the experimental and control groups on systolic and diastolic pressure ratio, and hearth rate ratio #.
Table 3. Comparison between the experimental and control groups on systolic and diastolic pressure ratio, and hearth rate ratio #.
DTISI z
Systolic pressure ratio at the 1st session1 (0.95–1.07)1 (0.94–1.08)−0.069
Systolic pressure ratio at the 30th session1 (0.95–1.08)1 (0.94–1.08)−1.180
Diastolic pressure ratio at the 1st session1 (0.97–1.07)1 (0.93–1.11)−0.687
Diastolic pressure ratio at the 30th session1 (0.93–1.07)1 (0.92–1.05)−0.661
Heart rate ratio at the 1st session1 (0.94–1.09)1 (0.93–1.04)−1.147
Heart rate ratio at the 30th session1 (0.97–1.11)0.98 (0.93–1.05)−1.946
Values denote median (iqr). # A ratio between immediately before versus ‘after 15 min’ was calculated for the first and 30th session of intervention. z value derived from the Mann–Whitney test.
Table
Table 4. Within-subject comparisons on cortisol levels by groups at pre- and post-intervention.
Table 4. Within-subject comparisons on cortisol levels by groups at pre- and post-intervention.
Post-intervention
Irrelevant #Effective ¥Negative
Pre-interventionIrrelevant #2 (28.6)2 (22.2)0 (0)
DTI (n = 19)Effective ¥3 (42.9)6 (66.7)3 (100)
Negative 2 (28.6)1 (11.1)0 (0)
Pre-interventionIrrelevant #4 (36.4)2 (25)2 (28.6)
SI (n = 26)Effective ¥3 (27.3)6 (75)4 (57.1)
Negative 4 (36.4)0 (0)1 (14.3)
Values denote frequency with percentages in parenthesis. # Irrelevant = ratio between ‘immediately before’ versus ‘after 15 minutes’ calculated for the first and 30th session: a ratio of one meant that the cortisol concentration, and therefore the stress level (both low and high), had not changed. ¥ Effective = ratio between ‘immediately before’ versus ‘after 15 min’ calculated for the first and 30th session: if the ratio was greater than one, the cortisol concentration, and therefore the stress level, was decreased. Negative = ratio between ‘immediately before’ versus ‘after 15 minutes’ calculated for the first and 30th session: if the ratio was less than one, the cortisol concentration, and therefore the stress level, was increased.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Molteni, V.; Vaccaro, R.; Ballabio, R.; Ceppi, L.; Cantù, M.; Ardito, R.B.; Adenzato, M.; Poletti, B.; Guaita, A.; Pezzati, R. Doll Therapy Intervention Reduces Challenging Behaviours of Women with Dementia Living in Nursing Homes: Results from a Randomized Single-Blind Controlled Trial. J. Clin. Med. 2022, 11, 6262. https://doi.org/10.3390/jcm11216262

AMA Style

Molteni V, Vaccaro R, Ballabio R, Ceppi L, Cantù M, Ardito RB, Adenzato M, Poletti B, Guaita A, Pezzati R. Doll Therapy Intervention Reduces Challenging Behaviours of Women with Dementia Living in Nursing Homes: Results from a Randomized Single-Blind Controlled Trial. Journal of Clinical Medicine. 2022; 11(21):6262. https://doi.org/10.3390/jcm11216262

Chicago/Turabian Style

Molteni, Valentina, Roberta Vaccaro, Roberta Ballabio, Laura Ceppi, Marco Cantù, Rita B. Ardito, Mauro Adenzato, Barbara Poletti, Antonio Guaita, and Rita Pezzati. 2022. "Doll Therapy Intervention Reduces Challenging Behaviours of Women with Dementia Living in Nursing Homes: Results from a Randomized Single-Blind Controlled Trial" Journal of Clinical Medicine 11, no. 21: 6262. https://doi.org/10.3390/jcm11216262

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop