You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Brain Sciences
Download PDF
  • Article
  • Open Access

3 July 2023

A Confirmatory and an Exploratory Factor Analysis of the Cohen-Mansfield Agitation Inventory (CMAI) in a European Case Series of Patients with Dementia: Results from the RECage Study

,
,
,
,
,
,
,
,
and
1
Department of Clinical Sciences and Community Health, Unit of Medical Statistics, Biometry and Bioinformatics “Giulio A. Maccacaro” Faculty of Medicine and Surgery, University of Milan, 20122 Milan, Italy
2
Laboratory of Psychology, Section of Cognitive and Experimental Psychology, Faculty of Philosophy, School of Psychology, Aristotle University of Thessaloniki (AUTh), 54124 Thessaloniki, Greece
3
Greek Association of Alzheimer’s Disease and Related Disorders (GAADRD), 54643 Thessaloniki, Greece
4
1st Department of Neurology, School of Medicine, Aristotle University of Thessaloniki (AUTh), 54124 Thessaloniki, Greece
Brain Sci.2023, 13(7), 1025;https://doi.org/10.3390/brainsci13071025
This article belongs to the Section Neurodegenerative Diseases
Version Notes
Order Reprints
Review Reports

Abstract

Background: One of the most widely used instruments for assessing agitation in dementia patients is the Cohen-Mansfield Agitation Inventory (CMAI), nevertheless no global score has been proposed. The aim of this study is: (a) to conduct a confirmatory (CFA) and exploratory factor analysis (EFA) of CMAI on people with dementia and Psychological and Behavioral Symptoms (BPSD), and (b) to propose an alternative structure, based on clinical criteria including all CMAI items. Methods: Confirmatory and exploratory factor analyses were carried out on the CMAI 29 items administered at baseline to 505 patients with dementia (PwD) and BPSD enrolled in the international observational RECage study. Results: The three-factor structure has not been confirmed by the CFA, whilst the EFA was carried out respectively on 25 items disregarding 4 items with a prevalence ≤5% and then on 20 items disregarding 9 items with a prevalence ≤10%. The four-factor structure explaining 56% of the variance comprised Physically Aggressive behavior, Verbally Aggressive behavior, Physically non-aggressive behavior, and Physically and verbally aggressive behavior. Conclusions: A new grouping of all items according to a clinical criterion is proposed, allowing for a more sensible evaluation of the symptoms leading to better differentiation.

1. Introduction

Agitation is a manifestation of various behaviors that characterize many neuropsychiatric disorders and syndromes, including cognitive impairment and dementia. A commonly accepted description does not exist, but the International Psychogeriatric Association (IPA) has recently attempted to set the criteria of agitated behavior by following the structure of the DSM-5 criteria (2013) [,]. According to the consensus definition, agitation describes a situation where: (a) it occurs during a neurocognitive disorder, such as cognitive impairment or dementia; (b) for a minimum of two weeks, the patient expresses behaviors associated with emotional distress, classified in three general categories (excessive motor activity, verbal aggression, and physical aggression); (c) the behaviors are severe enough to cause significant impairment in interpersonal relationships, in social functioning, as well as in activities of daily living; and (d) besides the fact that the co-morbidity of agitation with other conditions could be present, agitation is not attributable solely to another psychiatric, medical, or psychological condition [].
Agitation during dementia is a common behavioral symptom observed in such patients of any etiology, such as of dementia due to Alzheimer’s disease [], due to Lewy Body [] or due to frontotemporal dementia []. Agitation can be found in any stage of the course of dementia, from very mild to mild, and from moderate to severe, either in patients living at nursing homes [,] or in community-dwelling patients [] and it is usually associated with increased healthcare costs []. According to a recent review, there is a wide prevalence of agitation symptoms in dementia reported in the literature, with prevalence rates between 5% to 88% that vary by geographic region, and with lower ranges to be reported for Asia []
Until today, several rating scales that are utilized in people with dementia to assess neuropsychiatric symptoms and agitation exist. Such commonly utilized scales in clinical settings are the Neuropsychiatric Inventory (NPI) [], the Neurobehavioral Rating Scale (NBRS) [], the Behavioral Pathology in Alzheimer’s Disease Rating Scale (BEHAVE-AD) [], or the Pittsburgh Agitation Scale (PAS) [].
One of the most widely used instruments for assessing agitation in patients with dementia is the Cohen-Mansfield Agitation Inventory (CMAI) designed by Cohen-Mansfield et al., in 1989 []. Initially, the instrument was developed for use at nursing homes, but during the past years it was also used in various clinical settings. Different versions exist, including the short or the long form of the instrument, the long form with expanded definitions, as well as the community form and the disruptiveness form [].
The original (long form) of the CMAI rating questionnaire contains one question for each of the following 29 agitation behaviors: (1) pace, aimless wandering, (2) inappropriate dress or disrobing, (3) spitting, (4) cursing or verbal aggression, (5) constant unwarranted request for attention or help, (6) repetitive sentences or questions, (7) hitting (including self), (8) kicking, (9) grabbing onto people, (10) pushing, (11) throwing things, (12) strange noises (weird laughter or crying), (13) screaming, (14) biting, (15) scratching, (16) trying to get to a different place, (17) intentional falling, (18) complaining, (19) negativism, (20) eating/drinking inappropriate substances, (21) hurt self or other, (22) handling things inappropriately, (23) hiding things, (24) hoarding things, (25) tearing things or destroying property, (26) performing repetitious mannerisms, (27) making verbal sexual advances, (28) making physical sexual advances, and (29) general restlessness. Regarding scoring, there is a 7 points Likert rating scale for each behavior, ranging from 1 to 7 (1 = never, 7 = several times an hour), indicating the observation frequency of each behavior. The CMAI inventory requires approximately 20 min to be completed [].
As far as the psychometric properties are concerned, many studies in several countries attempted assessing the validity of the CMAI using factor analysis. Given the great diversity of the behaviors CMAI contains, constructors decided not to calculate a total score (by adding all items), but to identify different factors. One of the first validation studies was performed in three units of a nursing home and included 66 residents (15 men and 51 women, age range from 59 to 96 years) []. This study identified three factors of agitation: aggressive behavior (including hitting, kicking, pushing, scratching, tearing things, cursing or verbal aggression, grabbing onto people, biting, and spitting); physically non-aggressive behavior (pacing, inappropriate robing or disrobing, trying to get to a different place, handling things inappropriately, general restlessness, and repetitious mannerisms), and verbally agitated behavior (complaining, repeated requests for attention, negativism, repetition of sentences or questions, and screaming) [,]. The scores of the three aforementioned factors were considered along with the absolute and relative frequencies of the agitated or not agitated status for each factor. It is important to remark that the factors did not include all items of the CMAI rating scale, since items with low prevalence were disregarded. Other studies have also indicated a three-factor model that explains the behavioral symptoms [,,,] but with different items considered.
Similarly, the study of Choy et al. [] enrolled inpatients and outpatients (164 people with dementia, 57 men and 107 women with an age range from 55 to 102 years) from two hospital units in Hong Kong (Castle Peak Hospital and the Prince of Wales Hospital). According to their factor analysis, three different factors were identified: (a) physically aggressive behaviors, (b) physically non-aggressive behaviors, and (c) verbally agitated behaviors, while six of the twenty-nine behaviors were excluded from the analysis since they rarely occurred, being present in less than 5% of the subjects.
Some validation studies were performed in community-based populations. One of the first studies on a community-based sample was performed by Cohen-Mansfield []. According to that exploratory and confirmatory analysis, there was evidence for three factors (verbally agitated behavior, verbally-non agitated behavior, and physically non-aggressive behavior) in both staff and relatives’ ratings, whilst an extra factor of physically aggressive behavior was suggested in the relatives’ ratings. Due to insufficient data, the authors did not come to a conclusion as to whether a three- or a four-factor model was most appropriate for capturing the broad range of agitation.
Subsequently, Koss et al., (1997) conducted research on an English-speaking community-based population of 306 persons, 241 of which had Alzheimer Disease. In this study, an expanded and revised list of 38 items of the CMAI version was utilized. The two additional items derived from directly observable behaviors (temporal occurrence of agitation) and one open item was added as well (possible additional disruptive behavior). The analysis identified four factors: (1) physically non-aggressive behaviors, (2) physically aggressive behaviors, (3) verbally non-aggressive behaviors, and (4) verbally aggressive behaviors. However, the aforementioned four factors do not include all the 38 items of the community CMAI rating scale, since some had a low prevalence [].
Similarly, Rabinowitz, Davidson, De Deyn, Katz, Brodaty, and Cohen-Mansfield, in 2005, investigated a large sample of 1265 older people with BPSD from three nursing homes by using the 29 items nursing home version. This analysis also identified four factors: (a) Aggressive Behavior (hitting, kicking, scratching, biting, pushing, grabbing, throwing things, cursing or verbal aggression, spitting, tearing things/destroying property, hurting self or others, screaming), and (b) Physically Non-Aggressive Behavior (pacing, trying to get to a different place, general restlessness, inappropriate dressing or disrobing, handling things inappropriately, performing repetitious mannerisms), (c) Verbally Agitated Behavior (complaining, constant requests for attention, repetitive questions, negativism), and (d) Hiding and Hoarding. In his study, like in others, items with low frequency (less than 9%), were excluded from the analysis. These items were: eating inappropriate substances, physical sexual advances, verbal sexual advances, and intentional falling [].
More recently, Patrick et al., investigated a cohort of 609 persons with dementia, followed up by a memory clinic and their caregivers []. The researchers excluded from the analyses the CMAI items occurring in less than 5% of caregivers. The excluded items were: (a) intentionally falling, (b) hurting the self or others, (c) spitting, (d) scratching, (e) pushing, (f) biting, (g) kicking, (h) making physical and verbal sexual advances, (i) and eating/drinking inappropriate substances. The results of the exploratory factor analysis initially supported a seven-factor structure that explained 62% of the variance. However, based on the studies suggesting that the number of items in each factor should be greater than or equal to three [,], the number of the extracted factors was limited to three. Thus, three robust factors accounted for 52.45% of the total variance, and were (a) physically aggressive, (b) physically non-aggressive, and (c) verbally agitated.
There are also a few studies conducted with patients living in care facilities. Schnelli, Ott, Mayer, & Zeller (2021), studied the data retrieved from the nursing documents of 1182 clients seeking services of six home care organizations in Switzerland in patients with dementia, delirium, and other psychiatric disorders. In their study the factor analysis revealed five main factors, these being (a) searching behaviors, (b) physically aggressive behaviors, (c) disruptive behaviors, (d) verbally aggressive behaviors, and (e) importunate behaviors. Nevertheless, in this study the participants included not only people with dementia but also with other psychiatric disorders []. Finally, a relatively recent study highlighted the validity of an observation-based version of the CMAI (CMAI-O), which has proven to be a promising research tool for independently measuring agitation in people living with dementia in Nursing Home settings. This tool provides additional information based not only on proxy informants but also on direct observation that helps to better define the symptoms labeled as “agitation” and follow their changes over the time. This “new” tool could represent an interesting line of research and a new possibility in clinical practice [].

Objective of the Study

The aim of our study was to conduct a confirmatory and exploratory factor analysis of the long form of the CMAI inventory according to the manual [] on people with dementia and Behavioral and Psychological Symptoms of Dementia (BPSD) enrolled in the RECage clinical trial [].
We especially considered the problem of excluding items from the analysis due to their low prevalence. In fact, by excluding items with low frequency, apart from losing useful information, we would risk not being able to distinguish between patients with or without these behaviors, ultimately not reaching a valid assessment of the patient’s severity.
The study was based on all CMAI factors stated in the CMAI manual [], since according to it, it is not useful to calculate a total score by adding the individual scores of all items. Former researchers decided to exclude some items from the factor analyses based on their low prevalence (usually ≤5% or ≤10%.) and avoiding the consequent statistical problems occurring in the factor analysis calculation.
Therefore, the main aim of the study is to propose an alternative (more clinical) factor structure of the CMAI that includes all of its items.

2. Materials and Methods

2.1. Design and Procedure

The present study was part of the REspectful Caring for AGitated Elderly study (RECage). The RECage study was a longitudinal and multicultural study comprising 11 clinical centres from six European countries, (2018–2023). The main aim of the RECage project was to evaluate the short- and long-term clinical efficacy of the Special Medical Care Units (SCU-B) for people with dementia and Behavioral and Psychological Symptoms of Dementia (BPSD). A SCU-B was defined as a “residential medical structure lying outside of a nursing home, in a general hospital or elsewhere, e.g., in a private hospital or a geriatric or psychiatric hospital, where patients with BPSD are temporarily admitted when their behavioral disturbances are not amenable to control at home” []. Therefore, this study compared the first cohort of PwD followed by SCU-B with the possibility of being admitted for copying BPSD, and the second cohort (control-non SCU-B) without having this possibility. In comparison with the non SCU-B units, by means of the pattern of the NPI total score over the follow-up time of 36 months; the pattern of the three CMAI factors was moved from a co-primary objective to a secondary one with an ad hoc protocol amendment.
The RECage study also had secondary and tertiary objectives, such as assessing the Quality of Life of the patients and their caregivers, the cost-effectiveness of SCU-Bs, and psychotropic drug consumption. Finally, the assessment of SCU-Bs’ capacity to delay the time to Nursing Home Placement (NHP) was also an objective. For more details, please see Poptsi et al., 2021 [].

2.2. Participants

The total sample of the RECage study amounted to 508 patients. Three patients were excluded because of CMAI missing data. Therefore, the final study sample consisted of 505 patients. The participants were followed up every 6 months for three years by 11 European clinical centers (6 non-SCU-B and 5 SCU-B) [].
Of the enrolled patients, 279 were females (54.9%), with a mean age of 78.1 years (SD = ±7.95), and a mean education of 8.93 (±4.53) years. The mean score of the Mini Mental State Examination (MMSE) [] was 15.4 (±6.25), whilst the mean score in Neuropsychiatric Inventory (NPI) [] was 52.5 (±18.97).
The CMAI factor means were 12.2 (±4.77) for Factor 1 “Aggressive Behavior”, 17.1 (±8.17) for Factor 2 “Physically Non-aggressive Behavior”, and, finally, 16.5 (±7.30) for Factor 3 “Verbally Agitated Behavior”. It must be noted that these mean scores characterize a low involvement (9–63 is the interval of the values for Factor 1 with 9 items, 6–42 for Factor 2 with 6 items and, finally, 5–35 for Factor 3 with 5 items). Furthermore, the mean NPI score was lower than half of its maximum total value of 144. Moreover, there was an intermediate involvement for the MMSE with values ranging from 0 to 30 and we have to take into account that the 38.0% of the patients were categorized as “moderately severe/severely impaired AD patients” (<15), and the 37.8% were categorized as “Moderately impaired AD patients” (15–20 extremes included). Regarding the main relevant part of the protocol, it is presented in Poptsi et al. [].

2.3. Instruments

The battery of tests administered in the RECage study was: (1) for the assessment of the general cognitive status the MMSE [], (2) for the assessment of the general functional status the Activities of Daily Living scale (ADL) [], (3) for the assessment of neuropsychiatric symptoms the NPI [], and (4) for the assessment of the patient’s BPSD the Cohen-Mansfield Agitation Inventory (CMAI) []. Additional scales for the assessment of the quality of life were also administered [].

2.4. The CMAI

For the needs of the present study, the long CMAI questionnaire was used, which is the original version used in nursing home’s population, containing 29 items.
The main reason for utilizing the long form of the 29-item CMAI instead of the 37-item Community form (CMAI-C) was to reduce the burden of the caregivers in the RECage study and to reduce the time of each visit attended by outpatients. Therefore, the three-factor structure of the CMAI nursing home shown in its manual has been taken into consideration.

2.5. Statistical Methods

Descriptive statistics have been calculated for quantitative variables (mean, standard deviation) and for qualitative (categorical) variables (absolute and percent frequency).
Confirmatory and exploratory factor analyses (CFA and EFA, respectively) have been carried out with PROC CALIS and PROC FACTOR of SAS® Version 9.4, respectively. The CFA hypothesis is that a 3-factor structure is evident when considering the 20 items analyzed in the factor analysis reported in the CMAI manual. The adequacy of fitting has been assessed by the χ2 test and the values of the Root Mean Square Error of Approximation (RMSEA < 0.05 for a good model fit), the Standardized Root Mean Square Residual (SRMSR < 0.05 for a good model fit, and, finally, by the Bentler’s comparative fit index (>0.90 for a good model fit). EFA has been carried out on a different set of items, depending on their prevalence, aiming to obtain the most parsimonious factor structure with a clear clinical interpretation.
Several rotation methods (varimax, promax, etc.) have been used in EFA after the factor extraction with the Principal Component Analysis (PCA) to obtain a better differentiation of the factor loadings [,,].

3. Results

Nine items were recorded in less than 10% of our case series. Five of them (“Hurt self or others”, “Making physical sexual advances”, “Intentional falling”, “Eating/drinking inappropriate substances”, and “Making verbal sexual advances”) were excluded from the factor analysis according to the CMAI manual []. Four items (“Kicking”, “Biting”, “Scratching”, and “Spitting”) in our case series showed a prevalence lower than 10%. The low prevalence of some items lead us to disregard them from the factor analysis, in order to find a clinically sensible structure of their rating scale (Table 1).
Table 1. Items of the CMAI questionnaire, in absolute and percentage frequencies of the PwD with BPSD of the RECage study.

3.1. Confirmatory Factor Analysis (CFA)

After 12 iterations the convergence has been reached. The model fit χ2 was 960.156 (df = 167, p ≤ 0.0001), statistically rejecting the confirmatory factor model of the CMAI. Indeed, the Root Mean Square Error of Approximation (RMSEA) was 0.2289, greater than the conventional 0.05 value for a good model fit. The Standardized Root Mean Square Residual (SRMSR) was 0.0867, not close to the conventional 0.05 value for a good model fit. In addition, Bentler’s comparative fit index was 0.7262, much lower than the required value of at least 0.90, leading us to conclude that the model was very poorly fitted. So, taking into account that all the above four criteria consistently testify to an inadequate model fit, it is possible to conclude that the CMAI three-factor model proposed in the CMAI manual was not adequate for our data.

3.2. Exploratory Factor Analysis (EFA)

(1) The EFA carried out on all 29 CMAI items did not reach a computational result owing to the error that the maximum number of iterations has been exceeded.
(2) Therefore, we were obliged to discard from the analysis the items “14—Biting” (2.57%), “15—Scratching” (3.96%), “17—Intentional falling” (NC) (4.55%), and “28—Making physical sexual advances (NC)” (5.94%) with a prevalence lower than 6%. It must be noted that the items “17—Intentional falling”, and “28—Making physical sexual advances” were also not considered in the three-factor structure of the CMAI reported in the CMAI manual []. Furthermore, our analysis included the following seven items “11—Throwing things (14.65%)”, “12—Strange noises (weird laughter or crying) (33.27%)”, “20—Eating/drinking inappropriate substances (9.70%)”, “21—Hurt self or other (6.14%)”, “23—Hiding things (50.20%)”, “24—Hoarding things (41.39%)”, and “27—Making verbal sexual advances (7.52%)”, not considered in the CMAI structure, with three factors reported in the CMAI manual []. The four-factor structure of the CMAI, is presented in Table 2.
Table 2. Factor analysis on 25 items with a prevalence greater than 6%.
The variance explained by each factor before the “varimax” rotation was 5.069 (56%) for Factor 1, 2.127 (23%) for Factor 2, 1.323 (15%) for Factor 3, and 0.737 (8%) for Factor 4. After the “varimax” rotation, the variance explained by each factor was 2.824 (31%) for Factor 1, 2.565 (28%) for Factor 2, 2.441 (27%) for Factor 3, and 1.234 (14%) for Factor 4.
Factor 1 includes “Physically Aggressive behaviors” and comprises the following 6 items: “7—Hitting (including self) (F1)”, “8—Kicking (F1)”, “9—Grabbing onto people (F1)”, “10—Pushing (F1)”, “11—Throwing things (NC)”, and “21—Hurt self or other (NC)”. These items practically correspond to CMAI Factor 1 “Aggressive behavior” with the addition of items 11 and 21 not considered in the Factor analysis reported in the CMAI manual.
Factor 2 presents “Verbally Aggressive behaviors” and it is constituted by the following 6 items: “5—Constant unwarranted request for attention or help (F3)”, “6—Repetitive sentences or questions (F3)”, “12—Strange noises (weird laughter or crying) (NC)”, “18—Complaining (F3)”, “19—Negativism (F3)”, and “29—General restlessness (F2)”. These items practically correspond to CMAI Factor 3 “Verbally agitated behavior” with the addition of item 12 not considered in the Factor analysis reported in the CMAI manual. It must be noted that maybe the item “29—General restlessness (F2)” with a relevant load on this factor could be more pertinent to this factor than to the CMAI Factor 2.
Factor 3 comprises the following 9 items, which mostly include “Physically non-aggressive behaviors”: “1—Pace, aimless wandering (F2)”, “2—Inappropriate dress or disrobing (F2)”, “16—Trying to get to a different place (F2)”, “20—Eating/drinking inappropriate substances (NC)”, “22—Handling things inappropriately (F2)”, “23—Hiding things (NC)”, “24—Hoarding things (NC)”, “25—Tearing things or destroying property (F1)”, and “26—Performing repetitious mannerisms (F2)”. These items practically correspond to CMAI Factor 2 “Physically non-aggressive behavior” with the addition of items 20, 23, and 24 not considered in the Factor analysis reported in the CMAI manual. However, item “25—Tearing things or destroying property (F1)” with a prevalence of 14.06% loads very similarly also on Factor 1 (0.26794 and 0.29928, respectively). This item belongs to CMAI Factor 1 “Aggressive behavior”.
Factor 4 is constituted by “Physically and verbally aggressive behaviors” and consists of the following 4 items: “3—Spitting (F1)”, “4—Cursing or verbal aggression (F1)”, “13—Screaming (F3)”, and “27—Making verbal sexual advances (NC)”.
In addition, a further Factor Analysis on the 20 items with a prevalence greater than 10% has been carried out. However, only two items “4—Cursing or verbal aggression (F1)” and “13—Screaming (F3)” load on Factor 4, and, in addition, only three items “10—Pushing (F1)”, “11—Throwing things (NC)”, and “25—Tearing things or destroying property (F1)” load on Factor 3. Moreover, the latter item loads very similarly on Factor 2 and Factor 3 (0.24364 and 0.25429, respectively).
Therefore, we repeated the above Factor Analysis on the 20 items with a prevalence greater than 10% by constraining the Factor number to three. Thus, the items “3—Spitting” (7.33%), “8—Kicking (8.91%), “20—Eating/drinking inappropriate substances (NC)” (9.70%), “21—Hurt self or others (NC) (6.14%)”, and “27—Making verbal sexual advances (NC) (7.52%)” have been excluded. It must be noted that items “20—Eating/drinking inappropriate substances (NC)”, “21—Hurt self or others (NC), and “27—Making verbal sexual advances (NC)” were not considered in the factor analysis with three factors described in the CMAI manual (Table 3).
Table 3. Factor analysis on 20 items with a prevalence greater than 10.0% and constrained to three factors.
The variance explained by each factor was 4.602 (62.7%) for Factor 1, 1.748 (23.8%) for Factor 2, and 0.992 (13.5%) for Factor3. After the “varimax” rotation, the variance explained by each factor was 2.602 (35.4%) for Factor 1, 2.421 (32.9%) for Factor 2, and 2.320 (31.6%) for Factor 3.
We define Factor 1 as “Verbally Aggressive behavior” which comprises episodes of verbal abuse, without causing any physically harmful events to other people. Factor 1, is constituted by the following 6 items: ”5—Constant unwarranted request for attention or help (F3)”, 6—Repetitive sentences or questions (F3)”, “12—Strange noises (weird laughter or crying) (NC)”, “18—Complaining (F3)”, “19—Negativism (F3)”, and ”29—General restlessness (F2)”. These items practically correspond to CMAI Factor 3 “Verbally agitated behavior” with the addition of item 12 not considered in the Factor analysis reported in the CMAI manual. It has to be noted that maybe the item “29—General restlessness (F2)” with a relevant load on this factor could be more pertinent to this factor than to the CMAI Factor 2.
Then, we define Factor 2 as “Physically Aggressive behavior”, since it consists of severe aggressive behaviors, including physical force intending to harm another person or damage an object. According to other research, aggression in persons with dementia does not have an intent, but is rather an expression of anxiety, fear, cognitive decline and confusion. In any case, physically aggressive behavior is considered to lead to hospitalization of people with BPSD, as well as to physical and psychological distress of the caregivers. Therefore, the “Physically Aggressive behavior”, according to our results, is constituted by the following 7 items: “4—Cursing or verbal aggression (F1)”, “7—Hitting (including self) (F1)”, “9—Grabbing onto people (F1)”, “10—Pushing (F1)”, 11—Throwing things (NC)”, ”13—Screaming (F3)”, and 25—Tearing things or destroying property (F1). These items practically correspond to CMAI Factor 1 “Aggressive behavior” with the addition of item 11 not considered in the Factor analysis reported in the CMAI manual and of item “13—Screaming (F3)” considered in the CMAI Factor 3 “Verbally agitated behavior”.
However, item “25—Tearing things or destroying property (F1)” with a prevalence of 14.06% also loads very similarly on Factor 3 (0.26152 and 0.25112, respectively).
Finally, we name Factor 3 as “Physically non-aggressive behavior” since it consists of annoying behaviors expressed physically, which are not physically harmful either for the patient or others. According to our results the “Physically non-aggressive behavior” was constituted by the following 7 items: “1—Pace, aimless wandering (F2)”, “2—Inappropriate dress or disrobing (F2)”, “16—Trying to get to a different place (F2)”, “22—Handling things inappropriately (F2)”, “23—Hiding things (NC)”, “24—Hoarding things (NC)”, and “26—Performing repetitious mannerisms (F2)”. These items belong to CMAI Factor 2 “Physically non-aggressive behavior” with the addition of the items 23 and 24 not considered in the Factor analysis reported in the CMAI manual.

4. Discussion

We did not successfully confirm the three-factor structure of the 29-item CMAI. At the same time, we confirmed that it is not possible to include all 29 items of the CMAI in the factor analysis, since we obtained a computational error of having exceeded the maximum number of iterations from the output of the PROC FACTOR of SAS due to the lower prevalence (≤5%, at least) of some items. Furthermore, the exploratory analysis of 25 items with a prevalence greater than 6% led to a four-factor CMAI structure, which comprised 6 items loading to factor one, 6 items loading to factor two, 9 items loading to factor three, and 4 items loading to factor four.
Our results are in partial agreement with the CMAI exploratory analysis results, described in the CMAI manual and referred to community samples. To be accurate, it seems that 4 of the 6 items that loaded to our Factor 1 correspond to the “Aggressive behavior” (according to the CMAI manual), five of the six items loading on our Factor 2 correspond to CMAI Factor 3 “Verbally agitated behavior”, and, finally, six items of the nine items loading on our Factor 3 practically correspond to CMAI Factor 2 “Physically non-aggressive behavior” (according to the CMAI manual). In contrast to the baseline factor analyses of CMAI in nursing homes, in our study an extra factor (Factor 4) was revealed which included four items; three items (spitting, cursing or verbal aggression, and screaming) of these, four items are included in the CMAI Factor 1 (“Aggressive behavior”) whilst the item “Making verbal sexual advances” was not considered in the initial CMAI factor analysis in nursing homes shown in the CMAI manual. However, the fact that these three items are already considered in the CMAI Factor 1 “aggressive behavior” limits the relevance of our finding, allowing us to conclude that a further subdivision into four factors cannot have any clinical relevance.
Our study is also in agreement with other studies which suggest the presence of four factors of CMAI [,,].
Moreover, we attempted to conduct a further factor analysis on the twenty items with a prevalence greater than 10%. Therefore, nine items (spitting kicking, biting, scratching, intentional falling, eating/drinking inappropriate substances, hurt self or other, making verbal sexual advances, and making physical sexual advances) were excluded. It must be remarked that three of the items we excluded (eating/drinking inappropriate substances, hurt self or others, and making verbal sexual advances) were also excluded from the factor analysis with three factors shown in the CMAI manual.
The three constrained factors found were: (1) Factor 1 comprised six items, of which four corresponded to Factor 3 of CMAI, which is “Verbally agitated behavior”, with the exception of the items of strange noises (weird laughter or crying) that are not considered in the analysis according to the CMAI manual, as well as the item of general restlessness that could be more pertinent to this factor than to the CMAI Factor 2. (2) Out of the seven items of Factor 2, five corresponded to CMAI Factor 1 which was “Aggressive behavior”, whilst the item throwing things was not considered by the Factor analysis reported in the CMAI manual, while the item “screaming” was considered in the CMAI Factor 3 “Verbally agitated behavior”. It is worth mentioning that the item “tearing things or destroying property” which loads to Factor 1 of the CMAI manual was also loading on Factor 3. (3) Finally, Factor 3 comprised seven items, from which five items corresponded to CMAI Factor 2 “Physically non-aggressive behavior”. The items “hiding things” and “hoarding things” were not considered in the factor analysis reported in the CMAI manual.
Besides the fact that several previous studies did not include every item occurring in a percentage lower than 5% or 10% in the factor analysis [,,,], we considered that by disregarding some items the estimation of the clinical involvement would be biased. It is worth considering that, especially in longitudinal studies such as RECage, in which there were many follow-up visits over three years, the exclusion of some items would lead to missing useful clinical information. New symptoms may occur or earlier symptoms may improve or worsen over time; Therefore, we think that it is crucial to calculate all the items in the final scores.
According to our results, there was no sharp distribution of the items on the retained factors, leading to some interpretation problems. Indeed, besides the fact that we excluded items with low frequency (≤10%) from the analysis, there was a great heterogeneity regarding the frequencies of some of the remaining items, with prevalence ranging from 14% to 78%. Specifically, the items “tearing things or destroying property”, “throwing things”, “hitting (including self)”, “grabbing onto people and pushing”, have a prevalence lower than 25%.
We think that the recommendation to discard some items in the patients’ scoring of the CMAI with a prevalence of ≤5% raises a big problem. Indeed, if they are not reckoned in the three factors suggested in the CMAI analysis according to its manual, there is the risk of attributing the same scores to patients with and without some of the excluded items. Taking into account that the total of the 9 items not considered ranges from 9 to 63, it is possible to have a very different evaluation of the severity of the disease. In addition, this does not allow appropriate quantification of the pattern of the CMAI change over the time within the same person.

4.1. New Factors Proposal by Following a More Clinical Approach

We think that it would be useful to adopt a more sensible approach. A first suggestion could be to consider the excluded items of the 29 or 36 items of the CMAI in a separate cluster to be analyzed by itself. However, a more reasonable procedure would be to cluster all the CMAI items (versions of 29 or 37 items), according to a more “clinical approach”. Our proposal is that the 29 items of the CMAI be clustered as follows, according to De Vreese []:
Factor 1 according to CMAI manual was “Aggressive physical behavior”. In this section we could include the following eleven items “3—Spitting (F1)”, “7—Hitting (including self) (F1)”, “8—Kicking (F1)”, “9—Grabbing onto people (F1)”, “10—Pushing (F1)”, “11—Throwing things (NC)”, “14—Biting (F1)”, “15—Scratching (F1)”, “21—Hurt self or other (NC)”, “25—Tearing things or destroying property (F1)”, and “28—Making physical sexual advances (NC)”. All the above items/symptoms refer to behaviors that are quite aggressive and are expressed via a physical means. The items “throwing things”, “hurt self or other” and “making physical sexual advances”, which are not considered based on frequency criteria in factor one, should be included by following clinical criteria in this factor, since such behaviors have a totally aggressive and physically harmful impact to patient’s life or to other’s life.
On the other hand, Factor 2, according to the CMAI manual comprise the “Physical non aggressive behavior” and we believe that it should include the following 10 items: “1—Pace, aimless wandering (F2)”, “2—Inappropriate dress or disrobing (F2)”, “16—Trying to get to a different place (F2)”, “17—Intentional falling (NC)”, “20—Eating/drinking inappropriate substances (NC)”, “22—Handling things inappropriately (F2)”, “23—Hiding things (NC)”, “24—Hoarding things (NC)”, “26—Performing repetitious mannerisms (F2)”, “29—General restlessness (F2)”. The items “intentional falling”, “eating/drinking inappropriate substances”, as well as “hiding and hoarding things”, that are not considered factors, are believed to be clinically categorized in physically non-aggressive behavior, since they are very annoying symptoms expressed via the body but are not harmful to anyone.
Finally, Factor 3 which comprises the “Aggressive verbal behaviors” in our opinion could include the following 8 items: “4—Cursing or verbal aggression (F1)”, 5—Constant unwarranted request for attention or help (F3)”, “6—Repetitive sentences or questions (F3)”, “12—Strange noises (weird laughter or crying) (NC)”, “13—Screaming (F3)”, “18—Complaining (F3)”, “19—Negativism (F3)”, and “27—Making verbal sexual advances (NC)”. Regarding the items “strange noises” and “making verbal sexual advances”, besides the fact that they do not physically harm a person, are aggressive and annoying behaviors in a verbal way.
It is worth mentioning that the three above reported factors comprise the items included in the CMAI Factors 1, 2, and 3, by including the items related to the pertinent behaviors not included in the CMAI factor analysis because of their low prevalence. Therefore, it is essential to suggest a CMAI scoring method that considers all twenty-nine items based on a sound clinical approach, as De Vreese first proposed [].
Of course, the proposed “clinical structure” with three factors has to be validated in longitudinal studies in comparison to the “classical” three-factor CMAI structure reported in the CMAI manual in order to establish the more appropriate method for capturing the changes in symptomatology.

4.2. Study’s Strengths

As mentioned above, the CMAI’s Factor Analysis was conducted as a part of the RECage study, which was a longitudinal and multicultural study that comprised 11 clinical centers from six European countries (Italy, Germany, France, Greece, Switzerland, and Norway) and included a sample of 505 patients. It should be stated that, generally speaking, the sample size for studies with Factor Analysis should be at least 300 participants, whilst the variables that are subjected to factor analysis should have at least 5 to 10 observations each [,]. Besides the fact that many studies have investigated the CMAI factor structure, several of them included a limited number of participants and samples from the same country. Several of them are also referred to in the CMAI manual [,,,,,].
Furthermore, a second strength concerns the suggestion of a new, more clinical approach to the CMAI assessment which includes all items, avoiding exclusion of them based on their reduced prevalence. This new approach is based on grouping the items regarding their clinical meaning; therefore, important information regarding the clinical course of dementia and the effectiveness of the treatment will not missed.

5. Conclusions

Our study identified a similar underlying construct of the CMAI reported in its pertinent manual with a three-factor model characterized by: (a) Aggressive behavior, (b) Physically non-aggressive behavior, and (c) Verbally agitated behavior. Of course, there are several studies that found and proposed a different factor structure, a fact that makes sense since the clinical population differs, especially when the samples come from different cultural environments.
Furthermore, a confirmatory FA on our data did not confirm the three-factor structure proposed in the CMAI manual. We propose instead a three-dimensional model that includes all the CMAI items based on their clinical characteristics, aiming to include all items in the scoring procedure.

Author Contributions

Conceptualization, B.M.C. and E.P.; methodology, B.M.C.; software, B.M.C.; validation, B.M.C. and E.P.; formal analysis, B.M.C.; investigation, M.T., S.B., A.C., E.C., A.F., S.F., G.B.F., L.F., P.M. (Patrizia Mecocci), P.M. (Paola Merlo), O.P. and C.A.D.; resources, C.A.D.; data curation, M.C.J.; writing—original draft preparation, B.M.C. and E.P.; writing—review and editing, B.M.C., E.P., M.T., S.B., A.C., E.C., A.F., S.F., G.B.F., L.F., P.M. (Patrizia Mecocci), P.M. (Paola Merlo), O.P. and C.A.D.; resources, C.A.D.; visualization, B.M.C., E.P., M.T., S.B., A.C., E.C., A.F., S.F., G.B.F., L.F., P.M. (Patrizia Mecocci), P.M. (Paola Merlo), O.P. and C.A.D.; resources, C.A.D.; supervision, B.M.C., E.P., M.T., S.B., A.C., E.C., A.F., S.F., G.B.F., L.F., P.M. (Patrizia Mecocci), P.M. (Paola Merlo), O.P. and C.A.D.; resources, C.A.D.; project administration, C.A.D.; funding acquisition, C.A.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the European Union’s Horizon 2020 research and innovation programme under grant number No. 779237.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by independent Ethical Committees in the countries and centers that participated in this project and returned to the coordinating center that has experience of coordinating ethics approvals from multiple institutions.

Data Availability Statement

Data are available from Bruno Cesana and Carlo Alberto Defanti.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 5th ed.; American Psychiatric Publishing: Arlington, VA, USA, 2013. [Google Scholar]
  2. Cummings, J.; Mintzer, J.; Brodaty, H.; Sano, M.; Banerjee, S.; Devanand, D.P.; Gauthier, S.; Howard, R.; Lanctôt, K.; Lyketsos, C.G.; et al. Agitation in Cognitive Disorders: International Psychogeriatric Association Provisional Consensus Clinical and Research Definition. Int. Psychogeriatr. 2014, 27, 7–17. [Google Scholar] [CrossRef] [PubMed]
  3. Tumati, S.; Herrmann, N.; Marotta, G.; Li, A.; Lanctôt, K.L. Blood-Based Biomarkers of Agitation in Alzheimer’s Disease: Advances and Future Prospects. Neurochem. Int. 2022, 152, 105250. [Google Scholar] [CrossRef] [PubMed]
  4. Ballard, C.; Aarsland, D.; Francis, P.; Corbett, A. Neuropsychiatric Symptoms in Patients with Dementias Associated with Cortical Lewy Bodies: Pathophysiology, Clinical Features, and Pharmacological Management. Drugs Aging 2013, 30, 603–611. [Google Scholar] [CrossRef] [PubMed]
  5. Laganà, V.; Bruno, F.; Altomari, N.; Bruni, G.; Smirne, N.; Curcio, S.; Mirabelli, M.; Colao, R.; Puccio, G.; Frangipane, F.; et al. Neuropsychiatric or Behavioral and Psychological Symptoms of Dementia (BPSD): Focus on Prevalence and Natural History in Alzheimer’s Disease and Frontotemporal Dementia. Front. Neurol. 2022, 13, 832199. [Google Scholar] [CrossRef]
  6. Panca, M.; Livingston, G.; Barber, J.; Cooper, C.; La Frenais, F.; Marston, L.; Cousins, S.; Hunter, R.M. Healthcare Resource Utilisation and Costs of Agitation in People with Dementia Living in Care Homes in England—The Managing Agitation and Raising QUality of LifE in Dementia (MARQUE) Study. PLoS ONE 2019, 14, e0211953. [Google Scholar] [CrossRef]
  7. Livingston, G.; Barber, J.; Marston, L.; Rapaport, P.; Livingston, D.; Cousins, S.; Robertson, S.; La Frenais, F.; Cooper, C. Prevalence of and Associations with Agitation in Residents with Dementia Living in Care Homes: MARQUE Cross-Sectional Study. BJPsych Open 2017, 3, 171–178. [Google Scholar] [CrossRef]
  8. Halpern, R.; Seare, J.; Tong, J.; Hartry, A.; Olaoye, A.; Aigbogun, M.S. Using Electronic Health Records to Estimate the Prevalence of Agitation in Alzheimer Disease/Dementia. Int. J. Geriatr. Psychiatry 2018, 34, 420–431. [Google Scholar] [CrossRef]
  9. Anatchkova, M.; Brooks, A.; Swett, L.; Hartry, A.; Duffy, R.A.; Baker, R.A.; Hammer-Helmich, L.; Sanon Aigbogun, M. Agitation in Patients with Dementia: A Systematic Review of Epidemiology and Association with Severity and Course. Int. Psychogeriatr. 2019, 31, 1305–1318. [Google Scholar] [CrossRef]
  10. Cummings, J.L.; Mega, M.; Gray, K.; Rosenberg-Thompson, S.; Carusi, D.A.; Gornbein, J. The Neuropsychiatric Inventory: Comprehensive Assessment of Psychopathology in Dementia. Neurology 1994, 44, 2308. [Google Scholar] [CrossRef]
  11. Sultzer, D.L.; Levin, H.S.; Mahler, M.E.; High, W.M.; Cummings, J.L. Assessment of Cognitive, Psychiatric, and Behavioral Disturbances in Patients with Dementia: The Neurobehavioral Rating Scale. J. Am. Geriatr. Soc. 1992, 40, 549–555. [Google Scholar] [CrossRef]
  12. Reisberg, B.; Borenstein, J.; Salob, S.P.; Ferris, S.H.; Franssen, E.; Georgotas, A. Behavioral Symptoms in Alzheimer’s Disease: Phenomenology and Treatment. J. Clin. Psychiatry 1987, 48, 9–15. [Google Scholar]
  13. Rosen, J.; Burgio, L.; Kollar, M.; Cain, M.; Allison, M.; Fogleman, M.; Michael, M.; Zubenko, G.S. The Pittsburgh Agitation Scale: A User-Friendly Instrument for Rating Agitation in Dementia Patients. Am. J. Geriatr. Psychiatry 1994, 2, 52–59. [Google Scholar] [CrossRef] [PubMed]
  14. Cohen-Mansfield, J.; Marx, M.S.; Rosenthal, A.S. A Description of Agitation in a Nursing Home. J. Gerontol. 1989, 44, M77–M84. [Google Scholar] [CrossRef] [PubMed]
  15. Cohen-Mansfield, J. Instruction Manual for the Cohen-Mansfield Agitation Inventory (CMAI); Research Institute of the Hebrew Home of Greater Washington: Washington, DC, USA, 1991. [Google Scholar]
  16. Cohen-Mansfield, J. Agitated Behaviors in the Elderly: II. Preliminary Results in the Cognitively Deteriorated. J. Am. Geriatr. Soc. 1986, 34, 722–727. [Google Scholar] [CrossRef] [PubMed]
  17. Choy, C.N.P.; Lam, L.C.W.; Chan, W.C.; Li, S.W.; Chiu, H.F.K. Agitation in Chinese Elderly: Validation of the Chinese Version of the Cohen-Mansfield Agitation Inventory. Int. Psychogeriatr. 2001, 13, 325–335. [Google Scholar] [CrossRef]
  18. Zuidema, S.U.; de Jonghe, J.F.M.; Verhey, F.R.J.; Koopmans, R.T.C.M. Agitation in Dutch Institutionalized Patients with Dementia: Factor Analysis of the Dutch Version of the Cohen-Mansfield Agitation Inventory. Dement. Geriatr. Cogn. Disord. 2006, 23, 35–41. [Google Scholar] [CrossRef]
  19. Sommer, O.H.; Kirkevold, Ø.; Cvancarova, M.; Engedal, K. Factor Analysis of the Brief Agitation Rating Scale in a Large Sample of Norwegian Nursing Home Patients. Dement. Geriatr. Cogn. Disord. 2010, 29, 55–60. [Google Scholar] [CrossRef]
  20. Cohen-Mansfield, J.; Werner, P.; Watson, V.; Pasis, S. Agitation among Elderly Persons at Adult Day-Care Centers: The Experiences of Relatives and Staff Members. Int. Psychogeriatr. 1995, 7, 447–458. [Google Scholar] [CrossRef]
  21. Koss, E.; Weiner, M.; Ernesto, C.; Cohen-Mansfield, J.; Ferris, S.H.; Grundman, M.; Schafer, K.; Sano, M.; Thal, L.J.; Thomas, R.; et al. Assessing Patterns of Agitation in Alzheimer’s Disease Patients with the Cohen-Mansfield Agitation Inventory. Alzheimer Dis. Assoc. Disord. 1997, 11, 45–50. [Google Scholar] [CrossRef]
  22. Rabinowitz, J.; Davidson, M.; De Deyn, P.P.; Katz, I.; Brodaty, H.; Cohen-Mansfield, J. Factor Analysis of the Cohen-Mansfield Agitation Inventory in Three Large Samples of Nursing Home Patients with Dementia and Behavioral Disturbance. Am. J. Geriatr. Psychiatry 2005, 13, 991–998. [Google Scholar] [CrossRef]
  23. Patrick, K.S.; Gunstad, J.; Martin, J.T.; Chapman, K.R.; Drost, J.; Spitznagel, M.B. Specific Agitation Behaviours in Dementia Differentially Contribute to Aspects of Caregiver Burden. Psychogeriatrics 2022, 22, 688–698. [Google Scholar] [CrossRef] [PubMed]
  24. Altunoz, U.; Ozel Kizil, E.T.; Kirici, S.; Bastug, G.; Bicer Kanat, B.; Sakarya, A.; Er, O.; Turan, E. Dimensions of Agitation Based on the Cohen-Mansfield Agitation Inventory in Patients with Dementia. Turk. J. Psychiatry 2014, 26, 1–6. [Google Scholar] [CrossRef]
  25. Schnelli, A.; Ott, S.; Mayer, H.; Zeller, A. Factors Associated with Aggressive Behaviour in Persons with Cognitive Impairments Using Home Care Services: A Retrospective Cross-Sectional Study. Nurs. Open 2021, 8, 1345–1359. [Google Scholar] [CrossRef]
  26. Griffiths, A.W.; Albertyn, C.P.; Burnley, N.L.; Creese, B.; Walwyn, R.; Holloway, I.; Safarikova, J.; Surr, C.A. Validation of the Cohen-Mansfield Agitation Inventory Observational (CMAI-O) Tool. Int. Psychogeriatr. 2019, 32, 75–85. [Google Scholar] [CrossRef]
  27. Poptsi, E.; Tsolaki, M.; Bergh, S.; Cesana, B.M.; Ciccone, A.; Fabbo, A.; Frisoni, G.B.; Frölich, L.; Lavolpe, S.; Guazzarini, A.G.; et al. Rationale, Design, and Methodology of a Prospective Cohort Study for Coping with Behavioral and Psychological Symptoms of Dementia: The RECage Project. J. Alzheimer’s Dis. JAD 2021, 80, 1613–1627. [Google Scholar] [CrossRef]
  28. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-Mental State”: A Practical Method for Grading the Cognitive State of Patients for the Clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef] [PubMed]
  29. Galasko, D.; Bennett, D.; Sano, M.; Ernesto, C.; Thomas, R.; Grundman, M.; Ferris, S. An Inventory to Assess Activities of Daily Living for Clinical Trials in Alzheimer’s Disease. Alzheimer Dis. Assoc. Disord. 1997, 11, 33–39. [Google Scholar] [CrossRef]
  30. SAS/STAT 9.3 User’s Guide. Available online: https://support.sas.com/documentation/onlinedoc/stat/930/ (accessed on 25 June 2023).
  31. O’Rourke, N.; Psych, R.; Hatcher, L. A Step-by-Step Approach to Using SAS for Factor Analysis and Structural Equation Modeling, 2nd ed.; SAS Institute: Cary, NC, USA, 2013. [Google Scholar]
  32. Brown, T.A. Confirmatory Factor Analysis for Applied Research; Guilford: New York, NY, USA, 2015. [Google Scholar]
  33. Zare, M.; Birashk, B.; Ebrahimi, A.A. Reliability, Validity and Factor Analysis of Cohen-Mansfield Agitation Inventory (CMAI). Iran. J. Psychiatry Clin. Psychol. 2012, 18, 67–73. [Google Scholar]
  34. Novità in Medicina–GP Vecchi. Available online: https://www.gpvecchi.org/novita-in-medicina (accessed on 25 June 2023).
  35. Yong, A.G.; Pearce, S. A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis. Tutor. Quant. Methods Psychol. 2013, 9, 79–94. [Google Scholar] [CrossRef]
  36. Comrey, A.L.; Lee, H.B. A First Course in Factor Analysis; Psychology Press: London, UK, 2013. [Google Scholar]
  37. Zuidema, S.U.; Buursema, A.L.; Gerritsen, M.G.J.M.; Oosterwal, K.C.; Smits, M.M.M.; Koopmans, R.T.C.M.; de Jonghe, J.F.M. Assessing Neuropsychiatric Symptoms in Nursing Home Patients with Dementia: Reliability and Reliable Change Index of the Neuropsychiatric Inventory and the Cohen-Mansfield Agitation Inventory. Int. J. Geriatr. Psychiatry 2011, 26, 127–134. [Google Scholar] [CrossRef]
  38. Kupeli, N.; Vickerstaff, V.; White, N.; Lord, K.; Scott, S.; Jones, L.; Sampson, E.L. Psychometric Evaluation of the Cohen-Mansfield Agitation Inventory in an Acute General Hospital Setting. Int. J. Geriatr. Psychiatry 2017, 33, e158–e165. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Semantic Priming and Its Link to Verbal Comprehension and Working Memory in Children with Learning Disorders
Previous
Etomidate versus Propofol for Electroconvulsive Therapy in Patients with Major Depressive Disorders in Terms of Clinical Responses to Treatment: A Retrospective Analysis
Next