Next Article in Journal
Creating Healthy Change in the Preconception Period for Women with Overweight or Obesity: A Qualitative Study Using the Information–Motivation–Behavioural Skills Model
Next Article in Special Issue
Increased Psychological Distress during COVID-19 and Quarantine in Ireland: A National Survey
Previous Article in Journal
Metal on Metal Bearing in Total Hip Arthroplasty and Its Impact on Synovial Cell Count
Previous Article in Special Issue
A Predictive Study of Resilience and Its Relationship with Academic and Work Dimensions during the COVID-19 Pandemic
Article

Predicting Perceived Stress Related to the Covid-19 Outbreak through Stable Psychological Traits and Machine Learning Models

1
Associazione Novilunio Onlus, 35020 Ponte San Nicolò (PD), Italy
2
Department of General Psychology, University of Padua, 35131 Padua, Italy
3
Department of Human Neuroscience, Sapienza University of Rome, 00185 Rome, Italy
4
Department of Environmental Medicine and Public Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
5
Department of Political Science, Law, and International Studies, University of Padua, 35123 Padua, Italy
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
J. Clin. Med. 2020, 9(10), 3350; https://doi.org/10.3390/jcm9103350
Received: 22 September 2020 / Revised: 11 October 2020 / Accepted: 14 October 2020 / Published: 19 October 2020

Abstract

The global SARS-CoV-2 outbreak and subsequent lockdown had a significant impact on people’s daily lives, with strong implications for stress levels due to the threat of contagion and restrictions to freedom. Given the link between high stress levels and adverse physical and mental consequences, the COVID-19 pandemic is certainly a global public health issue. In the present study, we assessed the effect of the pandemic on stress levels in N = 2053 Italian adults, and characterized more vulnerable individuals on the basis of sociodemographic features and stable psychological traits. A set of 18 psycho-social variables, generalized regressions, and predictive machine learning approaches were leveraged. We identified higher levels of perceived stress in the study sample relative to Italian normative values. Higher levels of distress were found in women, participants with lower income, and participants living with others. Higher rates of emotional stability and self-control, as well as a positive coping style and internal locus of control, emerged as protective factors. Predictive learning models identified participants with high perceived stress, with a sensitivity greater than 76%. The results suggest a characterization of people who are more vulnerable to experiencing high levels of stress during the COVID-19 pandemic. This characterization may contribute to early and targeted intervention strategies.
Keywords: COVID-19; stress; personality; public health; mental health; coping COVID-19; stress; personality; public health; mental health; coping

1. Introduction

SARS-Cov-2 (severe acute respiratory syndrome coronavirus 2; henceforth referred to as COVID-19) is a strain of coronavirus that can infect humans, attacking the lungs and causing symptoms ranging from those of the common cold to those of severe acute respiratory syndrome (SARS) [1]. While approximately 80% of those who are infected recover with no special treatment (i.e., they are either asymptomatic or suffer from mild pneumonia) [2], recent data have confirmed that older persons (60+ years old) [3] and persons with certain pre-existing medical conditions are more likely to develop serious respiratory distress that can lead to death (3–4% of the population) [4]. COVID-19 spreads very easily between persons and, at the time of writing, no drugs or biologics have proven effective for preventing or treating the virus [5,6].
COVID-19 was first identified in the Chinese region of Wuhan in December 2019 [6]. Between December 2019 and April 2020, the virus spread throughout the world, causing more than 5,000,000 infections and over 300,000 deaths [7]. On 11 March 2020, the World Health Organization (WHO) declared COVID-19 a pandemic [8]. To contain the number of victims and prevent the collapse of the healthcare system, most national governments imposed strict restrictions on residents’ freedom, forcing those infected with the virus to self-isolate and requiring all residents (infected or not) to stay at home and move through public areas only for reasons of absolute necessity. Among Western governments, the Italian government was the first to apply such restrictions [9].

1.1. Psychological Impact of the Pandemic

A few weeks after the spread of COVID-19 in China, the first scientific studies investigating the psychological impact of the outbreak highlighted a mild to severe negative psychological impact of the event within a significant proportion of the Chinese population (53.8%); specifically, this impact on mental health was reported to include symptoms of anxiety, depression, and stress [10,11]. More broadly, the literature on the impact of infectious outbreaks on mental health shows that pandemics are extremely stressful events that force people to cope with totally unexpected, ambiguous, and uncertain situations [12]. Specifically, two main aspects of pandemics have been found to affect people’s mental state. The first relates to danger (i.e., the fear of contagion), which can increase perceived threat and sometimes lead to panic, behavioral contagion, and an emotional epidemic [13,14]. The second regards the multiple and rapid changes to social, working, and familiar habits, due to self-isolation and social distancing measures [15,16,17,18,19]. The longer the duration of self-isolation, the more people experience frustration and boredom, along with concerns about infection [15].
Well-documented psychological reactions to epidemics include emotional distress, anxiety behaviors, sleeping disorders, fear, anger, depression, health concerns, a sense of powerlessness, and uncertainty [13,16,20,21,22,23,24]. Furthermore, studies examining the long-term consequences of infectious epidemics have shown that some individuals may even develop symptoms of post-traumatic stress disorder (PTSD) [12,25,26]. One review indicated that those who develop PTSD may experience the symptoms for 3 years following the end of the epidemic [15,23].
Stress is defined as an adaptive psycho-physical reaction to a physical, social or psychological stimulus, called a stressor [27]. Stress-related responses may be cognitive, emotional, behavioral, or physiological. Depending on the type, timing, and severity of exposure to a stressor, the resulting stress may become a risk factor for a number of illnesses, including those of a psychiatric or cardiovascular nature [28,29,30,31,32]. An emergency such as the COVID-19 outbreak might rightly be considered a severe stressor, as it is a new and unexpected situation with a potentially serious impact on health (experienced both personally and through loved ones) that also involves social restrictions [13]. Nevertheless, no event, in and of itself, is the precipitating cause of pathology and illness. Rather, it is the perception of stress (i.e., the degree to which one considers the event stressful) that accounts for the varying physical and mental responses to the situation [33]. In this sense, it is important to detect vulnerable persons early, and to promote effective preventive programs in order to treat such persons rapidly and limit negative psychological outcomes. The identification of psycho-social risk and resilience factors for psychological distress during the COVID-19 emergency comprises a significant step in this direction [34].

1.2. Risk and Protective Factors for Psychological Outcomes During the Pandemic

To date, studies on the psychological impact of COVID-19 have mainly focused on the role of sociodemographic variables (e.g., gender, age, education level, and social connections) in moderating reactions to the outbreak [10,11,24,35,36]. The identified sociodemographic risk factors for psychological distress include gender (female), age (18–30 or 60+ years old), student status, education level, perception of the public health system, specific physical symptoms (e.g., coryza, cough, sore throat, headache), and a low reported level of health [10,24,35]. For Chinese students, living in an urban area, having a stable family income, living with parents, and having good social support were also found to protect against anxiety [11].
In addition to investigating sociodemographic factors, many studies have also outlined the role of certain dispositional traits in modulating responses to stressful events. However, these constructs have been poorly investigated in relation to the psychological impact of epidemics and, specifically, COVID-19. More generally, research has shown how individual differences, including dispositional traits, can explain life outcomes [37]. According to the theory of traits (or dispositional theory), individual differences may be explained by certain predispositions (traits), which are expressed in a relatively stable way across situations and time. Traits comprise a person’s manner of thinking, feeling, perceiving, and relating to others [37]. Based on these considerations, dispositional traits might play a relevant role in predicting perceived stress in relation to COVID-19.
Coping is one of the most widely studied dispositional traits, and it has been found to be significant in modulating responses to stressful events. Coping is defined as the effort to solve personal and interpersonal problems in an attempt to master, minimize, or tolerate stress and conflict [38]. Distinct coping strategies have been found to be differentially associated with specific emotional responses [39], physiological stress responses [40], and self-efficacy [41,42]. One investigation into the role of coping strategies during a virus outbreak (2009 H1N1 flu pandemic) found coping style to influence the perceived risk of contagion and vaccination intentions among Canadian adults [43]. Another study, based in Singapore, found coping strategies to be associated with post-traumatic outcomes within visitors to community health care services during the national outbreak of SARS [23].
As regards emotional self-regulation and adaptation to the world, self-control may represent a significant protective factor. There is empirical evidence that people with high dispositional self-control have better psychological adjustment and impulse control [44]; this suggests that good self-control may mitigate the influence of a negative environment. Similarly, perceived control over life outcomes has been shown to be positively associated with well-being and health-related quality of life, and negatively associated with emotional distress, in the context of stressful events [45,46]. In this regard, locus of control [47] is a relevant concept, describing the degree to which an individual believes that they have control over the outcome of life events, as opposed to feeling that their life is subject to external forces beyond their control. Finally, among the many individual difference variables that might influence reactions to COVID-19, personality traits merit significant attention. Several studies have highlighted an association between the Big Five personality traits [48] and various health behaviors, such as sedentary behavior [49], sexual health behavior [50], physical activity [51], and alcohol consumption [52].
The first aim of the present study was to investigate the impact of the COVID-19 pandemic and the related government-imposed restrictions on perceived stress in a Western country (i.e., Italy). As reported above, most studies on the psychological impact of COVID-19 have related to the Chinese population. However, countries differ from one another in many important aspects (i.e., social, cultural, political, and economic aspects, to name only a few); consequently, psychological responses may also vary between contexts and communities, revealing unique qualitative and quantitative psychological reactions and psychological needs.
Hypothesis 1
(H1).Our sample of Italian adults, collected during the COVID-19 outbreak, would show higher levels of perceived stress compared to Italian normative values.
The second aim of the study was to confirm the role of certain sociodemographic factors in modulating stress responses to the COVID-19 pandemic, as reported in the recent literature.
Hypothesis 2
(H2).(a) Participants who were female, younger, and students, with a lower level of education and lower income, would report higher levels of stress, and (b) participants who were living with others would report lower levels of psychological distress.
The third aim of the work was to investigate the association between certain stable psychological traits and psychological distress relating to the current situation. To this end, participants were tested for coping strategies, self-control, locus of control, and select personality traits.
Hypothesis 3
(H3).(a) Participants with positive coping strategies, higher levels of self-control, an internal locus of control, and higher levels of emotional stability would report lower levels of stress, and (b) participants with negative coping strategies, lower levels of self-control, an external locus of control, and lower levels of emotional stability would report higher levels of stress.
Finally, with the goal of anticipating persons in need of treatment and improving the targeting and overall effectiveness of preventive programs, we aimed at developing machine learning models to predict individual psychological responses to the COVID-19 pandemic, based on sociodemographic and psychological variables with maximal sensitivity in classifying subjects with high versus low levels of perceived stress.
To summarize, the study was novel in the following two respects: first, it considered the role of not only sociodemographic variables, but also stable psychological traits, as predictors of a stressful reaction to COVID-19; and second, it leveraged machine learning techniques to identify people at the greatest risk of developing severe and negative psychological outcomes due to the pandemic.

2. Materials and Methods

2.1. Measures

To test the abovementioned hypotheses, we implemented a cross-sectional study. Using Google Forms, we designed an ad hoc online questionnaire to collect data on participants’ stress reactions to COVID-19, demographical variables, and psychological traits. The questionnaire also assessed the following sociodemographic factors: gender, age, education, number of family members and/or others living in the household, monthly household income, and student status. Subsequently, we administered five standardized questionnaires, as follows:
  • The Italian Version of the 10-item Perceived Stress Scale (PSS-10; Cronbach’s alpha = 0.74) [53]. The PSS-10 is a frequently used psychological instrument to measure perceived stress [54]. Respondents are asked to answer 10 questions pertaining to the frequency of experiences of stressful situations during the last month on a five-point scale ranging from 0 (never) to 4 (very often) [55,56]. Example items include “In the last month, how often have you been upset because of something that happened unexpectedly?” Higher scores indicate higher levels of perceived stress. Moreover, in the present study, the score corresponding to 1.5 SD above the Italian normative score [53] was used as a cut-off to divide participants into two classes: low perceived stress (males: PSS-10 score <24.35; females: PSS-10 score <24.55) and high perceived stress (males: PSS-10 score ≥24.35; females: PSS score-10 ≥24.55);
  • The Italian Shortened Version of the Coping Orientations to the Problems Experienced (COPE-NVI-25; Cronbach’s alpha of factors range 0.68–0.95) [57]. The COPE-NVI-25 is a multi-dimensional inventory that assesses individual differences in coping styles. It is comprised of 25 items, which are rated on a 4-point scale ranging from 1 (I usually don’t do this at all) to 4 (I usually do this a lot) [58]. The instrument includes five subscales corresponding to five different coping styles: social support, avoidance strategies, positive attitude, problem solving, and turning to religion [59]. An example item is “I admit to myself that I can’t deal with it, and quit trying” (avoidance strategies). A higher score on a particular subscale indicates a greater use of that specific coping strategy.
  • The Italian translation of the Brief Self-Control Scale (BSCS; Cronbach’s alpha = 0.85) [44]. The BSCS measures individual differences in dispositional capacity for self-control. The scale is comprised of 13 items that are rated on a five-point scale ranging from 1 (not at all) to 5 (very much). An example item is “I do certain things that are bad for me, if they are fun.” Higher scores on the BSCS indicate a greater capacity for self-control, and they are also correlated with better psychological adjustment, interpersonal skills, and emotional responses [44];
  • The Italian Short Version of the Locus of Control (LOC) Scale [60]. This 20-item questionnaire is used to measure generalized expectancies relating to an internal versus external locus of control, rated via dichotomous options (“Yes” vs. “No”), similar to Rotter’s original Internal–External Locus of Control Scale [47]. An example item is “To do well in life, luck is more important than commitment.” Respondents with an internal locus of control (i.e., a high score on the internal LOC scale) tend to attribute life outcomes—and general life events—to their own behavior, whereas those with a prevalent external locus of control (i.e., a high score on the external LOC scale) tend to attribute life events to fate, others, or external causes beyond their control [47,61];
  • The Italian Version of the 10-item Big Five Inventory (BFI-10; Spearman–Brown coefficients ≥0.50) [62]. The BFI-10 assesses personality traits according to the five-factor approach [63]. It is comprised of 10 items rated on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree), measuring five dimensions of personality, which are extraversion, agreeableness, conscientiousness, neuroticism (or, if reversed, emotional stability), and openness [64]. An example item is “I see myself as someone who is outgoing, sociable” (extraversion). The higher the score on a particular subscale, the more that specific dimension represents a characteristic trait of the respondent’s personality.
The complete list of 18 variables that were extracted from the responses to the questionnaire is provided in the Supplementary Materials. The present research was designed in accordance with the Declaration of Helsinki and approved by the ethics committee for psychological research at the University of Padova (protocol number 3576, unique code 189B46FE116994F1A8D1077B835D83BB).

2.2. Participants and Procedure

Data were collected during the period of 20–31 March, 2020. Participants were recruited online through an invitation posted on social media (Facebook and WhatsApp). This approach of online recruitment was selected primarily due to the lockdown situation, which prevented us from collecting data in the laboratory. According to the aim of the study, it was necessary for us to capture the psychological state of participants at the time of the pandemic; thus an a posteriori study would not have provided useful and reliable information.
Participants were invited to complete an anonymous online questionnaire to report their personal experiences with the COVID-19 emergency and their mental state. The inclusion criteria were the following: (a) living in Italy at the time of data collection and (b) being aged 18+ years (18 years is the legal age in Italy, defined by the capacity to act and be emancipated). Participation was voluntary. All participants were required to read and provide informed consent before beginning the online questionnaire. They received no compensation for their participation. In total, 2072 volunteers took part in the study. Of these, 19 were excluded on the basis that they responded to the questionnaire twice (we kept only their first response). Thus, the final sample was comprised of 2053 participants, of whom 1555 were female, 480 were male, and 18 were reported as “other.” The participants’ average age was 35.81 (SD = 13.19; range: 18–83) and their average education level was 15.35 years (SD = 3.43; range: 5–21). A more detailed description of the sample’s demographic characteristics is provided in the Supplementary Materials. It has been calculated that a sample size of 2053 is sufficiently large to achieve at least a statistical power (1-β) = 0.90 in a linear multivariable regression analysis involving 18 predictors, given a significance level α = 0.05 and an effect size of 0.356 [65]. Data are provided in the Supplementary Materials.

2.3. Data Analysis Methodology

2.3.1. Statistics

Data analysis was conducted using the JASP software [66]. A single sample t-test (t, two-sided) was performed in relation to the PSS-10 score, in order to determine whether the sample’s true mean (μ) was statistically different from that of the known population (m0). A multivariable regression analysis was run to investigate the relationship between the PSS-10 score and the independent variables that were hypothesized to impact the level of perceived stress. The collinearity assumption was checked prior to running the model, using the tolerance and variance inflation factor (VIF). As a rule of thumb, if VIF >10 and tolerance <0.1, the assumption is greatly violated, whereas if VIF >1 and tolerance <0.2, the model may be biased [67]. The results indicated that the collinearity assumption was not violated by any of the independent variables entered in the regression model. The analysis was performed using the stepwise variable selection method, which identified predictors with a significant (p < 0.05) individual association with the outcome (PSS-10 score). The results were reported using unstandardized coefficients, as recommended by Friedrich [68].

2.3.2. Classification Models

Recently, researchers in different scientific fields, including the clinical and social sciences, have emphasized the utility of focusing on prediction, rather than explanation, during data analysis [69,70,71,72]. This increased attention to predictive models may be largely attributed to the significant spread of machine learning (ML)—a branch of artificial intelligence that trains algorithms on data samples (i.e., training sets) in order to make predictions on completely new data (i.e., test sets) without being explicitly programmed to do so [73]. As regards psychology, ML techniques have been shown to be particularly useful for predicting human behavior, including high-risk behavior; thus, they may be applied to improve the effectiveness and targeting of preventive programs and interventions [74]. In brief, ML models are capable of predicting the behavior of individual subjects, allowing greater attention to be paid to those considered most critical [69].
In the present study, ML algorithms were trained on psycho-social data to identify subjects who were more likely to present high levels of perceived stress during the COVID-19 emergency, and who were consequently at the greatest risk of developing psychological symptoms, including those of PTSD. For this purpose, participants were split into two classes: high perceived stress and low perceived stress. The high perceived stress class included participants with a PSS-10 score of more than 1.5 SD above the Italian population mean (n = 393) for men and women, respectively. Conversely, the low perceived stress class included participants whose PSS-10 did not exceed 1.5 SD above the Italian normative value (n = 1642). It should be noted that participants who reported their gender as “other” (n = 18) were excluded from this analysis, as the Italian normative values were available for males and females only [53].
As ML models are built to fit particular data, it is important to test how each model fits new (i.e., unseen) data. For this reason, part of the data (the training set) is generally used to train and validate the model, while another part (the test set) is used to test the model’s accuracy on new examples [73,75]. This procedure guarantees the model generalization and increases the replicability of the results [76,77]. In the present study, 20% [73,75] of the participants were randomly chosen and retained as the test set. Accordingly, the training set consisted of 1628 participants (314 with high perceived stress and 1314 with low perceived stress), and the test set consisted of 407 participants (79 with high perceived stress and 328 with low perceived stress).
In the first step, feature selection was performed to remove redundant and irrelevant features and to increase model generalization by reducing overfitting and noise in the data [78]. A good strategy for feature selection is to identify the subset of features that are highly correlated with the class to predict, but not correlated with each other [78]. This procedure was performed in the present study using the correlation-based feature selector (CFS) in the WEKA 3.9 software [79].
The problem of class imbalance was addressed while running the classification algorithms. The ratio between participants with high perceived stress and those with low perceived stress was approximately 1:5. As ML methods work best with balanced datasets, it is necessary to account for any class imbalance, especially when training examples are limited—a condition that is frequently met by datasets in health and clinical psychology [80]. At the same time, it is equally important for ML models to be built on samples that are representative of the population, reflecting real distribution [80].
One strategy to overcome these two limitations consists of altering the relative costs associated with misclassifying the minority and majority classes, in order to compensate for the class imbalance [81]. In the present study, ML algorithms were set in such a way that any algorithmic error made in classifying the minority class (high perceived stress) was weighted four times more than any error in classifying the majority class (low perceived stress). This cost-modifying strategy has been shown to provide better results than other methods in addressing the class imbalance problem [81]. Moreover, it should be noted that, for the goal of the present task, it was more beneficial to minimize false negatives than to minimize false positives (i.e., to have a model with high sensitivity rather than high specificity). In other words, it was more important to identify people who were truly at risk than to avoid misclassifying people who were not truly at risk.
ML models were trained and validated on the training sample (n= 1628) through a 10-fold cross-validation procedure using the WEKA 3.9 software [79]. The different algorithms (i.e., logistic regression [82], support vector machine (SVM) [83], Naïve Bayes [84], random forest [85]) were chosen as representatives of different classification strategies, to ensure that the results would be stable across classifiers and not dependent on specific model assumptions (details on the parameters of the ML classifiers are reported in the Supplementary Materials). K-fold cross-validation is a resampling procedure that seeks to reduce the variance in model performance relative to the performance that may be obtained from a single training set and a single test set. The procedure consists of portioning the sample into k subsets (i.e., folds; in the present study, k = 10), and using k-1 (i.e., 9) subsets to train the model and the remaining subset to validate the model’s accuracy. This is repeated k (i.e., 10) times [86]. The final model metrics are obtained by averaging the metrics obtained in all validation subsets. In the present study, the models developed from the 10-fold cross-validation procedure were tested on the test sample (n = 407).

3. Results

The main results of the data analysis are reported in this section. A more complete descriptive analysis of each variable, including the composition of high perceived stress versus low perceived stress samples, is reported in the Supplementary Materials.

3.1. Perceived Stress in Response to the COVID-19 Emergency

The average PSS-10 score of the entire sample was 18.81 (SD = 6.25). Analyzing the responses of males and females separately (note that participants who reported a gender of “other” were excluded from this analysis due to a lack of normative data), males obtained an average score of 16.71 (SD = 6.91) and females obtained an average score of 19.44 (SD = 6.79). To determine whether the sample mean statistically differed from that of the Italian normative population (males: average = 15.2, SD = 6.1; females: average = 16.3, SD = 5.5) [53], a one-sample t-test was run separately for each gender. Statistically significant results emerged for both males (t(479) = 4.79, p < 0.001, d = 0.22, 95% CI for Cohen’s d (0.13, 0.31)) and females (t(1554) = 18.21, p < 0.001, d = 0.46, 95% CI for Cohen’s d (0.41, 0.510)). The PSS-10 items showing the greatest increase in participants with high perceived stress (1.5 SD above the population mean) were those related to loss of control over one’s life (M = 3.09, SD = 0.80) and frequently feeling nervous or stressed (M = 3.51, SD = 0.60) (see Supplementary Materials for an item-by-item analysis).

3.2. The Role of Sociodemographic Variables in Predicting Perceived Stress

A first multiple regression analysis was run, including sociodemographic variables that have been shown to potentially impact the level of perceived stress during a pandemic [10,11,24,35,36]. The PSS-10 score was set as the dependent variable, while gender (male), age, education, monthly income, number of family members, and student status (student) were entered as covariates. The final model accounted for a significant proportion of the variance in the level of perceived stress (R2 = 0.103, adjusted R2 = 0.101, F-change(1,2029) = 12.009, p < 0.001). All of the aforementioned variables, with the exception of student status, were found to contribute to the level of perceived stress. Results are reported in Table 1.

3.3. The Role of Stable Psychological Traits in Predicting Perceived Stress

To better understand the role of stable psychological traits in predicting the level of perceived stress (PSS-10 score), a second multiple linear regression was run, adding to the previous model the scores of the five coping styles measured by the COPE-NVI-25 (COPE positive, COPE problem, COPE avoidance, COPE religion and COPE support), the BSCS total score, the internal LOC score, and the scores for the five personality traits measured by the BFI-10 (BFI-10 agreeableness, BFI-10 conscientiousness, BFI-10 emotional stability, BFI-10 extraversion and BFI-10 openness). This second model accounted for a larger proportion of the variance in the level of perceived stress (R2 = 0.356, adjusted R2 = 0.352, F-change(1,2022) = 5.908, p < 0.05) compared to the previous model. BFI-10 emotional stability, COPE positive, age, BCSC total score, gender (male), COPE avoidance, internal LOC, number of family members, COPE support, monthly income, and BFI-10 conscientiousness were identified as significant predictors of the level of perceived stress during the COVID-19 epidemic (see Table 2). Education, COPE religion, COPE problem solving, BFI-10 agreeableness, BFI-10 extraversion and BFI-10 openness were excluded.

3.4. Machine Learning Classification Models

Ultimately, the 18 questionnaire variables were considered predictors of perceived stress. The entire list of predictors, along with their descriptions, is provided in the Supplementary Materials. Of these 18 variables, the following 7 were identified as the best set of predictors using correlation-based feature selection: age, monthly income, COPE avoidance, COPE positive, BSCS total score, BFI-10 emotional stability, and BFI-10 agreeableness. Using these predictors, ML algorithms were trained and tested according to the procedure described in the “Data Analysis” section. Classification results for the test set are reported in Table 3, which quantifies predictive performance according to the following metrics: receiver operating characteristic curve (ROC) area, precision, recall and F-measure (F1 score). It is worth noting that the classifiers showed an ROC area ranging from 0.70 to 0.78 in the test set. However, the random forest algorithm highlighted the lower sensitivity (recall) of the high perceived stress class compared to the other classifiers, making it a weaker model for the purposes of prediction.

4. Discussion and Conclusions

The present study measured the impact of the COVID-19 emergency on perceived levels of stress, taking into account sociodemographic variables and stable psychological traits. The results confirmed that participants perceived the COVID-19 crisis as a stressful experience; in the present sample, the level of perceived stress was higher than that of the general population in a non-emergency condition. Indeed, almost 20% of the sample scored above the results from the normative data on measures of perceived stress. These results are in line with the findings of recent studies on the psychological impact of COVID-19 [10,11,87] and the international literature on epidemic outbreaks [13]. The mean values of the single items of the PSS-10 suggest that, in addition to nervousness and stress, feelings of being unable to control one’s personal life accounted for the majority of participants’ perceived stress. This suggests that the unpredictability and uncontrollability of the pandemic may play a significant role in determining levels of perceived stress during the crisis. Moreover, it may reflect participants’ attitudes toward the significant lifestyle changes demanded of them due to the lockdown and other restrictive measures.
As regards sociodemographic variables, the results suggest that the female gender is associated with higher levels of stress. This is consistent with the literature indicating gender differences in the psychological response to COVID-19 [24,88] and other epidemics [89]; it is also in line with the normative data for the general population. Consistent with other studies (20,30), the present study found an association between higher incomes and lower levels of perceived stress. One explanation for this is that higher incomes might be related to less concern about the economic effects of self-isolation and/or with more comfortable housing solutions (e.g., larger living spaces, access to outdoor spaces (such as gardens), and access to leisure activities). Moreover, people with higher incomes may be more likely to perform work that can easily and fully transition to the online environment, thereby reducing some sources of stress.
In the present sample, older age was found to be associated with lower levels of stress. This finding might appear surprising, since it contradicts both the results of studies on the Chinese population [24,35] and the association between older age and higher COVID-19 mortality. However, the result is in line with recent Italian data [87]. Several studies have indicated age-related differences in coping and locus of control, with older adults presenting greater self-control and emotional self-regulation relative to younger adults [90,91,92]. Considering the current pandemic, older people may be more used to staying at home, so their daily routines might be less impacted by mandatory self-isolation measures. Data from previous investigations on age differences in stress responses to the SARS epidemic reflect inconsistencies [93,94], but sociopolitical and cultural aspects, such as differences in elder care services and policies, might account for these discrepancies.
The present study did not find education to be a significant predictor of the level of perceived stress. The large percentage of highly educated participants in the sample might partially explain this finding. However, prior research on this subject has generated mixed results—recent studies on the Chinese population [24,35] have found that education does not seem to affect mental health, while data from a Spanish sample [88] and from previous studies on psychological adjustment to SARS [95] have confirmed an association between a higher level of education and better mental health. The present study also found that living alone or with few family members was a protective factor against perceived stress. We might argue that this condition both conveys a sense of protection from contagion and offers continuity with pre-epidemic economic and social conditions. Moreover, when cohabiting with family members, concern for loved ones might contribute to increasing perceived stress.
As regards psychological variables, emotional stability was found to be an important protective factor. According to the five-factors model [63], people with high emotional stability remain calm in response to stressful situations, and view problems in proportion to their importance. As a result, they tend to worry less about problems than do people with low emotional stability [96]. Many studies have found that emotional stability is able to buffer stress responses to adverse events [97,98]. In the present study, conscientiousness and agreeableness were found to predict psychological distress. Generally speaking, individuals who score high on agreeableness tend to dislike conflict and be less suspicious of others; generally, they seek to pacify and mediate. In this sense, agreeable people might be more flexible and accepting when faced with unexpected and undesired situations, such as restrictions and changes to daily routines. Conscientious people are more likely to perceive lower levels of stress (see correlation analysis). According to the literature, they are more aware of their actions and tend to exhibit more goal-oriented behavior. In this sense, in times of self-isolation, conscientious people might have a greater tolerance for frustration and imposition relative to less conscientious people, who might engage in more impulsive behavior [96]. Moreover, previous research has indicated that conscientiousness may influence adaptive behavior, especially in health-related programs [99,100].
The present study found higher levels of dispositional self-control to predict lower levels of psychological distress in response to the COVID-19 emergency. Personal self-control skills may play a role in determining tolerance to restrictions to personal freedom during self-isolation. This result further suggests that self-regulatory processes may have a strong influence on responses to the outbreak. The results regarding dispositional self-control are consistent with those relating to the emotional stability trait of the Big Five model. In fact, these dimensions are often correlated [101], and this specific pattern may indicate the importance of personal skills, such as the ability to remain calm, and maintain emotional balance and a sense of acceptance. In this sense, practices that enhance emotional stability and acceptance, such as mindfulness, could be useful in reducing the stressful impact of the emergency [102,103]. The results concerning coping styles are also in line with this. Besides confirming the protective effect of functional coping styles and the adverse impact of dysfunctional coping styles, the results of the present study suggest that people who use a positive attitude as a coping strategy may be much less likely to experience psychological distress during the present emergency. Such persons may appraise the emergency as a unique opportunity, and feel less need for psychological support. In contrast, the present findings suggest that people who use avoidance strategies may be more likely to experience higher levels of stress during the emergency. These results are consistent with the findings of previous investigations into the relation between coping style and response to an epidemic [23,43,104,105].
The results relating to dispositional locus of control indicate that people with an internal locus of control may be less likely to feel stressed. Again, these results suggest that the more people are inclined to confidently rely on themselves, the better they will cope with uncertainty and change. Several studies have indicated an association between an internal locus of control and self-efficacy and emotional stability [106,107]. Furthermore, previous studies have found a relation between an internal locus of control and the positive appraisal of an emerging infectious disease outbreak [93]. It could be hypothesized that people with an internal locus of control interpret self-isolating as something that they determine and enact for themselves as a protective behavior, rather than something that is imposed on them; this might account for their lower levels of perceived stress.
Overall, the results of the present study identify some population subgroups that may be more vulnerable to experiencing stress during the COVID-19 emergency. Specifically, a set of seven psycho-social variables may identify a high percentage of people experiencing high stress during the COVID-19 pandemic, with sensitivity approaching 0.759 (ROC area of predictive models ranging between 0.70 and 0.78). According to this model, we may develop targeted preventive interventions. Furthermore, self-regulatory skills (including emotional stability, an internal locus of control, and self-control) were shown to be a protective factor, indicating the importance of raising awareness of these skills during the emergency and offering training and education to increase personal abilities in these areas (e.g., mindfulness programs).
The present research aimed at improving our understanding of the possible risk and protective factors for high perceived stress during the COVID-19 outbreak. It is worth noting that all data were collected from an Italian population. Therefore, the findings were inevitably influenced by specific contextual and socio-cultural aspects. Further investigations involving people of different ethnicities and residents of other countries would deepen our understanding of the generalizability of these results, and the effective influence of psychological traits. In this regard, the open access data reported in the Supplementary Materials may contribute to facilitating comparisons between ethnicities, countries, and specific traumatic events.
Some further limitations should be considered when interpreting the findings of this research. First, participants were recruited via an online link posted on social networks. While online recruitment guarantees large samples, it does not guarantee sample representativeness. For this reason, very vulnerable groups, such as homeless or low-income persons, may not be well represented in this study. Similarly, the average age of the sample was young, predominantly female, and largely well educated, as indicative of a sample that is more likely to participate in an online survey. Second, the use of self-report measures did not enable us to verify the reliability of the responses, or to ensure that participants correctly understood the questions. Future research should aim at overcoming these shortcomings.
Finally, future research should also investigate the interplay and mutual interrelationship between protective and risk factors, to improve the targeting and overall effectiveness of preventive programs and interventions. Indeed, the literature suggests that, during a pandemic, it is extremely important for people to sustain their use of psychological services, either online or in the context of social distancing [108,109]. This is particularly essential for those who are more vulnerable to experiencing high levels of stress, and it is important that we ensure that such persons can access timely and high-quality psychological services in order to prevent the development of chronic outcomes, including PTSD.

Supplementary Materials

The following are available online at https://www.mdpi.com/2077-0383/9/10/3350/s1, Table S1: Descriptive statistics, Table S2: Item by item analysis of the PSS-10, Table S3: List of predictors, Table S4: Details on ML classifiers parameters.

Author Contributions

Conceptualization, L.F. and M.M.; methodology, M.M., L.F. and V.F.; formal analysis, M.M., V.F. and E.C.; investigation, M.M., L.F. and V.F.; data curation, M.M. and V.F.; writing—original draft preparation, M.M., L.F. and V.F.; writing—review and editing, M.M., V.F., L.F., C.M., E.C., B.S. and P.R.; supervision, C.M. and P.R.; funding acquisition, P.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Rothan, H.A.; Byrareddy, S.N. The epidemiology and pathogenesis of coronavirus disease (COVID-19) outbreak. J. Autoimmun. 2020, 109, 102433. [Google Scholar] [CrossRef] [PubMed]
  2. Wu, Z.; McGoogan, J.M. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China. JAMA 2020, 323, 1239. [Google Scholar] [CrossRef]
  3. Kluge, H.H.P.; WHO Regional Office for Europe. Supporting Older People During the COVID-19 Pandemic is Everyone’s Business. Available online: https://www.euro.who.int/en/health-topics/health-emergencies/coronavirus-covid-19/news/news/2020/4/supporting-older-people-during-the-covid-19-pandemic-is-everyones-business (accessed on 11 July 2020).
  4. Wang, Y.; Wang, Y.; Chen, Y.; Qin, Q. Unique epidemiological and clinical features of the emerging 2019 novel coronavirus pneumonia (COVID-19) implicate special control measures. J. Med. Virol. 2020, 92, 568–576. [Google Scholar] [CrossRef]
  5. Cunningham, A.C.; Goh, H.P.; Koh, D. Treatment of COVID-19: Old tricks for new challenges. Crit. Care 2020, 24, 1–2. [Google Scholar] [CrossRef]
  6. Wu, D.; Wu, T.; Liu, Q.; Yang, Z. The SARS-CoV-2 outbreak: What we know. Int. J. Infect. Dis. 2020, 94, 44–48. [Google Scholar] [CrossRef] [PubMed]
  7. WHO Coronavirus Disease (COVID-19) Dashboard. Available online: https://covid19.who.int (accessed on 11 July 2020).
  8. World Health Organization (WHO). WHO Director-General’s Opening Remarks at the Mission Briefing on COVID-19. Available online: https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-mission-briefing-on-covid-19---26-february-2020 (accessed on 11 March 2020).
  9. Ren, X. Pandemic and lockdown: A territorial approach to COVID-19 in China, Italy and the United States. Eurasian Geogr. Econ. 2020, 1–12. [Google Scholar] [CrossRef]
  10. Wang, C.; Pan, R.; Wan, X.; Tan, Y.; Xu, L.; Ho, C.S.H.; Ho, R.C.M. Immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health 2020, 17, 1729. [Google Scholar] [CrossRef] [PubMed]
  11. Cao, W.; Fang, Z.; Hou, G.; Han, M.; Xu, X.; Dong, J.; Zheng, J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020, 287, 112934. [Google Scholar] [CrossRef]
  12. Mak, I.W.C.; Chung, Y.M.G.; Pan, P.C.; Yiu, M.G.C.; Chan, V.L. Long-term psychiatric morbidities among SARS survivors. Gen. Hosp. Psychiatry 2009, 31, 318–326. [Google Scholar] [CrossRef]
  13. Huremović, D. Psychiatry of Pandemics: A Mental Health Response to Infection Outbreak; Springer International Publishing: Cham, Switzerland, 2019. [Google Scholar]
  14. Li, Z.; Ge, J.; Yang, M.; Feng, J.; Qiao, M.; Jiang, R.; Bi, J.; Zhan, G.; Xu, X.; Wang, L.; et al. Vicarious traumatization in the general public, members, and non-members of medical teams aiding in COVID-19 control. Brain Behav. Immun. 2020, 88, 916–919. [Google Scholar] [CrossRef]
  15. Brooks, S.K.; Webster, R.K.; Smith, E.L.; Woodland, L.; Wessely, S.; Greenberg, N.; Rubin, G.J. The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. Lancet 2020, 395, 912–920. [Google Scholar] [CrossRef]
  16. Xiang, Y.-T.; Yang, Y.; Li, W.; Zhang, L.; Zhang, Q.; Cheung, T.; Ng, C.H. Timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. Lancet Psychiatry 2020, 7, 228–229. [Google Scholar] [CrossRef]
  17. Mazza, C.; Ricci, E.; Marchetti, D.; Fontanesi, L.; Di Giandomenico, S.; Verrocchio, M.C.; Roma, P. How personality relates to distress in parents during the COVID-19 lockdown: The mediating role of child’s emotional and behavioral difficulties and the moderating effect of living with other people. Int. J. Environ. Res. Public Health 2020, 17, 6236. [Google Scholar] [CrossRef]
  18. Fontanesi, L.; Marchetti, D.; Mazza, C.; Di Giandomenico, S.; Roma, P.; Verrocchio, M.C. The effect of the COVID-19 lockdown on parents: A call to adopt urgent measures. Psychol. Trauma Theory Res. Pract. Policy 2020, 12, S79–S81. [Google Scholar] [CrossRef] [PubMed]
  19. Roma, P.; Monaro, M.; Muzi, L.; Colasanti, M.; Ricci, E.; Biondi, S.; Napoli, C.; Ferracuti, S.; Mazza, C. How to Improve Compliance with Protective Health Measures during the COVID-19 Outbreak: Testing a Moderated Mediation Model and Machine Learning Algorithms. Int. J. Environ. Res. Public Health 2020, 17, 7252. [Google Scholar] [CrossRef]
  20. Cheng, C.; Cheung, M.W.L. Psychological responses to outbreak of severe acute respiratory syndrome: A prospective, multiple time-point study. J. Pers. 2005, 73, 261–285. [Google Scholar] [CrossRef]
  21. Asmundson, G.J.; Taylor, S. How health anxiety influences responses to viral outbreaks like COVID-19: What all decision-makers, health authorities, and health care professionals need to know. J. Anxiety Disord. 2020, 71, 102211. [Google Scholar] [CrossRef] [PubMed]
  22. Yu, H.-Y.; Ho, S.C.; So, K.F.E.; Lo, Y.L. The psychological burden experienced by Hong Kong midlife women during the SARS epidemic. Stress Health 2005, 21, 177–184. [Google Scholar] [CrossRef]
  23. Sim, K.; Chan, Y.H.; Chong, P.N.; Chua, H.C.; Soon, S.W. Psychosocial and coping responses within the community health care setting towards a national outbreak of an infectious disease. J. Psychosom. Res. 2010, 68, 195–202. [Google Scholar] [CrossRef]
  24. Liu, N.; Zhang, F.; Wei, C.; Jia, Y.; Shang, Z.; Sun, L.; Wu, L.; Sun, Z.; Zhou, Y.; Wang, Y.; et al. Prevalence and predictors of PTSS during COVID-19 outbreak in China hardest-hit areas: Gender differences matter. Psychiatry Res. 2020, 287, 112921. [Google Scholar] [CrossRef]
  25. Xu, J.; Zheng, Y.; Wang, M.; Zhao, J.; Zhan, Q.; Fu, M.; Wang, Q.; Xiao, J.; Cheng, Y. Predictors of symptoms of posttraumatic stress in Chinese university students during the 2009 H1N1 influenza pandemic. Med. Sci. Monit. 2011, 17, PH60–PH64. [Google Scholar] [CrossRef] [PubMed]
  26. Di Crosta, A.; Palumbo, R.; Marchetti, D.; Ceccato, I.; La Malva, P.; Maiella, R.; Cipi, M.; Roma, P.; Mammarella, N.; Verrocchio, M.C.; et al. Individual differences, economic stability, and fear of contagion as risk factors for PTSD symptoms in the COVID-19 emergency. Front. Psychol. 2020, 11. [Google Scholar] [CrossRef] [PubMed]
  27. McEwen, B.S. Neurobiological and systemic effects of chronic stress. Chronic Stress 2017, 1, 247054701769232. [Google Scholar] [CrossRef]
  28. Toussaint, L.; Shields, G.S.; Dorn, G.; Slavich, G.M. Effects of lifetime stress exposure on mental and physical health in young adulthood: How stress degrades and forgiveness protects health. J. Health Psychol. 2014, 21, 1004–1014. [Google Scholar] [CrossRef] [PubMed]
  29. Yang, L.; Zhao, Y.; Wang, Y.; Liu, L.; Zhang, X.; Li, B.; Cui, R. The effects of psychological stress on depression. Curr. Neuropharmacol. 2015, 13, 494–504. [Google Scholar] [CrossRef]
  30. Mayer, S.E.; Lopez-Duran, N.L.; Sen, S.; Abelson, J.L. Chronic stress, hair cortisol and depression: A prospective and longitudinal study of medical internship. Psychoneuroendocrinology 2018, 92, 57–65. [Google Scholar] [CrossRef]
  31. Schultebraucks, K.; Rombold-Bruehl, F.; Wingenfeld, K.; Hellmann-Regen, J.; Otte, C.; Roepke, S. Heightened biological stress response during exposure to a trauma film predicts an increase in intrusive memories. J. Abnorm. Psychol. 2019, 128, 645–657. [Google Scholar] [CrossRef]
  32. Colodro-Conde, L.; Couvy-Duchesne, B.; Zhu, G.; Coventry, W.L.; Byrne, E.M.; Gordon, S.; Wright, M.J.; Montgomery, G.W.; Madden, P.A.F.; Ripke, S.; et al. A direct test of the diathesis–stress model for depression. Mol. Psychiatry 2017, 23, 1590–1596. [Google Scholar] [CrossRef]
  33. Lazarus, R.S. Psychological stress and coping in adaptation and illness. Int. J. Psychiatry Med. 1974, 5, 321–333. [Google Scholar] [CrossRef]
  34. Zhang, J.; Wu, W.; Zhao, X.; Zhang, W. Recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in China: A model of West China Hospital. Precis. Clin. Med. 2020, 3, 3–8. [Google Scholar] [CrossRef]
  35. Qiu, J.; Shen, B.; Zhao, M.; Wang, Z.; Xie, B.; Xu, Y. A nationwide survey of psychological distress among Chinese people in the COVID-19 epidemic: Implications and policy recommendations. Gen. Psychiatry 2020, 33, e100213. [Google Scholar] [CrossRef] [PubMed]
  36. Li, S.; Wang, Y.; Xue, J.; Zhao, N.; Zhu, T. The impact of COVID-19 epidemic declaration on psychological consequences: A study on active weibo users. Int. J. Environ. Res. Public Health 2020, 17, 2032. [Google Scholar] [CrossRef] [PubMed]
  37. Roberts, B.W.; Kuncel, N.R.; Shiner, R.; Caspi, A.; Goldberg, L.R. The Power of Personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspect. Psychol. Sci. 2007, 2, 313–345. [Google Scholar] [CrossRef]
  38. Folkman, S.; Lazarus, R.S. An analysis of coping in a middle-aged community sample. J. Health Soc. Behav. 1980, 21, 219. [Google Scholar] [CrossRef]
  39. Folkman, S.; Lazarus, R.S. Coping as a mediator of emotion. J. Pers. Soc. Psychol. 1988, 54, 466–475. [Google Scholar] [CrossRef] [PubMed]
  40. Connor-Smith, J.K.; Compas, B.E. Coping as a moderator of relations between reactivity to interpersonal stress, health status, and internalizing problems. Cogn. Ther. Res. 2004, 28, 347–368. [Google Scholar] [CrossRef]
  41. Bandura, A. Human agency in social cognitive theory. Am. Psychol. 1989, 44, 1175. [Google Scholar] [CrossRef] [PubMed]
  42. Bandura, A. Self-Efficacy: The Exercise of Control; Freeman: New York, NY, USA, 1997. [Google Scholar]
  43. Taha, S.A.; Matheson, K.; Anisman, H. The 2009 H1N1 influenza pandemic: The role of threat, coping, and media trust on vaccination intentions in Canada. J. Health Commun. 2013, 18, 278–290. [Google Scholar] [CrossRef]
  44. Tangney, J.P.; Baumeister, R.F.; Boone, A.L. High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. J. Pers. 2004, 72, 271–324. [Google Scholar] [CrossRef]
  45. Smith, N.B.; Sippel, L.M.; Presseau, C.; Rozek, D.; Mota, N.; Gordon, C.; Horvath, M.; Harpaz-Rotem, I. Locus of control in US combat veterans: Unique associations with posttraumatic stress disorder 5-factor model symptom clusters. Psychiatry Res. 2018, 268, 152–156. [Google Scholar] [CrossRef]
  46. Rizza, F.; Gison, A.; Bonassi, S.; Dall’Armi, V.; Tonto, F.; Giaquinto, S. ‘Locus of control’, health-related quality of life, emotional distress and disability in Parkinson’s disease. J. Health Psychol. 2015, 22, 844–852. [Google Scholar] [CrossRef] [PubMed]
  47. Rotter, J.B. Generalized expectancies for internal versus external control of reinforcement. Psychol. Monogr. Gen. Appl. 1966, 80, 1–28. [Google Scholar] [CrossRef]
  48. McCrae, R.R.; Costa, P.T. Personality in Adulthood; The Guilford Press: New York, NY, USA, 1990. [Google Scholar]
  49. Allen, M.S.; Walter, E.E.; McDermott, M.S. Personality and sedentary behavior: A systematic review and meta-analysis. Health Psychol. 2017, 36, 255–263. [Google Scholar] [CrossRef] [PubMed]
  50. Allen, M.S.; Walter, E.E. Linking big five personality traits to sexuality and sexual health: A meta-analytic review. Psychol. Bull. 2018, 144, 1081–1110. [Google Scholar] [CrossRef]
  51. Wilson, K.E.; Dishman, R.K. Personality and physical activity: A systematic review and meta-analysis. Pers. Individ. Differ. 2015, 72, 230–242. [Google Scholar] [CrossRef]
  52. Malouff, J.M.; Thorsteinsson, E.B.; Rooke, S.E.; Schutte, N.S. Alcohol involvement and the five-factor model of personality: A meta-analysis. J. Drug Educ. 2007, 37, 277–294. [Google Scholar] [CrossRef]
  53. Mondo, M.; Sechi, C.; Cabras, C. Psychometric evaluation of three versions of the Italian Perceived Stress Scale. Curr. Psychol. 2019, 1–9. [Google Scholar] [CrossRef]
  54. Maroufizadeh, S.; Foroudifard, F.; Navid, B.; Ezabadi, Z.; Sobati, B.; Omani-Samani, R. The Perceived Stress Scale (PSS-10) in women experiencing infertility: A reliability and validity study. Middle East Fertil. Soc. J. 2018, 23, 456–459. [Google Scholar] [CrossRef]
  55. Cohen, S.; Kamarck, T.; Mermelstein, R. A global measure of perceived stress. J. Health Soc. Behav. 1983, 24, 385. [Google Scholar] [CrossRef]
  56. Cohen, S.; Hoberman, H.M. Positive events and social supports as buffers of life change Stress1. J. Appl. Soc. Psychol. 1983, 13, 99–125. [Google Scholar] [CrossRef]
  57. Caricati, L.; Foà, C.; Fruggeri, L.; Tonarelli, A. COPE-NVI-25: Validazione italiana della versione ridotta della Coping orientation to the problems experienced (COPE-NVI). Psicol. Salut. 2015, 2, 123–140. [Google Scholar] [CrossRef]
  58. Carver, C.S.; Scheier, M.F.; Weintraub, J.K. Assessing coping strategies: A theoretically based approach. J. Pers. Soc. Psychol. 1989, 56, 267–283. [Google Scholar] [CrossRef]
  59. Sica, C.; Magni, C.; Ghisi, M.; Altoè, G.; Sighinolfi, C.; Chiri, L.R.; Franceschini, S. Coping orientation to problems experienced-nuova versione italiana (COPE-NVI): Uno strumento per la misura degli stili di coping. Psicoter. Cogn. Comport. 2008, 14, 27–53. [Google Scholar]
  60. Meazzini, P. Vivere Con il Sole in Tasca: Cos’ è la Psicologia Positiva E Cosa Può Fare per Noi; FrancoAngeli: Milan, Italy, 2017. [Google Scholar]
  61. Nigro, G. Contributo alla taratura italiana della scala di controllo interno-esterno (I-E) di Rotter. Boll. Psicol. Appl. 1983, 168, 29–41. [Google Scholar]
  62. Guido, G.; Peluso, A.M.; Capestro, M.; Miglietta, M. An Italian version of the 10-item Big Five Inventory: An application to hedonic and utilitarian shopping values. Pers. Individ. Differ. 2015, 76, 135–140. [Google Scholar] [CrossRef]
  63. McCrae, R.R.; Costa, P. The stability of personality: Observations and evaluations. Curr. Dir. Psychol. Sci. 1994, 3, 173–175. [Google Scholar] [CrossRef]
  64. Rammstedt, B.; John, O.P. Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. J. Res. Pers. 2007, 41, 203–212. [Google Scholar] [CrossRef]
  65. Faul, F.; Erdfelder, E.; Lang, A.-G.; Buchner, A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 2007, 39, 175–191. [Google Scholar] [CrossRef]
  66. JASP, Version0.14. Available online: https://jasp-stats.org/2020/10/14/introducing-jasp-0-14/ (accessed on 14 October 2020).
  67. Goss-Sampson, M.A. Statistical Analysis in JASP: A Guide for Students. Available online: https://static.jasp-stats.org/Statistical%20Analysis%20in%20JASP%20-%20A%20Students%20Guide%20v2.pdf (accessed on 1 October 2018).
  68. Friedrich, R.J. In defense of multiplicative terms in multiple regression equations. Am. J. Polit. Sci. 1982, 26, 797. [Google Scholar] [CrossRef]
  69. Yarkoni, T.; Westfall, J. Choosing prediction over explanation in psychology: Lessons from machine learning. Perspect. Psychol. Sci. 2017, 12, 1100–1122. [Google Scholar] [CrossRef]
  70. Mazza, C.; Orrù, G.; Burla, F.; Monaro, M.; Ferracuti, S.; Colasanti, M.; Roma, P. Indicators to distinguish symptom accentuators from symptom producers in individuals with a diagnosed adjustment disorder: A pilot study on inconsistency subtypes using SIMS and MMPI-2-RF. PLoS ONE 2019, 14, e0227113. [Google Scholar] [CrossRef] [PubMed]
  71. Mazza, C.; Monaro, M.; Orrù, G.; Burla, F.; Colasanti, M.; Ferracuti, S.; Roma, P. Introducing machine learning to detect personality faking-good in a male sample: A new model based on minnesota multiphasic personality inventory-2 restructured form scales and reaction times. Front. Psychiatry 2019, 10. [Google Scholar] [CrossRef] [PubMed]
  72. Orrù, G.; Mazza, C.; Monaro, M.; Ferracuti, S.; Sartori, G.; Roma, P. The development of a short version of the SIMS using machine learning to detect feigning in forensic assessment. Psychol. Injury Law 2020. [Google Scholar] [CrossRef]
  73. Orrù, G.; Monaro, M.; Conversano, C.; Gemignani, A.; Sartori, G. machine learning in psychometrics and psychological research. Front. Psychol. 2020, 10. [Google Scholar] [CrossRef] [PubMed]
  74. Dwyer, D.B.; Efallkai, P.; Koutsouleris, N. Machine learning approaches for clinical psychology and psychiatry. Annu. Rev. Clin. Psychol. 2018, 14, 91–118. [Google Scholar] [CrossRef]
  75. Nelles, O. Nonlinear system identification. In From Classical Approaches to Neural Networks and Fuzzy Models; Springer: Berlin/Heidelberg, Germany, 2001. [Google Scholar]
  76. Cumming, G. Replication and p intervals: P values predict the future only vaguely, but confidence intervals do much better. Perspect. Psychol. Sci. 2008, 3, 286–300. [Google Scholar] [CrossRef] [PubMed]
  77. Dwork, C.; Feldman, V.; Hardt, M.; Pitassi, T.; Reingold, O.; Roth, A. The reusable holdout: Preserving validity in adaptive data analysis. Science 2015, 349, 636–638. [Google Scholar] [CrossRef] [PubMed]
  78. Hall, M.A. Correlation-Based Feature Selection for Machine Learning. Ph.D. Thesis, University of Waikato, Hamilton, New Zealand, 1999. [Google Scholar]
  79. Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I.H. The WEKA data mining software. ACM SIGKDD Explor. Newsl. 2009, 11, 10–18. [Google Scholar] [CrossRef]
  80. Japkowicz, N. (Ed.) The class imbalance problem: Significance and strategies. In Proceedings of the International Conference on Artificial Intelligence (ICAI), Las Vegas, NV, USA, 28 June–1 July 2000; pp. 111–117. [Google Scholar]
  81. Japkowicz, N.; Stephen, S. The class imbalance problem: A systematic study1. Intell. Data Anal. 2002, 6, 429–449. [Google Scholar] [CrossRef]
  82. Le Cessie, S.; Van Houwelingen, J.C. Ridge estimators in logistic regression. J. R. Stat. Soc. Ser. C Appl. Stat. 1992, 41, 191. [Google Scholar] [CrossRef]
  83. Keerthi, S.S.; Shevade, S.K.; Bhattacharyya, C.; Murthy, K.R.K. Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput. 2001, 13, 637–649. [Google Scholar] [CrossRef]
  84. John, G.H.; Langley, P. Estimating continuous distributions in Bayesian classifiers. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, 18–20 August 1995; pp. 338–345. [Google Scholar]
  85. Breiman, L. Random forest. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
  86. Kohavi, R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Mateo, CA, USA, August 1995; pp. 1137–1143. [Google Scholar]
  87. Mazza, C.; Ricci, E.; Biondi, S.; Colasanti, M.; Ferracuti, S.; Napoli, C.; Roma, P. A nationwide survey of psychological distress among Italian people during the COVID-19 pandemic: Immediate psychological responses and associated factors. Int. J. Environ. Res. Public Health 2020, 17, 3165. [Google Scholar] [CrossRef] [PubMed]
  88. Losada-Baltar, A.; Jiménez-Gonzalo, L.; Gallego-Alberto, L.; Pedroso-Chaparro, M.D.S.; Fernandes-Pires, J.; Márquez-González, M. “We Are Staying at Home.” Association of self-perceptions of aging, personal and family resources, and loneliness with psychological distress during the lock-down period of COVID-19. J. Gerontol. Ser. B 2020. [Google Scholar] [CrossRef]
  89. Chew, Q.H.; Wei, K.C.; Vasoo, S.; Chua, H.C.; Sim, K. Narrative synthesis of psychological and coping responses towards emerging infectious disease outbreaks in the general population: Practical considerations for the COVID-19 pandemic. Singap. Med. J. 2020, 61, 350–356. [Google Scholar] [CrossRef]
  90. Labouvie-Vief, G. Dynamic integration: Affect, cognition, and the self in adulthood. Curr. Dir. Psychol. Sci. 2003, 12, 201–206. [Google Scholar] [CrossRef]
  91. Cummings, E.M.; Greene, A.L.; Karraker, K.H. Life-Span Developmental Psychology: Perspectives on Stress and Coping; Lawrence Erlbaum Associates, Inc.: Hillsdale, NJ, USA, 1991. [Google Scholar]
  92. Carstensen, L.L. Social and emotional patterns in adulthood: Support for socioemotional selectivity theory. Psychol. Aging 1992, 7, 331. [Google Scholar] [CrossRef]
  93. Peng, E.Y.-C.; Lee, M.-B.; Tsai, S.-T.; Yang, C.-C.; Morisky, D.E.; Tsai, L.-T.; Weng, Y.-L.; Lyu, S.-Y. Population-based post-crisis psychological distress: An example from the SARS outbreak in Taiwan. J. Formos. Med Assoc. 2010, 109, 524–532. [Google Scholar] [CrossRef]
  94. Main, A.; Zhou, Q.; Ma, Y.; Luecken, L.J.; Liu, X. Relations of SARS-related stressors and coping to Chinese college students’ psychological adjustment during the 2003 Beijing SARS epidemic. J. Couns. Psychol. 2011, 58, 410–423. [Google Scholar] [CrossRef]
  95. Mak, W.W.; Law, R.W.; Woo, J.; Cheung, F.M.; Lee, D. Social support and psychological adjustment to SARS: The mediating role of self-care self-efficacy. Psychol. Health 2009, 24, 161–174. [Google Scholar] [CrossRef]
  96. Caprara, G.V.; Barbaranelli, C.; Borgogni, L.; Perugini, M. The “big five questionnaire”: A new questionnaire to assess the five factor model. Pers. Individ. Differ. 1993, 15, 281–288. [Google Scholar] [CrossRef]
  97. Rodríguez-Ramos, Á.; Moriana, J.A.; García-Torres, F.; Ruiz-Rubio, M. Emotional stability is associated with the MAOA promoter uVNTR polymorphism in women. Brain Behav. 2019, 9, e01376. [Google Scholar] [CrossRef]
  98. Cohrdes, C.; Mauz, E. Self-Efficacy and emotional stability buffer negative effects of adverse childhood experiences on young adult health-related quality of life. J. Adolesc. Health 2020, 67, 93–100. [Google Scholar] [CrossRef] [PubMed]
  99. Sanatkar, S.; Baldwin, P.; Clarke, J.; Fletcher, S.; Gunn, J.; Wilhelm, K.; Campbell, L.; Zwar, N.; Harris, M.; Lapsley, H.; et al. The influence of personality on trajectories of distress, health and functioning in mild-to-moderately depressed adults with type 2 diabetes. Psychol. Health Med. 2019, 25, 296–308. [Google Scholar] [CrossRef]
  100. Fisher, L.; Hessler, D.; Masharani, U.; Strycker, L. Impact of baseline patient characteristics on interventions to reduce diabetes distress: The role of personal conscientiousness and diabetes self-efficacy. Diabet. Med. 2014, 31, 739–746. [Google Scholar] [CrossRef] [PubMed]
  101. Digman, J.M. Higher-order factors of the Big Five. J. Pers. Soc. Psychol. 1997, 73, 1246. [Google Scholar] [CrossRef]
  102. Alexander, M. Let’s conquer COVID-19 and sustain our abilities. Spinal Cord Ser. Cases 2020, 6, 19. [Google Scholar] [CrossRef]
  103. Pascoe, M.C.; Thompson, D.R.; Jenkins, Z.M.; Ski, C.F. Mindfulness mediates the physiological markers of stress: Systematic review and meta-analysis. J. Psychiatr. Res. 2017, 95, 156–178. [Google Scholar] [CrossRef] [PubMed]
  104. Phua, D.H.; Tang, H.K.; Tham, K.Y. Coping responses of emergency physicians and nurses to the 2003 severe acute respiratory syndrome outbreak. Acad. Emerg. Med. 2005, 12, 322–328. [Google Scholar] [CrossRef]
  105. Teasdale, E.; Yardley, L.; Schlotz, W.; Michie, S. The importance of coping appraisal in behavioural responses to pandemic flu. Br. J. Health Psychol. 2011, 17, 44–59. [Google Scholar] [CrossRef]
  106. Judge, T.A.; Bono, J.E. Relationship of core self-evaluations traits—self-esteem, generalized self-efficacy, locus of control, and emotional stability—with job satisfaction and job performance: A meta-analysis. J. Appl. Psychol. 2001, 86, 80–92. [Google Scholar] [CrossRef] [PubMed]
  107. Ajzen, I. Perceived behavioral control, self-efficacy, locus of control, and the theory of planned behavior 1. J. Appl. Soc. Psychol. 2002, 32, 665–683. [Google Scholar] [CrossRef]
  108. Duan, L.; Zhu, G. Psychological interventions for people affected by the COVID-19 epidemic. Lancet Psychiatry 2020, 7, 300–302. [Google Scholar] [CrossRef]
  109. Zhou, X.; Snoswell, C.L.; Harding, L.E.; Bambling, M.; Edirippulige, S.; Bai, X.; Smith, A.C. The role of telehealth in reducing the mental health burden from COVID-19. Telemed. e-Health 2020, 26, 377–379. [Google Scholar] [CrossRef] [PubMed]
Table 1. Multiple Linear Regression Model Predicting the Level of Perceived Stress Based on Sociodemographic Variables.
Table 1. Multiple Linear Regression Model Predicting the Level of Perceived Stress Based on Sociodemographic Variables.
95% CI
ΔR2Unstandardized
Coefficients (B)
S.E.tpLower BoundUpper Bound
(Intercept) 25.3420.98725.6654.845e−12623.40527.278
Age0.067−0.1210.012−10.5063.540e−25−0.143−0.098
Gender (male)0.024−2.4060.345−6.9684.321e−12−3.084−1.729
Education 0.004−0.0810.044−1.8420.066−0.1680.005
Household members0.0030.4530.1293.5004.748e−40.1990.706
Monthly income0.005−0.4630.134−3.4655.403e−4−0.725−0.201
Note: Root Mean Square Error (RMSE) = 6.56. Analysis of Variance (ANOVA) F(5,2029) = 46.65, p < 0.001.
Table 2. Multiple Linear Regression Model Predicting the Level of Perceived Stress Based on Sociodemographic Variables and Stable Psychological Traits.
Table 2. Multiple Linear Regression Model Predicting the Level of Perceived Stress Based on Sociodemographic Variables and Stable Psychological Traits.
95% CI
ΔR2Unstandardized
Coefficients (B)
S.E.tpLower BoundUpper Bound
(Intercept)0.00036.2381.54023.531<0.00133.21839.258
BFI-10 emotional stability0.221−0.9590.067−14.346<0.001−1.091−0.828
COPE positive 0.047−2.3570.211−11.195<0.001−2.770−1.944
Age0.033−0.0700.010−6.814<0.001−0.090−0.050
BSCS total score0.018−0.1270.019−6.646<0.001−0.165−0.090
Gender (male)0.016−1.8830.302−6.228<0.001−2.476−1.290
COPE avoidance0.0071.4110.3284.307<0.0010.7682.053
Household members0.0030.4060.1083.751<0.0010.1940.619
BFI-10 conscientiousness0.0030.2760.0982.8090.0050.0830.469
COPE support0.0020.5190.1683.0870.0020.1890.849
Monthly income0.002−0.2830.111−2.5430.011−0.502−0.065
Internal LOC0.002−0.1450.060−2.4310.015−0.262−0.028
Note: RMSE = 5.57. ANOVA F(12,2022) = 101.49, p < 0.001. BFI-10 = 10-item Big Five Inventory; COPE = Coping Orientations to the Problems Experienced; BSCS = Brief Self-Control Scale; LOC = Locus of Control.
Table 3. Metrics of the ML Models Tested on the 407 New Participants (Test Set).
Table 3. Metrics of the ML Models Tested on the 407 New Participants (Test Set).
AlgorithmROC AreaClassPrecisionRecallF-Measure
Logistic0.782High perceived stress0.3490.7590.478
Low perceived stress0.9190.6590.767
SVM0.697High perceived stress0.3370.7470.465
Low perceived stress0.9140.6460.757
Naïve Bayes0.776High perceived stress0.3610.7090.479
Low perceived stress0.9090.6980.790
Random forest0.776High perceived stress0.4380.5320.480
Low perceived stress0.8810.8350.858
Note: ROC = receiver operating characteristic curve; SVM = support vector machine.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop