Next Article in Journal
The Effects of Leisure Activities on Self-Efficacy and Social Adjustment: A Study of Immigrants in South Korea
Previous Article in Journal
Impact of the COVID-19 Pandemic on the Mental Health of Nurses and Auxiliary Nursing Care Technicians—A Voluntary Online Survey
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Three Versions of the Perceived Stress Scale: Psychometric Evaluation in a Nationally Representative Sample of Chinese Adults during the COVID-19 Pandemic

1
Shanghai Key Laboratory of Mental Health and Psychological Crisis Intervention, Affiliated Mental Health Center (ECNU), School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
2
Shanghai Changning Mental Health Center, Shanghai 200335, China
3
Mental Health Center, Hainan Medical University, Haikou 571199, China
4
Mental Health Center, Wuhan Polytechnic University, Wuhan 430074, China
*
Author to whom correspondence should be addressed.
These two authors contributed equally to the study. They should be regarded as joint first authors.
Int. J. Environ. Res. Public Health 2021, 18(16), 8312; https://doi.org/10.3390/ijerph18168312
Submission received: 11 June 2021 / Revised: 3 August 2021 / Accepted: 4 August 2021 / Published: 5 August 2021

Abstract

:
(1) Background: The COVID-19 outbreak has created pressure in people’s daily lives, further threatening public health. Thus, it is important to assess people’s perception of stress during COVID-19 for both research and practical purposes. The Perceived Stress Scale (PSS) is one of the most widely used instruments to measure perceived stress; however, previous validation studies focused on specific populations, possibly limiting the generalization of results. (2) Methods: This study tested the psychometric properties of three versions of the Chinese Perceived Stress Scale (CPSS-14, CPSS-10, and CPSS-4) in the Chinese general population during the COVID-19 pandemic. A commercial online survey was employed to construct a nationally representative sample of 1133 adults in Mainland China (548 males and 585 females) during a one-week period. (3) Results: The two-factor (positivity and negativity) solution for the three versions of the CPSS showed a good fit with the data. The CPSS-14 and CPSS-10 had very good reliability and the CPSS-4 showed acceptable reliability, supporting the concurrent validity of the CPSS. (4) Conclusions: All three versions of the CPSS appear to be appropriate for use in research with samples of adults in the Chinese general population under the COVID-19 crisis. The CPSS-10 and CPSS-14 both have strong psychometric properties, but the CPSS-10 would have more utility because it is shorter than the CPSS-14. However, the CPSS-4 is an acceptable alternative when administration time is limited.

1. Introduction

People may feel stressed when they perceive a situation to be threatening but lack effective resources to cope with the threat [1]. This explanation fits with the high stress reported by Chinese in the context of the current COVID-19 crisis [2]. A similar situation was also found in previous pandemics. For example, the general public in Hong Kong demonstrated increased stress in their family and work settings under the severe acute respiratory syndrome (SARS) crisis [3]. Pandemics bring unprecedented social fear about a virus that is highly infectious and potentially deadly, together with quarantine, potential economic losses, and conflicting and inadequate information that the public obtain on the internet [2,4,5]. All of these may make people experience stress. Stress during the COVID-19 crisis may further decrease people’s well-being and increase other health-related problems such as anxiety and depression [6,7]. The current crisis in China allows us to test the psychometric properties of a measure of perceived stress in a high stress context.
The Perceived Stress Scale (PSS) is one of the most widely used stress perception assessment instruments in the world [8]. The scale was originally developed in 1983 by Cohen et al. [9] and was designed to assess the degree of stress people felt in unpredictable, out-of-control, and overloaded situations. The original version of the PPS had 14 items (PSS-14) with seven negative items (e.g., “Unable to control the important things in your life?”) and seven positive items (e.g., “Confident about your ability to handle your personal problems?”). Based on factor analysis, the researchers removed the four items with the lowest factor loadings on the PSS-14 to create a shortened 10-item version (PPS-10) [10]. An even briefer four-item version (PSS-4) was developed for ease of use when there are time constraints on data collection (e.g., in telephone interviews) [10].
To date, the PSS has been translated into more than 20 languages, and its psychometric properties have also been verified in multiple countries. Studies have shown that both the PSS-14 and PSS-10 have satisfactory internal consistency, with alphas ranging from 0.74 [11] to 0.91 [12]; reports of the internal consistency of the PSS-4 have ranged from marginal (0.60) [10] to good (0.82) [12]. The two-factor structure for the PPS-14 and PPS-10 has also been widely validated across cultures [8,10] including China [13,14].
There are two gaps in this literature that are addressed in the current study. First, there have been mixed findings concerning the factor structure of the PSS-4. Most researchers (including the original authors of the PSS) found that a one-factor solution fit the data best [10,15,16]. However, other studies failed to replicate these findings and found that the proposed single-factor of the PSS-4 model indicated a poor fit [17], found that a two-factor solution fit the model best [18], or found that both the one- and two-factor solutions were acceptable [19]. These inconsistent results warrant further investigation. Understanding the factor structure of the PSS-4 will inform future researchers about whether the measure should be regarded as one scale to assess stress level as a whole or can also be used as two subscales similar to the longer forms (PSS-10 and PSS-14).
Second, studies that have compared the reliability, validity, and utility of the three versions of the PSS have been in specific populations, such as college students [16], workers [18], or cardiac patients [20]. The interest of this study was to compare the three versions in a wider population, namely a nationally representative sample in Mainland China, using the Chinese Perceived Stress Scale (CPSS). The 14-item and 10-item versions of the CPSS have good psychometric properties and are widely used in Mainland China [13,14,21,22], but we could not locate any psychometric studies on the CPSS-4. No studies to date have tested whether all three CPSS versions (especially the CPSS-4) are appropriate for use in general samples in Mainland China.
To address these two gaps in the literature, the aim of this study was to assess the construct validity, reliability, and concurrent validity of the three versions of the CPSS, and to compare their appropriateness in a representative sample of adults in Mainland China. Given that the entire world is currently under the burden of COVID-19, more studies are needed to investigate pandemic-related stress and its negative effects. The authors of this study hope to identify a reliable, valid, and feasible version of the CPSS for both research and practical purposes (e.g., for use in telephone interviews and in combination with other measures).

2. Materials and Methods

2.1. Participants and Procedure

The commercial online survey research firm Wenjuanxing was employed to construct a nationally representative sample from 25 February to 2 March 2020, about one month after the massive outbreak of COVID–19 in China. Wenjuanxing has a large database with information from over 2.6 million Chinese registered users across China. With the permission of customers, Wenjuanxing identified potential participants to create a nationally representative adult community sample based on gender, age, education, and region. The potential participants were contacted by email with information about the study. As an incentive, participants were told that at the end of the study they would receive reward points that could be converted into cash (about USD 0.8 for this survey). In the given period, 1133 response sets were obtained from the online survey platform. This number excluded data from participants who dropped out of the study after completing only part of the survey. Because a forced choice format was used in the survey, there were no missing values in the data collected (completers answered all the items in the survey in the final sample), and data from all 1133 participants were retained for analyses. The final sample included 548 (48.37%) males and 585 (51.63%) females, with ages ranging from 18 to 88 years (M = 31.23, SD = 8.73). The participants came from 232 cities across 29 provinces and municipalities of Mainland China. Table 1 presents the sociodemographic characteristics of the sample.
Participants were first informed about the research aim, the main content of the questionnaires, and completion time. They were also informed that the questionnaires were anonymous, and research data would only be used for research purposes. If participants consented to participate, they were then directed to the online data collection interface. Participants completed a series of questionnaires, but only the questionnaires related to the current study were analyzed. The study was approved by the Research Ethics Committee of the institution with which the first author is affiliated (HR151-2020).

2.2. Measures

2.2.1. Perceived Stress Scale

The full 14-item version of the Perceived Stress Scale includes two subscales: the negativity subscale (items 1, 2, 3, 8, 11, 12, and 14) and the positivity subscale (items 4, 5, 6, 7, 9, 10, and 13) [9]. The 10-item version retained the original two-factor structure: a negativity subscale with six items (items 1, 2, 3, 8, 11, and 14) and a positivity subscale with four items (items 6, 7, 9, and 10) [10]. The PSS-4, the briefest version of the PSS, includes only four items (items 2, 6, 7, and 14). Two items (items 6 and 7) are positively worded and the other two (items 2 and 14) are negatively worded. The original authors of the PSS reported that these four items loaded on a single factor [10].
Participants are asked to rate how often they experienced stressful situations in the past month on a five-point scale, from 0 (never) to 4 (very often). The PSS total score is calculated by reverse-coding the positive items and summing the scores for all items. The total scores range from 0 to 56, from 0 to 40, and from 0 to 16 for the PSS-14, PSS-10, and PSS-4, respectively. A higher score indicates more perceived stress. The PSS-14 was translated to Chinese to create the CPPS-14, which was validated in a sample of cancer survivors [14]. The CPPS-10 and CPPS-4 were created using the same subsets of items used to create the PSS-10 and PSS-4.

2.2.2. Health Questionnaire Depression Nine-Item Scale (PHQ-9)

The PHQ-9 is a nine-item scale designed to measure symptoms of depression [23]. Participants are asked to rate how often they have been bothered by depression-related problems in the past two weeks using a four-point scale, from 0 (not at all) to 3 (nearly every day). The PHQ-9 total score is calculated by adding all nine item scores. The total score ranges from 0 to 27, with a higher score indicating more severe depressive symptoms. The Chinese PHQ-9 has demonstrated good reliability and validity in the general population [24]. In the present study, the internal consistency was Cronbach’s alpha = 0.86.

2.2.3. Generalized Anxiety Disorder-7

The GAD-7 is a seven-item scale designed to measure anxiety symptoms [25]. Participants are asked to assess how often they have been bothered by anxiety-related problems in the past two weeks, using a four-point scale from 0 (not at all) to 3 (nearly every day). The GAD-7 total score is calculated by adding the seven item scores. The total scores range from 0 to 21, with a higher score indicating more severe anxiety symptoms. The GAD-7 has demonstrated good reliability and validity in the Chinese population [26,27]. In the present study, the internal consistency was Cronbach’s alpha = 0.87.

2.3. Data Analysis

The data analyses were conducted using the IMB Statistical Package for the Social Sciences software 21.0 (SPSS 21.0) and Mplus version 8.3 [28].
Descriptive statistics, internal consistency reliability, concurrent validity, and group comparisons were calculated in SPSS 21.0. Descriptive statistics were generated as means, standard deviations, and correlations; internal consistency reliability was evaluated by Cronbach’s alpha coefficient; concurrent validity was evaluated based on Pearson’s correlation between the CPSS total score and health-related variables (total scores on the PHQ-9 and GAD-7); group comparisons on gender, age, and educational level were made using t-tests and one-way ANOVAs [18,20]. All participants were split into three age cohorts for purposes of comparison. Given the whole sample was skewed toward the young (M = 31.23, SD = 8.73), the three age cohorts were labeled younger (less than 26 years old), middle (26–34), and older (35 and above) in this study (a reasonable number was considered in each cohort). All tests of significance were two-tailed, and results were considered significant at p < 0.05.
Confirmatory factor analysis (CFA) was conducted in Mplus 8.3 as a test of construct validity. CFAs were performed for each version of the CPSS (CPSS-4, CPSS-10, and CPSS-14). The parameter estimates for the CFAs were obtained using the maximum likelihood method (MLM). For the CFA analysis, the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR) were used to evaluate model fit. The fit was considered good if RMSEA was below 0.06, SRMR was below 0.08, and CFI and TLI were above 0.95 [29].

3. Results

3.1. Confirmatory Factor Analysis

This study used CFA to test for the well-established two-factor structure in the two longer versions of the CPSS, CPSS-14 and CPSS-10. In the expected two-factor structure, all positively worded items load on one factor and all negatively worded items load on the other. For the CPPS-4 we used CFA to test both the one-factor model (all four items on one factor) and the two-factor model (two positive items on one factor, and two negative items on the other factor).
Table 2 presents the fit indices for the proposed factor structures of the three versions of the CPSS. All three versions had a two-factor solution. The one-factor solution for the CPPS-4 was not supported in the present sample, as indicated by a poor fit with CFI, TLI, and RMSEA values. However, the two-factor solution for the CPPS-4 showed a near-perfect model fit, with both the CFI and TLI values above 0.95, and the RMSEA and SRMR values close to 0.
The standardized factor loadings of the two-factor models for the CPSS-14, CPSS-10, and CPSS-4 were almost all statistically significant and above 0.50, with the exception of item 12 on the CPSS-14 (with a statistically significant but low factor loading of 0.39 on the CPSS-14). Table 3 presents the standardized factor loadings of the two-factor models for all three versions of the CPPS.

3.2. Internal Consistency Reliability

For the CPSS-14, the Cronbach’s alpha was 0.85 for the full scale, 0.82 for the negative subscale, and 0.81 for the positive subscale. The corresponding values for the CPSS-10 were 0.83, 0.83, and 0.72, and for the CPSS-4 they were 0.66, 0.68, and 0.58. All values exceeded 0.60 as the criterion of acceptable internal consistency, except in the case of the positive subscale of the CPSS-4 (0.58).

3.3. Concurrent Validity

Concurrent validity was established by calculating the correlations between each of the versions of the CPSS (CPSS-14, CPSS-10, and CPSS-4) and the measures of depression and anxiety (the PHQ-9 and GAD-7, respectively). As presented in Table 4, the total scores on all three versions of the CPSS (CPSS-14, CPSS-10, and CPSS-4) were significantly related to the PHQ-9 and GAD-7 total scores. Concurrent validity was satisfied for all three CPSS versions.

3.4. Gender, Age, and Educational Level Differences

Table 5 presents the average total scores on the CPSS-4, CPSS-10, and CPSS-14 by gender, age, and educational level. The results showed that women as a group had a higher average score than men on all three versions of the CPSS (on CPSS-14, d = 0.25; on CPSS-10: d = 0.25; on CPSS-4, d = 0.24). The younger age cohort had a higher average score on perceived stress than the middle age and older age cohorts across the CPSS-14 (younger vs. middle/older cohort, d = 0.46), and CPSS-10 (younger vs. middle/older cohort, d = 0.39). On the CPSS-4, the average score in the younger cohort was significantly higher than the middle cohort (younger vs. middle cohort, d = 0.36) and older cohort (younger vs. older cohort, d = 0.43); the average scores for middle and older cohorts were not significantly different from each other (p > 0.05, d = 0.08). There was no difference in the average scores based on education level on any of the three CPSS measures.

4. Discussion

The current study addressed two gaps in the literature on the Chinese Perceived Stress Scale. First, although the CPSS-14 and CPSS-10 have been shown to have good psychometric properties, there have been no psychometric studies on the CPSS-4 in Mainland China. Second, research comparing the psychometric properties of all three versions of the PSS has been conducted in very specific populations, such as cardiac patients [20]. The current study compared the psychometric properties of the three versions of the CPSS in a representative sample in Mainland China.
A two-factor structure of all three versions of the CPSS was well supported by confirmatory factor analysis in this study. These findings echoed those of previous research on the PSS-10 and PSS-14 [8]. However, the two-factor structure for the CPSS-4 was inconsistent with the one-factor structure reported by the original authors of the CPSS-4 [10], and with another study that found both the one-factor and two-factor structures to be acceptable [19]. In fact, the two-factor solution for the CPSS-4 showed a good fit to the data in our sample, and was consistent with a recent study in Italy in which EFA and CFA supported a two-factor structure for the original PSS-4 [18]. The difference in the factor structure across studies may be due to the nature of the samples and the statistical approach [10]. The original PSS-4 had a one-factor solution in college samples [15,16], whereas it had a two-factor solution in community samples [17,18,19,20,30] with the exception of Cohen and Williamson’s original study, which found a one-factor solution in a community sample [10]. However, Cohen and Williamson used EFA to test the PSS-4’s original factor structure [10], and later researchers used CFA to confirm the structure. The present study sampled from a community population and the results supported the two-factor solution for the PSS-4 found in most community samples.
The internal consistencies for the CPSS-14 and CPSS-10 were very good, with Cronbach’s alphas both above 0.80. The Cronbach’s alpha value was lower (0.66) for the CPSS-4, although it was slightly higher than in the original authors’ report (0.60) [10] and was very close to the value reported in research on the CPSS-4 in a Hong Kong sample (0.67) [20]. The internal consistency reliability for the CPSS-4 in the present study is considered acceptable, especially given the small number of items, as the alpha coefficient increases with more items [31]. Given the adequate reliability and well-established two-factor structure for the CPSS-4, the CPSS-4 could be used as two subscales in future research in China.
The concurrent validity of the CPSS was also established in this study. The CPSS-14, CPSS-10, and CPSS-4 all showed strong correlations with the PHQ-9 and GAD-7. The results suggested that more stress perceived by individuals under the COVID-19 pandemic may be linked to higher depression and anxiety symptoms. It is worth noting that the correlation coefficients between the CPSS-4 and the criterion variables (PHQ-9 and GAD-7) were comparable with those of the longer CPSS-10 and CPSS-14, providing further validity information for the four-item CPSS. This should increase researchers’ confidence in using this briefest version in future clinical research and practice.
Furthermore, the present study found gender and age cohort differences on the CPSS. In line with prior work [20], women perceived higher levels of stress than men, while the older population tended to perceive less stress than the younger population. However, the total scores did not vary based on education level, consistent with a study conducted on the original PSS in Italy [18]. It is worth noting that on the CPSS-4 the average scores were highest in the younger cohort (less than 26 years), then trended downward and remained stable in the middle cohort (26–34 years) and the older cohort (more than 34 years). However, this finding must be interpreted with caution because the older cohort had fewer participants and a wider age range than the other cohorts. Future studies may include more participants from the older population to examine the relationship between age cohort and CPSS-4 total score. It will also be important to identify the reasons for the high CPSS scores among the youngest participants. Given that women and the younger population in our sample experienced higher stress under the COVID-19 pandemic, these groups may need more mental health care during this challenging time. In future research the CPSS could be tested as a screening measure for targeted intervention, in line with the idea of personalized medicine proposed in the existing literature [32].
This study has several limitations that should be taken into account when interpreting the results. First, some of the correlations may have been inflated because of shared method variance, as all variables were assessed using self-report questionnaires. Future researchers can use more objective stress measures, such as physiological or behavioral indicators, to investigate the construct validity of the CPSS. Second, this study did not investigate the test–retest reliability of the CPSS. Evidence of this type of reliability would be useful for longitudinal research on the perception of stress. Third, although one of this study’s strengths was the use of a large, nationally representative sample, the subgroup of older adults was relatively small and there were other subgroups that were not examined in the current study, such as health care workers [33]. Finally, the results obtained in this community sample do not provide direct evidence of whether the CPSS will be useful for screening and intervention in a clinical population.

5. Conclusions

To our knowledge, this was the first study to examine the factor structure of the CPSS-4 and to compare the psychometric properties of all three versions of the CPSS in a large and representative sample from Mainland China. The CPSS-14, CPSS-10, and CPSS-4 demonstrated adequate to good internal consistency, construct validity, and concurrent validity in the Chinese general population under the high-stress context of the COVID-19 pandemic. In addition, two subscales were identified in all three CPSS versions. All three CPSS versions would be useful for research purposes and may have utility as stress screening measures for mental health intervention in the Chinese population. Given the greater length of the CPSS-14 and the fact that some items on the measure have low factor loadings, and the lower alpha coefficient for the CPSS-4, it appears that the CPSS-10 would be the best choice for general use. However, the CPSS-4 is still an acceptable measure of perceived stress, especially under time constraints (e.g., telephone interviews).

Author Contributions

Conceptualization, Z.S., J.X., and D.L.; methodology, Z.S.; software, Z.S. and W.Z.; data curation, Z.S., D.L., W.Z., N.Z., and K.J.; writing—original draft preparation, Z.S. and D.L.; writing—review and editing, W.Z., N.Z., and J.X.; project administration, D.L.; funding acquisition, D.L. and J.X. All authors read and approved the final version of the manuscript.

Funding

This work was supported by East China Normal University’s “COVID-19 Prevention” special project (Grant No. 2020ECNU-FZZX-06); the National Social Science Foundation of China (Grant No. 18BSH129); the Shanghai Philosophy and Social Science Planning Project (Grant No. 2017BSH004); the Shanghai Pujiang Program of China (Grant No. 12PJC037); the Research Project of Shanghai Science and Technology Commission (Grant No. 20DZ2260300); and the Science and Technology Innovation Action Plan of the Shanghai Science and Technology Commission (Grant No. 20DZ2304400).

Institutional Review Board Statement

This study was approved by the first author’s University Research Ethics Committee (HR151-2020).

Informed Consent Statement

All procedures were in accordance with the ethical standards of the 1964 Helsinki Declaration and its later amendments. All participants enrolled in the study provided voluntary written informed consent. Participants’ responses were anonymous.

Data Availability Statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Lazarus, R.S.; Folkman, S. Stress, Appraisal, and Coping; Springer: New York, NY, USA, 1984; pp. 19–21. [Google Scholar]
  2. Qi, M.; Li, P.; Moyle, W.; Weeks, B.; Jones, C. Physical activity, health-related quality of life, and stress among the Chinese adult population during the COVID-19 pandemic. Int. J. Environ. Res. Public Health 2020, 17, 6494. [Google Scholar] [CrossRef]
  3. Lau, J.T.; Yang, X.; Pang, E.; Tsui, H.Y.; Wong, E.; Wing, Y.K. SARS-related perceptions in Hong Kong. Emerg. Infect. Dis. 2005, 11, 417–424. [Google Scholar] [PubMed]
  4. Pedrozo-Pupo, J.C.; Pedrozo-Cortés, M.J.; Campo-Arias, A. Perceived stress associated with COVID-19 epidemic in Colombia: An online survey. Cadernos de Saude Publica 2020, 36, e00090520. [Google Scholar] [CrossRef]
  5. Fofana, N.K.; Latif, F.; Sarfraz, S.; Bashir, M.F.; Komal, B. Fear and agony of the pandemic leading to stress and mental illness: An emerging crisis in the novel coronavirus (COVID-19) outbreak. Psychiatry Res. 2020, 291, 113230. [Google Scholar] [CrossRef] [PubMed]
  6. Li, J.; Yang, Z.; Qiu, H.; Wang, Y.; Jian, L.; Ji, J.; Li, K. Anxiety and depression among general population in China at the peak of the COVID-19 epidemic. World Psychiatry 2020, 19, 249–250. [Google Scholar] [CrossRef]
  7. Salari, N.; Hosseinian-Far, A.; Jalali, R.; Vaisi-Raygani, A.; Rasoulpoor, S.; Mohammadi, M.; Khaledi-Paveh, B. Prevalence of stress, anxiety, depression among the general population during the COVID-19 pandemic: A systematic review and meta-analysis. Global. Health 2020, 16, 1–11. [Google Scholar] [CrossRef] [PubMed]
  8. Lee, E.H. Review of the psychometric evidence of the perceived stress scale. Asian Nurs. Res. 2012, 6, 121–127. [Google Scholar] [CrossRef] [Green Version]
  9. Cohen, S.; Kamarck, T.; Mermelstein, R. A global measure of perceived stress. J. Health Soc. Behav. 1983, 24, 385–396. [Google Scholar] [CrossRef] [PubMed]
  10. Cohen, S.; Williamson, G.M. Perceived stress in a probability sample of the United States. In The Social Psychology of Health; Spacapan, S., Oskamp, S., Eds.; Sage: Newbury Park, CA, USA, 1988; pp. 31–67. [Google Scholar]
  11. Chaaya, M.; Osman, H.; Naassan, G.; Mahfoud, Z. Validation of the Arabic version of the Cohen perceived stress scale (PSS-10) among pregnant and postpartum women. BMC Psychiatry 2010, 10, 111. [Google Scholar] [CrossRef] [Green Version]
  12. Mitchell, A.M.; Crane, P.A.; Kim, Y. Perceived stress in survivors of suicide: Psychometric properties of the Perceived Stress Scale. Res. Nurs. Health. 2008, 31, 576–585. [Google Scholar] [CrossRef] [PubMed]
  13. Wang, Z.; Chen, J.; Boyd, J.E.; Zhang, H.; Jia, X.; Qiu, J.; Xiao, Z. Psychometric properties of the Chinese version of the Perceived Stress Scale in policewomen. PLoS ONE 2011, 6, e28610. [Google Scholar] [CrossRef] [PubMed]
  14. Deng, Q.; Ruoyu, H.U.; Wang, J.; Yuan, Z.; Jinming, Y.U. Validity and reliability of the Perceived Stress Scale among cancer survivors. Chin. Ment. Health J. 2018, 32, 15–20. [Google Scholar]
  15. Herrero, J.; Meneses, J. Short Web-based versions of the perceived stress (PSS) and Center for Epidemiological Studies-Depression (CESD) Scales: A comparison to pencil and paper responses among Internet users. Comput. Hum. Behav. 2006, 22, 830–846. [Google Scholar] [CrossRef]
  16. Ruisoto, P.; López-Guerra, V.M.; Paladines, M.B.; Vaca, S.L.; Cacho, R. Psychometric properties of the three versions of the Perceived Stress Scale in Ecuador. Physiol. Behav. 2020, 224, 113045. [Google Scholar] [CrossRef]
  17. Ingram, P.B., IV; Clarke, E.; Lichtenberg, J.W. Confirmatory factor analysis of the perceived stress Scale-4 in a community sample. Stress Health 2016, 32, 173–176. [Google Scholar] [CrossRef] [PubMed]
  18. Mondo, M.; Sechi, C.; Cabras, C. Psychometric evaluation of three versions of the Italian perceived stress scale. Curr. Psychol. 2021, 40, 1884–1892. [Google Scholar] [CrossRef]
  19. Andreou, E.; Alexopoulos, E.C.; Lionis, C.; Varvogli, L.; Gnardellis, C.; Chrousos, G.P.; Darviri, C. Perceived stress scale: Reliability and validity study in Greece. Int. J. Environ. Res. Public Health 2011, 8, 3287–3298. [Google Scholar] [CrossRef]
  20. Leung, D.Y.P.; Lam, T.; Chan, S.S.C. Three versions of Perceived Stress Scale: Validation in a sample of Chinese cardiac patients who smoke. BMC Public Health 2010, 10, 513. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Yu, R.; Ho, S.C. Psychometric evaluation of the perceived stress scale in early postmenopausal Chinese women. Psychology 2010, 1, 1–8. [Google Scholar] [CrossRef] [Green Version]
  22. Huang, F.; Wang, H.; Wang, Z.; Zhang, J.; Du, W.; Su, C. Psychometric properties of the perceived stress scale in a community sample of Chinese. BMC Psychiatry 2020, 20, 1–7. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Kroenke, K.; Spitzer, R.L. The PHQ-9: A new depression diagnostic and severity measure. Psychiatr. Ann. 2002, 32, 509–515. [Google Scholar] [CrossRef] [Green Version]
  24. Wang, W.; Bian, Q.; Zhao, Y.; Li, X.; Wang, W.; Du, J.; Zhang, G.; Zhou, Q.; Zhao, M. Reliability and validity of the Chinese version of the Patient Health Questionnaire (PHQ-9) in the general population. Gen. Hosp. Psych. 2014, 36, 539–544. [Google Scholar] [CrossRef] [PubMed]
  25. Spitzer, R.L.; Kroenke, K.; Williams, J.B.; Löwe, B. A brief measure for assessing generalized anxiety disorder: The GAD-7. Arch. Intern. Med. 2006, 166, 1092–1097. [Google Scholar] [CrossRef] [Green Version]
  26. Tong, X.; An, D.; McGonigal, A.; Park, S.-P.; Zhou, D. Validation of the Generalized Anxiety Disorder-7 (GAD-7) among Chinese people with epilepsy. Epilepsy Res. 2016, 120, 31–36. [Google Scholar] [CrossRef]
  27. Gong, Y.; Zhou, H.; Zhang, Y.; Zhu, X.; Wang, X.; Shen, B.; Xian, J.; Ding, Y. Validation of the 7-item Generalized Anxiety Disorder scale (GAD-7) as a screening tool for anxiety among pregnant Chinese women. J. Affect. Disord. 2021, 282, 98–103. [Google Scholar] [CrossRef]
  28. Muthén, L.K.; Muthén, B.O. Mplus User’s Guide, 8th ed.; Muthén Muthén: Los Angeles, CA, USA, 2017. [Google Scholar]
  29. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equat. Modeling Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  30. Hong, G.R.S.; Kang, H.K.; Oh, E.; Park, Y.; Kim, H. Reliability and validity of the Korean version of the perceived stress scale-10 (K-PSS-10) in older adults. Res. Gerontol. Nurs. 2016, 9, 45–51. [Google Scholar] [CrossRef] [PubMed]
  31. Peterson, R.A. A meta-analysis of Cronbach’s coefficient alpha. J. Consum. Res. 1994, 21, 381–391. [Google Scholar] [CrossRef]
  32. Wang, J.; Li, J.; Li, Y.; Wong, W.K. A model-based multithreshold method for subgroup identification. Stat. Med. 2019, 38, 2605–2631. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Wang, H.; Liu, Y.; Hu, K.; Zhang, M.; Du, M.; Huang, H.; Yue, X. Healthcare workers’ stress when caring for COVID-19 patients: An altruistic perspective. Nurs. Ethics 2020, 27, 1490–1500. [Google Scholar] [CrossRef]
Table 1. Sociodemographic characteristics of the sample.
Table 1. Sociodemographic characteristics of the sample.
Variables(N = 1133) n (%)
Gender
Males548 (48.4%)
Females585 (51.6%)
Age
≤25298 (26.3%)
26–34517 (45.6%)
≥35318 (28.1%)
Educational level
Below high school48 (4.2%)
High school degree76 (6.7%)
Associate degree166 (14.7%)
Bachelor degree740 (65.3%)
Graduate degree103 (9.1%)
Table 2. Goodness of fit indexes for the proposed CPSS models.
Table 2. Goodness of fit indexes for the proposed CPSS models.
χ2dfpCFITLIRMSEASRMR
CPPS-14
two-factor model148.26676<0.0010.9810.9770.0290.034
CPPS-10
two-factor model65.82534<0.0010.9880.9840.0290.023
CPPS-4
one-factor model86.0342<0.0010.8650.5940.1930.064
two-factor model0.07610.7831.0001.0090.0000.001
Note: χ2, chi-square value; df = degrees of freedom; CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual.
Table 3. Standardized factor loadings for three versions of the CPSS.
Table 3. Standardized factor loadings for three versions of the CPSS.
ItemsCPSS-14CPSS-10CPSS-4
NegativePositiveNegativePositiveNegativePositive
1 Upset by something happening unexpectedly?0.68800.6850
2 Unable to control the important things in your life?0.71700.71600.7750
3 Nervous and stressed?0.67700.6770
8 Could not cope with all the things that you had to do?0.60200.6030
11 Angered because of things that were outside your control?0.64800.6460
12 Thinking about things that you have to accomplish?0.3870
14 Difficulties were piling up so high that you could not overcome them?0.71000.70900.6620
4 Dealt successfully with day-to-day problems and annoyances?00.637
5 Effectively coping with important changes that were occurring in your life?00.622
6 Confident about your ability to handle your personal problems?00.68700.64900.710
7 Things were going your way?00.58900.59600.570
9 Dealt successfully with irritating life hassles?00.58400.600
10 You were on top of things?00.62300.658
13 Able to control the way you spend your time?00.569
Factor correlation0.530.520.56
Cronbach’s alpha0.820.810.830.720.680.58
0.850.830.66
Table 4. Correlations between three versions of the CPSS and criterion-related measures.
Table 4. Correlations between three versions of the CPSS and criterion-related measures.
CPSS-14CPSS-10CPSS-4
PHQ-90.61 ***0.62 ***0.54 ***
GAD-70.59 ***0.62 ***0.51 ***
Note: PHQ-9, Patient Health Questionnaire-9; GAD-7, Generalized Anxiety Disorder-7. *** p < 0.001.
Table 5. Mean differences by gender, age, and educational level.
Table 5. Mean differences by gender, age, and educational level.
CPSS-14CPSS-10CPSS-4
MeanSDF/tESpMeanSDF/tESpMeanSDF/tESp
Gender a 4.12d = 0.25<0.001 4.14d = 0.25<0.001 4.01d = 0.24<0.001
Male (n = 557)36.037.48 25.725.91 9.662.68
Female (n = 595)37.978.30 27.246.45 10.332.91
Age (years) b (Mean = 31.23) 2.52η2 = 0.09<0.001 2.10η2 = 0.08<0.001 2.02η2 = 0.07<0.001
≤25 (younger; n = 298)39.628.24 28.206.40 10.822.95
26–34 (middle; n = 517)36.537.70 26.246.06 9.802.74
≥35 (older; n = 318)35.417.57 25.366.06 9.592.71 .
Education level b 0.60η2 < 0.000.67 0.65η2 < 0.000.63 0.66η2 < 0.000.63
Below high school (n = 48)38.157.54 27.696.20 10.652.82
High school (n = 76)37.727.79 26.816.04 10.032.58
Associate degree (n = 166)37.177.34 26.455.99 10.002.64
Bachelor (n = 740)36.948.02 26.486.24 9.982.84
Graduate degree (n = 103)36.358.88 26.016.80 9.913.15
Note: SD, standard deviation; ES, effect size. a The value was calculated with independent t tests; b the value was calculated with one-way analysis of variance (ANOVA).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

She, Z.; Li, D.; Zhang, W.; Zhou, N.; Xi, J.; Ju, K. Three Versions of the Perceived Stress Scale: Psychometric Evaluation in a Nationally Representative Sample of Chinese Adults during the COVID-19 Pandemic. Int. J. Environ. Res. Public Health 2021, 18, 8312. https://doi.org/10.3390/ijerph18168312

AMA Style

She Z, Li D, Zhang W, Zhou N, Xi J, Ju K. Three Versions of the Perceived Stress Scale: Psychometric Evaluation in a Nationally Representative Sample of Chinese Adults during the COVID-19 Pandemic. International Journal of Environmental Research and Public Health. 2021; 18(16):8312. https://doi.org/10.3390/ijerph18168312

Chicago/Turabian Style

She, Zhuang, Dan Li, Wei Zhang, Ningning Zhou, Juzhe Xi, and Kang Ju. 2021. "Three Versions of the Perceived Stress Scale: Psychometric Evaluation in a Nationally Representative Sample of Chinese Adults during the COVID-19 Pandemic" International Journal of Environmental Research and Public Health 18, no. 16: 8312. https://doi.org/10.3390/ijerph18168312

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop