The Effect of Psychological Distress on Measurement Invariance in Measures of Mental Wellbeing
Abstract
:1. Introduction
1.1. Measurement Invariance of the Mental Health Continuum Short Form
1.2. Aim of the Current Study
2. Materials and Method
2.1. Participants and Procedures
2.2. Measures
2.3. Exploratory and Confirmatory Factor Analysis
2.4. Invariance Testing
2.5. Configural Invariance Testing
2.6. Metric Invariance Testing
3. Results
3.1. Exploratory Factor Analysis
3.2. Confirmatory Factor Analysis
3.3. Invariance Testing
3.4. Configural Invariance Testing
3.5. Metric Invariance Testing
4. Discussion
4.1. Implications for Theory
4.2. Limitations and Future Directions
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Jahoda, M. Current Concepts of Positive Mental Health. Basic Books. 1958. Available online: https://psycnet.apa.org/PsycBOOKS/toc/11258 (accessed on 6 July 2022).
- Fontana, A.F.; Marcus, J.L.; Dowds, B.N.; Hughes, L.A. Psychological impairment and psychological health in the psychological well-being of the physically ill. Psychosom. Med. 1980, 42, 279–288. [Google Scholar] [CrossRef]
- Massé, R.; Poulin, C.; Dassa, C.; Lambert, J.; Bélair, S.; Battaglini, A. The structure of mental health: Higher-order confirmatory factor analyses of psychological distress and well-being measures. Soc. Indic. Res. 1998, 45, 475–504. [Google Scholar] [CrossRef]
- Greenspoon, P.J.; Saklofske, D.H. Toward an integration of subjective well-being and psychopathology. Soc. Indic. Res. 2001, 54, 81–108. [Google Scholar] [CrossRef]
- Keyes, C.L. Mental illness and/or mental health? Investigating axioms of the complete state model of health. J. Consult. Clin. Psychol. 2005, 73, 539. [Google Scholar] [CrossRef]
- Iasiello, M.; Van Agteren, J.; Muir-Cochrane, E. Mental health and/or mental illness: A scoping review of the evidence and implications of the dual-continua model of mental health. Evid. Base A J. Evid. Rev. Key Policy Areas 2020, 1, 1–45. [Google Scholar] [CrossRef]
- Keyes, C.L. Promoting and protecting mental health as flourishing: A complementary strategy for improving national mental health. Am. Psychol. 2007, 62, 95. [Google Scholar] [CrossRef]
- Slade, M. Mental illness and well-being: The central importance of positive psychology and recovery approaches. BMC Health Serv. Res. 2010, 10, 26. [Google Scholar] [CrossRef]
- Grant, F.; Guille, C.; Sen, S. Well-being and the risk of depression under stress. PLoS ONE 2013, 8, e67395. [Google Scholar] [CrossRef]
- Joseph, S.; Wood, A. Assessment of positive functioning in clinical psychology: Theoretical and practical issues. Clin. Psychol. Rev. 2010, 30, 830–838. [Google Scholar] [CrossRef]
- Wood, A.M.; Taylor, P.J.; Joseph, S. Does the CES-D measure a continuum from depression to happiness? Comparing substantive and artifactual models. Psychiatry Res. 2010, 177, 120–123. [Google Scholar] [CrossRef]
- Blanco-Canitrot, D.; Alvarado, J.M.; Ondé, D. Consequences of Disregarding Metric Invariance on Diagnosis and Prognosis Using Psychological Tests. Front. Psychol. 2018, 9, 167. [Google Scholar] [CrossRef]
- Putnick, D.L.; Bornstein, M.H. Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev. Rev. 2016, 41, 71–90. [Google Scholar] [CrossRef]
- Santini, Z.I.; Torres-Sahli, M.; Hinrichsen, C.; Meilstrup, C.; Madsen, K.R.; Rayce, S.B.; Baker, M.M.; Ten Have, M.; Schotanus-Dijkstra, M.; Koushede, V. Measuring positive mental health and flourishing in Denmark: Validation of the mental health continuum-short form (MHC-SF) and cross-cultural comparison across three countries. Health Qual. Life Outcomes 2020, 18, 297. [Google Scholar] [CrossRef]
- Fonte, C.; Silva, I.; Vilhena, E.; Keyes, C.L.M. The Portuguese Adaptation of the Mental Health Continuum-Short Form for Adult Population. Community Ment. Health J. 2020, 56, 368–375. [Google Scholar] [CrossRef]
- Joshanloo, M.; Wissing, M.P.; Khumalo, I.P.; Lamers, S.M. Measurement invariance of the Mental Health Continuum-Short Form (MHC-SF) across three cultural groups. Personal. Individ. Differ. 2013, 55, 755–759. [Google Scholar] [CrossRef]
- Bornstein, M.H. Form and Function: Implications for Studies of Culture and Human Development. Cult. Psychol. 1995, 1, 123–137. [Google Scholar] [CrossRef]
- Prinzing, M.M. Positive psychology is value-laden—It’s time to embrace it. J. Posit. Psychol. 2021, 16, 289–297. [Google Scholar] [CrossRef]
- Wilson, T.D.; Gilbert, D.T. Affective Forecasting. In Advances in Experimental Social Psychology; Zanna, M.P., Ed.; Elsevier Academic Press: Amsterdam, The Netherlands, 2003; Volume 35, pp. 345–411. [Google Scholar] [CrossRef]
- Haybron, D.M. Taking the satisfaction (and the life) out of life satisfaction. Philos. Explor. 2011, 14, 249–262. [Google Scholar] [CrossRef]
- Widaman, K.; Reise, S. Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In The Science of Prevention: Methodological Advances from Alcohol and Substance Abuse Research; Bryant, K.J., Windle, M., West, S.G., Eds.; American Psychological Association: Washington, DC, USA, 1997; pp. 281–324. [Google Scholar]
- Franken, K.; Lamers, S.M.; Ten Klooster, P.M.; Bohlmeijer, E.T.; Westerhof, G.J. Validation of the Mental Health Continuum-Short Form and the dual continua model of well-being and psychopathology in an adult mental health setting. J. Clin. Psychol. 2018, 74, 2187–2202. [Google Scholar] [CrossRef]
- Ferentinos, P.; Yotsidi, V.; Porichi, E.; Douzenis, A.; Papageorgiou, C.; Stalikas, A. Well-being in patients with affective disorders compared to nonclinical participants: A multi-model evaluation of the Mental Health Continuum-Short Form. J. Clin. Psychol. 2019, 75, 1585–1612. [Google Scholar] [CrossRef]
- Iasiello, M.; van Agteren, J.; Schotanus-Dijkstra, M.; Lo, L.; Fassnacht, D.; Westerhof, G.J. Assessing mental wellbeing using the Mental Health Continuum—Short Form: A systematic review and meta-analytic structural equation modelling. Clin. Psychol. Sci. Pract. 2022; Advance online publication. [Google Scholar]
- Keyes, C.L.M.; Wissing, M.; Potgieter, J.P.; Temane, M.; Kruger, A.; van Rooy, S. Evaluation of the mental health continuum–short form (MHC–SF) in setswana-speaking South Africans. Clin. Psychol. Psychother. 2008, 15, 181–192. [Google Scholar] [CrossRef] [PubMed]
- Henry, J.D.; Crawford, J.R. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br. J. Clin. Psychol. 2005, 44 Pt 2, 227–239. [Google Scholar] [CrossRef]
- Crawford, J.R.; Henry, J.D. The Depression Anxiety Stress Scales (DASS): Normative data and latent structure in a large non-clinical sample. Br. J. Clin. Psychol. 2003, 42, 111–131. [Google Scholar] [CrossRef] [PubMed]
- O’connor, B.P. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behav. Res. Methods Instrum. Comput. 2000, 32, 396–402. [Google Scholar] [CrossRef] [PubMed]
- Schermelleh-Engel, K.; Moosbrugger, H.; Müller, H. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods Psychol. Res. Online 2003, 8, 23–74. [Google Scholar]
- Byrne, B.M. Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, 2nd ed.; Routledge: New York, NY, USA, 2010. [Google Scholar]
- Pirritano, M. Gamma Hat McDonald’s NCI Model Chi-Square Model df # of Observed Variables. 2018. Available online: https://www.researchgate.net/publication/323839709_Gamma_Hat_McDonald%27s_NCI_Model_chi-square_model_df_of_observed_variables?channel=doi&linkId=5aaed39d458515ecebe971dc&showFulltext=true (accessed on 6 July 2022).
- Cheung, G.W.; Rensvold, R.B. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equ. Model. 2002, 9, 233–255. [Google Scholar] [CrossRef]
- Hides, L.; Quinn, C.; Stoyanov, S.; Cockshaw, W.; Kavanagh, D.J.; Shochet, I.; Deane, F.; Kelly, P.; Keyes, C.L. Testing the interrelationship between mental well-being and mental distress in young people. J. Posit. Psychol. 2020, 15, 314–324. [Google Scholar] [CrossRef]
- Steenkamp, J.-B.E.M.; Baumgartner, H. Assessing Measurement Invariance in Cross-National Consumer Research. J. Consum. Res. 1998, 25, 78–107. [Google Scholar] [CrossRef]
- Chen, F.F. What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. J. Pers. Soc. Psychol. 2008, 95, 1005–1018. [Google Scholar] [CrossRef]
- Newman, A.; Donohue, R.; Eva, N. Psychological safety: A systematic review of the literature. Hum. Resour. Manag. Rev. 2017, 27, 521–535. [Google Scholar] [CrossRef]
- Ryan, R.M. Psychological needs and the facilitation of integrative processes. J. Personal. 1995, 63, 397–427. [Google Scholar] [CrossRef] [PubMed]
- Deci, E.L.; Ryan, R.M. Self-Determination Theory. In Handbook of Theories of Social Psychology; Van Lange, P.A.M., Kruglanski, A.W., Higgins, E.T., Eds.; Sage Publications Ltd.: Thousand Oaks, CA, USA, 2012; pp. 416–436. [Google Scholar] [CrossRef]
- Mjøsund, N.H.; Eriksson, M.; Norheim, I.; Keyes, C.L.M.; Espnes, G.A.; Vinje, H.F. Mental health as perceived by persons with mental disorders—An interpretative phenomenological analysis study. Int. J. Ment. Health Promot. 2015, 17, 215–233. [Google Scholar] [CrossRef]
- Leamy, M.; Bird, V.; Boutillier, C.L.; Williams, J.; Slade, M. Conceptual framework for personal recovery in mental health: Systematic review and narrative synthesis. Br. J. Psychiatry 2011, 199, 445–452. [Google Scholar] [CrossRef]
- Wenze, S.J.; Gunthert, K.C.; German, R.E. Biases in affective forecasting and recall in individuals with depression and anxiety symptoms. Personal. Soc. Psychol. Bull. 2012, 38, 895–906. [Google Scholar] [CrossRef]
- Colombo, D.; Suso-Ribera, C.; Fernández-Álvarez, J.; Cipresso, P.; Garcia-Palacios, A.; Riva, G.; Botella, C. Affect Recall Bias: Being Resilient by Distorting Reality. Cogn. Ther. Res. 2020, 44, 906–918. [Google Scholar] [CrossRef]
- Rios, J.A. Is differential noneffortful responding associated with type I error in measurement invariance testing? Educ. Psychol. Meas. 2021, 81, 957–979. [Google Scholar] [CrossRef] [PubMed]
- Meade, A.; Bauer, D.J. Power and precision in confirmatory factor analytic tests of measurement invariance. Struct. Equ. Model. A Multidiscip. J. 2007, 14, 611–635. [Google Scholar] [CrossRef]
Total (n = 8406) | Non-Distressed (n = 6420) | Distressed (n = 1986) | |
---|---|---|---|
Age (years), mean (SD) | 42.1 (13.4) | 43.5 (13.4) | 37.7 (13.4) |
Gender (female), n (%) | 4104 (48.8) | 3183 (49.6) | 920 (46.3) |
Unemployed, n (%) | 1716 (20.4) | 1099 (17.1) | 617 (31.1) |
Studying, n (%) | 959 (11.4) | 649 (10.1) | 311 (15.7) |
Item | Emotional Wellbeing | Social Wellbeing | Psychological Wellbeing |
---|---|---|---|
1 happy | 0.82 | ||
2 interested in life | 0.73 | ||
3 satisfied with life | 0.74 | ||
4 that you had something important to contribute to society | 0.56 | ||
5 that you belonged to a community (like a social group, or your neighborhood) | 0.32 | ||
6 that our society is a good place, or is becoming a better place, for all people | 0.82 | ||
7 that people are basically good | 0.74 | ||
8 that the way our society works makes sense to you | 0.80 | ||
9 that you liked most parts of your personality | 0.72 | ||
10 good at managing the responsibilities of your daily life | 0.66 | ||
11 that you had warm and trusting relationships with others | 0.61 | ||
12 that you had experiences that challenged you to grow and become a better person | 0.73 | ||
13 confident to think or express your own ideas and opinions | 0.81 | ||
14 that your life has a sense of direction or meaning to it | 0.66 |
Model | χ2 | RMSEA | CFI | TLI | Δχ2 |
---|---|---|---|---|---|
Configural | 2138.50 (148) | 0.043 | 0.96 | 0.95 | 59.41 p < 0.001 |
Metric | 2197.91 (159) | 0.042 | 0.96 | 0.96 |
Item | Factor | Item Loading | |
---|---|---|---|
Non-Distressed | Distressed | ||
1 | Emotional wellbeing | 0.75 | 0.73 |
2 | Emotional wellbeing | 0.80 | 0.83 |
3 | Emotional wellbeing | 0.83 | 0.83 |
4 | Psychological wellbeing | 0.84 | 0.85 |
5 | Psychological wellbeing | 0.78 | 0.69 |
6 | Social wellbeing | 0.79 | 0.75 |
7 | Social wellbeing | 0.73 | 0.71 |
8 | Social wellbeing | 0.66 | 0.64 |
9 | Psychological wellbeing | 0.73 | 0.68 |
10 | Psychological wellbeing | 0.61 | 0.54 |
11 | Psychological wellbeing | 0.69 | 0.65 |
12 | Psychological wellbeing | 0.63 | 0.69 |
13 | Psychological wellbeing | 0.68 | 0.63 |
14 | Psychological wellbeing | 0.82 | 0.81 |
Relaxed Item: | χ2 | df | Δχ2 | Δdf | p |
---|---|---|---|---|---|
1 | 2196.24 | 158 | 1.66 | 1 | 0.198 |
2 | 2179.20 | 158 | 18.70 | 1 | <0.001 |
3 | 2189.50 | 158 | 8.40 | 1 | 0.004 |
4 | 2193.98 | 158 | 3.92 | 1 | 0.048 |
5 | 2182.95 | 158 | 14.95 | 1 | <0.001 |
6 | 2196.90 | 158 | 1.00 | 1 | 0.317 |
7 | 2197.80 | 158 | 0.10 | 1 | 0.752 |
8 | 2197.90 | 158 | 0.00 | 1 | 1.000 |
9 | 2196.94 | 158 | 0.96 | 1 | 0.327 |
10 | 2197.90 | 158 | 0.00 | 1 | 1.000 |
11 | 2192.85 | 158 | 5.05 | 1 | 0.025 |
12 | 2180.50 | 158 | 17.40 | 1 | <0.001 |
13 | 2197.40 | 158 | 0.50 | 1 | 0.479 |
14 | 2197.50 | 158 | 0.40 | 1 | 0.527 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Iasiello, M.; Muir-Cochrane, E.; van Agteren, J.; Fassnacht, D.B. The Effect of Psychological Distress on Measurement Invariance in Measures of Mental Wellbeing. Int. J. Environ. Res. Public Health 2022, 19, 10072. https://doi.org/10.3390/ijerph191610072
Iasiello M, Muir-Cochrane E, van Agteren J, Fassnacht DB. The Effect of Psychological Distress on Measurement Invariance in Measures of Mental Wellbeing. International Journal of Environmental Research and Public Health. 2022; 19(16):10072. https://doi.org/10.3390/ijerph191610072
Chicago/Turabian StyleIasiello, Matthew, Eimear Muir-Cochrane, Joep van Agteren, and Daniel B. Fassnacht. 2022. "The Effect of Psychological Distress on Measurement Invariance in Measures of Mental Wellbeing" International Journal of Environmental Research and Public Health 19, no. 16: 10072. https://doi.org/10.3390/ijerph191610072