Revaluating the Dimensionality of Academic Engagement: A Bifactor Analysis of the UWES in Higher Education

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This is a research note examining the performance of the Utrecht Work Engagement Scale (UWES). This study is worthwhile given the cited experiences of researchers who used this instrument in prior studies.

The authors point out possible limitations of confirmatory factor analysis and possible resolution of testing different models using Schaufeli et al.'s (2002) dimensions.

The literature cited is appropriate. No changes needed

The sample population and sample size are both appropriate. No changes needed.

The analyses used are appropriate. No changes needed.

The writing is fine as it is. No changes needed. 

The authors' recommendations are appropriate given the task presented in the introduction. While no changes are needed, it seems that an underlying problem of the UWES instrument may be Schaufeli et al.'s (2002) assumption that engagement is the opposite of burnout. 

Author Response

Dear Editor and Reviewers,
We thank you for your careful evaluation of our manuscript. Below we provide a point‑by‑point response to each reviewer. All requested changes have been incorporated into the revised version (v02.5), where modifications appear highlighted in green for ease of verification.

Comment 1: “The writing is fine as it is. No changes needed. The authors’ recommendations are appropriate. While no changes are needed, it seems that an underlying problem of the UWES instrument may be Schaufeli et al.’s (2002) assumption that engagement is the opposite of burnout.”

Response: We thank the reviewer for the positive evaluation. We agree with the conceptual clarification regarding the distinction between engagement and burnout. In the revised manuscript, we strengthened this point by explicitly noting that engagement is treated as a positive well‑being resource rather than the opposite pole of burnout. (Section 2.2 Instrument).

We thank all reviewers for their constructive feedback. All requested revisions have been incorporated into the manuscript (v02.5), with changes highlighted in green and line‑numbered for verification. We believe the revised version is substantially strengthened and clearer in its methodological, conceptual, and applied contributions.

Reviewer 2 Report

Comments and Suggestions for Authors

Summary of the manuscript and key contributions

The manuscript re-evaluates the dimensionality of the UWES (UWES-17 and UWES-9) in a sample of 755 Chilean university students, comparing unidimensional, three-factor, second-order, and bifactor solutions using robust ordinal estimation (WLSMV) and reporting bifactor indices (ECV, PUC, ωh) to assess essential unidimensionality.

Results indicate that the UWES-17 shows evidence of factor collapse and inadmissible solutions (including non-convergence of the second-order model), whereas the UWES-9 yields a more stable “classical” three-factor structure. However, bifactor results suggest that, in both versions, a dominant general factor accounts for most of the common variance (ECV ≈ 0.85; ωh > 0.90), while the subscales retain very limited reliable specific variance, particularly for Absorption.

From an applied standpoint, the contribution is clear: for institutional monitoring in higher education, the authors recommend prioritizing the UWES-9 total score as a more pragmatic and psychometrically defensible approach.

Comments by section

Abstract

Strengths
The abstract is highly transparent and technically clear. It frames the dimensionality debate surrounding the UWES, specifies the sample and competing models, and reports the use of robust ordinal estimation (WLSMV) alongside bifactor indices (e.g., ECV, PUC, ωh) that directly support the interpretation of essential unidimensionality.

Suggestions
- A brief micro-clause (one line) noting that the UWES-17 yields inadmissible/unstable solutions (e.g., non-convergence of the second-order model and/or factor collapse) would help readers understand why the applied recommendation relies more strongly on the bifactor evidence and the UWES-9.

-  The applied statement about “student well-being monitoring” could be slightly calibrated. It would be more precise to state that the UWES-9 total score appears psychometrically defensible for assessing general academic engagement in this sample, while institutional monitoring for well-being would ideally be further supported by criterion-related (predictive/concurrent) validity evidence.

Introduction

Strengths
The Introduction provides a well-grounded rationale for revisiting UWES dimensionality in higher education, and it clearly motivates why clarifying scoring decisions matters for applied use in student contexts.

Suggestions

- The applied framing around student well-being monitoring/screening is relevant but somewhat repetitive and could be condensed. Relatedly, these applied claims should be slightly calibrated: the study provides structural validity evidence for UWES scores, but does not test criterion-related links with well-being, distress, burnout, academic performance, or dropout in this sample; therefore, institutional monitoring implications are best framed as potential applications pending external validity evidence.

- The manuscript should more explicitly clarify the construct transfer from work engagement to academic/student engagement and the exact instrument version administered (UWES vs student-adapted wording/UWES-S). Please state whether item wording was modified for students and how “academic engagement” is conceptualized in the present study.

- The research gap could be sharpened by foregrounding what is distinctive in the Chilean context. Prior work has already examined UWES-9S in undergraduates; the added value here appears to be the simultaneous UWES-17 vs UWES-9 comparison, the use of WLSMV for ordinal data, the deliberate absence of post-hoc modifications, and the reporting of bifactor indices to inform scoring decisions. These contributions should be made explicit earlier.

- The closing of the Introduction would read more compellingly if it foreshadowed the central applied scoring question that structures the paper—namely, under what conditions a total score is warranted versus when subscale reporting would be meaningful—so readers are prepared for the later “3-factor CFA vs essential unidimensionality” message.

Method

Strengths
The Method section is generally clear and makes several sound psychometric choices. The use of WLSMV is appropriate for ordinal UWES items, and the systematic comparison of classical CFA solutions (unidimensional/three-factor/second-order) with bifactor models is well aligned with the research question. Reporting bifactor indices (ECV, PUC, ωh) is also a methodological strength, and the sample size is adequate for the proposed analyses.

Suggestions
- Specify the exact response scale used for the UWES (number of categories and anchors), as this is relevant for the ordinal modeling rationale.

- State explicitly which UWES version was administered (original UWES vs student-adapted wording/UWES-S), whether item wording was modified for a student context, and whether the UWES-9 was administered independently or extracted from the UWES-17 item pool.

- Describe the ordinal invariance sequence (e.g., configural/threshold/metric) and report the criteria used to judge invariance (e.g., ΔCFI/ΔRMSEA thresholds).

-The age range is unusually wide for a university sample (18-70). This does not invalidate the study, but a brief justification and/or a sensitivity check excluding extreme age cases would strengthen confidence in the robustness of the results.

- Clarify how missing data (if any) were handled under WLSMV in lavaan.

-Note explicitly that the second-order model was estimated only for the UWES-17 (and not for the UWES-9), and ensure this is stated consistently across the Abstract/Introduction/Methods.

Results

Strengths
- The Results section strikes an excellent balance between global fit reporting and interpretability. For the UWES-17, the manuscript clearly documents factor collapse (e.g., a vigor–dedication correlation exceeding unity) and the non-convergence of the second-order model, which appropriately precludes a classical subscale interpretation.

-  For the UWES-9, the authors justify the three-factor solution as the best-performing classical CFA representation, supported by stronger stability and convergent validity evidence (CR/AVE).

- The bifactor results are the core strength of the section: the reported indices of essential unidimensionality and the pattern of very limited reliable specific variance (particularly for Absorption) provide a coherent empirical basis for the scoring recommendation.

Suggestions
- Add a bridging sentence to reconcile “best classical model” vs “general factor dominance”. The UWES-9 three-factor model is described as the “best representation” within classical CFA, yet the bifactor results support essential unidimensionality and motivate prioritizing the total score. These statements are compatible, but readers will benefit from an explicit bridge (e.g., the three-factor model is the best classical solution, whereas bifactor indices indicate that most reliable variance is attributable to the general engagement factor).

- Label the UWES-17 three-factor solution as inadmissible/improper. A factor correlation greater than 1 indicates an improper solution; fit indices from such a model should be interpreted with caution and should not be used to support subscale meaning.

- Nuance the interpretation of CR/AVE in a strong general-factor context. High reliability and AVE in an oblique three-factor model may largely reflect general engagement variance rather than reliable dimension-specific variance. A brief note would align this reporting with the paper’s central argument against subscale interpretation.

-  Give greater interpretive prominence to uwes_12. The strongly negative specific loading (λ = −.683) is not merely a limitation note; it may help explain part of the instability observed for UWES-17 and merits brief discussion as an explanatory finding.

Discussion

Strengths
The Discussion translates the psychometric evidence into scoring decisions in a clear and practically useful way, explicitly warning against subscale interpretations when reliable specific variance is minimal (especially Absorption) and articulating implications for applied use in higher education.

Suggestions
-The applied recommendations (e.g., institutional monitoring, screening thresholds, and referral pathways) are valuable, but they should be framed as provisional and contingent on external validation. The present study provides strong structural evidence, yet it does not directly test criterion/predictive validity or sensitivity to change; this nuance is important to keep the applied claims appropriately calibrated.

- The statement that essential unidimensionality is robust “independent of cultural context or sample type” should be softened. The Chilean student evidence aligns with prior studies, but broader generalization would benefit from additional cross-cultural and occupational replications (which the manuscript itself acknowledges as a future direction).

- The interpretation of uwes_12 should remain cautious: a large negative specific loading may reflect item-specific variance or local instability. It would be helpful to briefly report the item content and avoid overinterpreting it without item-level follow-up (e.g., DIF/IRT), which the manuscript appropriately proposes for future work.

Conclusions

Strengths
The Conclusions are concise and well aligned with the bifactor findings, clearly stating that engagement behaves largely as a dominant general construct in this sample and that the UWES-9 total score is the most pragmatic scoring approach for applied use.

Suggestions

The Conclusions should make explicit that these conclusions are sample-specific and grounded primarily in structural validity evidence. The recommendation to prioritize the UWES-9 total score is reasonable; however, claims about institutional “well-being monitoring/screening” should be framed as potential applications pending criterion-related evidence. A slightly more cautious wording would be that the UWES-9 total score appears to be the most psychometrically defensible approach for assessing general academic engagement in this Chilean university sample, while future work should test predictive validity, sensitivity to change, and usefulness for institutional decision-making.

Overall evaluation

This manuscript offers a rigorous and applied psychometric contribution. It uses an appropriate ordinal estimation approach, compares competing structural solutions with an interpretability-oriented rationale, and reports bifactor evidence that directly informs scoring decisions. The paper also makes a clear effort to translate the psychometric findings into practical implications for higher-education monitoring, while remaining largely transparent about model behavior (including inadmissible solutions in UWES-17).

The requested revisions are relatively contained and mainly concern (i) construct coherence (work vs academic engagement and the exact UWES version administered), (ii) methodological traceability (response scale anchors, invariance decision criteria, and missing-data handling), and (iii) editorial clarity in reconciling classical CFA preferences with the bifactor-based total-score recommendation. Addressing these points should strengthen clarity and replicability without altering the core contribution.

Author Response

Dear Editor and Reviewers,
We thank you for your careful evaluation of our manuscript. Below we provide a point‑by‑point response to each reviewer. All requested changes have been incorporated into the revised version (v02.5), where modifications appear highlighted in green for ease of verification.

We thank the reviewer for the detailed and constructive feedback. All suggestions have been addressed. Below we summarize the changes by topic.

Abstract

Comment: Include a brief note on UWES‑17 instability; calibrate applied claims.

Response: Done. We added a clause noting factor collapse and non‑convergence in UWES‑17 and softened the applied recommendation. (Abstract).

Introduction

Comment 1: Calibrate and reduce repetition regarding well‑being monitoring.

Response: Done. We condensed the applied framing and clarified that institutional applications require criterion‑related evidence. (Introduction).

Comment 2: Clarify construct transfer and specify the exact instrument (UWES‑S).

Response: Done. We now explicitly state that the study used the Utrecht Work Engagement Scale for Students (UWES‑S), Spanish version (Valdez‑Bonilla et al., 2011), and explain how academic engagement is conceptualized. (Introduction and Section 2.2 Instrument).

Comment 3: Highlight the distinctive contribution of the Chilean context and methodological design.

Response: Done. A paragraph was added emphasizing the simultaneous UWES‑17 vs UWES‑9 comparison, WLSMV estimation, absence of post‑hoc modifications, and bifactor indices. (Introduction).

Comment 4: Foreshadow the central scoring question.

Response: Done. (End of Introduction).

Method

Comment 1: Specify response scale.

Response: Done (0–6 Likert frequency scale). (Section 2.2 Instrument).

Comment 2: Clarify UWES‑S version and derivation of UWES‑9S.

Response: Done. (Section 2.2 Instrument).

Comment 3: Describe invariance sequence and criteria.

Response: Done (ΔCFI ≤ .010; ΔRMSEA ≤ .015). (Section 2.4 Data analysis)

Comment 4: Justify wide age range.

Response: Done, including a note on non‑traditional students and sensitivity checks. (Section 2.3 Procedure).

Comment 5: Clarify missing data handling.

Response: Done (no missing responses observed). (Section 2.4 Data analysis).

Comment 6: Clarify that second‑order model was estimated only for UWES‑17.

Response: Done. (Section 3.2 Results).

Results

Comment 1: Add bridging sentence between classical CFA and bifactor dominance.

Response: Done. (Section 3.2).

Comment 2: Label UWES‑17 three‑factor model as inadmissible.

Response: Done. (Section 3.2).

Comment 3: Nuance CR/AVE interpretation.

Response: Done. (Section 3.3).

Comment 4: Give more prominence to item uwes_12.

Response: Done. (Section 3.3).

Discussion

Comment 1: Calibrate applied recommendations.

Response: Done. (Discussion).

Comment 2: Soften generalization claims.

Response: Done. (Discussion).

Comment 3: Maintain caution regarding uwes_12.

Response: Done. (Discussion).

Conclusions

Comment: Make conclusions explicitly sample‑specific and structurally grounded.

Response: Done. (Conclusions)

We thank all reviewers for their constructive feedback. All requested revisions have been incorporated into the manuscript (v02.5), with changes highlighted in green and line‑numbered for verification. We believe the revised version is substantially strengthened and clearer in its methodological, conceptual, and applied contributions.

Reviewer 3 Report

Comments and Suggestions for Authors

1. Line 59: to correct Carmona Halty et al. 2019. Correct is ”Carmona-Halty”

References: Carmona-Halty, M.A., Schaufeli, W.B., & Salanova, M. (2019). The Utrecht Work Engagement Scale for 703 students (UWES–9S): Factorial validity, reliability, and measurement invariance in a Chilean sample of undergraduate university 704 students. Frontiers in Psychology, 10, 1017. https://doi.org/10.3389/fpsyg.2019.01017

 

2. Line 742: (Rodríguez González et al., 2023) Rodríguez-González, R., Martínez-Santos, A.E., Vicho De La Fuente, N., López-Pérez, M.E., Fer-742 nández-De-La-Iglesia, J.C. (2023). Identifying engagement and associated factors in nursing students: An exploratory study. Journal 743 of Professional Nursing, 48, 77-83. https://doi.org/10.1016/j.profnurs.2023.06.003

To be corrected in parentheses: Rodríguez-González

Author Response

Dear Editor and Reviewers,
We thank you for your careful evaluation of our manuscript. Below we provide a point‑by‑point response to each reviewer. All requested changes have been incorporated into the revised version (v02.5), where modifications appear highlighted in green for ease of verification.

Comment 1: Correct “Carmona Halty” → “Carmona‑Halty”.

Response: Corrected. Introduction (now corrected in v02.5).

Comment 2: Correct “Rodríguez González” → “Rodríguez‑González”.

Response: Corrected. References and in‑text citation (now corrected in v02.5).

We thank all reviewers for their constructive feedback. All requested revisions have been incorporated into the manuscript (v02.5), with changes highlighted in green and line‑numbered for verification. We believe the revised version is substantially strengthened and clearer in its methodological, conceptual, and applied contributions.