Review Reports - Development and Initial Validation of the Multidimensional Employee Engagement Instrument (MEEI): A Mixed-Methods Scale Development Study

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The rationale for the study is not as strong as claimed, as the current literature has been overlooked, and the assertion that a new scale is needed remains weak.
The sample in the qualitative phase of the study exhibits significant imbalance. There are substantial discrepancies between the theoretical structure and the empirical findings in the factor analysis; the findings appear contradictory.
Since the study's rationale is inconsistent and the analysis contains errors, the policy recommendations also lose their significance.

Author Response

Rationale for the study is not as strong as claimed, as the current literature has been overlooked, and the assertion that a new scale is needed remains weak.

Response: We examined the literature on employee engagement from 1990 to 2024, from which we created the first set of qualitative interview questions. We examined the literature about Generation Z, Generation Y, and Generation X from 2009–2024 and added one additional interview question.

One of our research concerns was whether Generation Z and/or Generation Y’s understanding of employee engagement differed from Generation X. We determined this by comparing responses among Generation Z, Generation Y, and Generation X and did not find differences.

Following the qualitative study analyses and the qualitative theming, we developed scale development items that followed the five themes from the person-environment fit theories of person-job, person-colleague, person-supervisor, person-community, and person-organization fit. In addition, we developed scale development items from the employee wellbeing theme.

In the scale development analysis, we found three factors that connected to three of the six qualitative themes. Thus, we showed alignment from final factors back to the literature.

The sample in the qualitative phase of the study exhibits significant imbalance. There are substantial discrepancies between the theoretical structure and the empirical findings in the factor analysis; the findings appear contradictory.

Response: Our response to the prior concern answers this concern as well. There is alignment from the initial literature review to the qualitative interview questions to the themes to the scale development items and to the final factors.

Since the study’s rationale is inconsistent and the analysis contains errors, the policy recommendations also lose their significance.

Response: Based on Reviewer 3’s recommendation to reanalyze our data, we used Principal Axis Factoring and conducted a new factor analysis. The revised manuscript shows the results, which are quite different from what we had using principal component analysis.

The new factors showed the same connection to our literature results and the qualitative themes. However, two of the six factors from the revised study are collections of items from three of the six themes. This led us to more recent literature on the focus of the two themes and found support for two differences in Generation Z and Generation Y’s reasons for employee engagement. However, there was no change in the previous finding of no differences between Generation Z and Generation Y’s understanding of what employee engagement is compared to Generation X. One of the mixed theme factors focused on values/passion[purpose (all are about higher-order values), and the other mixed theme focused on flexibility. We added studies from the literature supporting the findings. We believe this strengthens the value of the study by showing the importance of higher-order values and flexibility in employee engagement.

Reviewer 2 Report

Comments and Suggestions for Authors

This paper offers a thoughtful and systematic effort to develop a new employee engagement measurement instrument using a mixed-methods approach grounded in an established scale development framework. It addresses an important topic, and the resulting instrument demonstrates promising preliminary psychometric properties. But there are some improvements that need attention, as outlined below:

You should strengthen your justification for developing a new engagement scale. The argument is interesting, as the paper argues that existing instruments may not adequately measure engagement among Generation Z employees. However, the qualitative findings ultimately indicate no meaningful differences across generations. You should reconcile this apparent inconsistency and more clearly explain why a new instrument remains necessary despite this finding.
It is important that you also expand the discussion of the transition from six themes to three factors. The qualitative phase identifies six conceptual themes, whereas the exploratory factor analysis retains only three factors. Your paper should provide a stronger theoretical explanation of why Person-Job Fit, Person-Group/Colleague Fit, and Personal Wellbeing were eliminated or merged into the final factor structure.
I strongly encourage you to provide better transparency regarding the qualitative coding process. Your paper does a good job of explaining that AI-generated themes produced by Intellectus Qualitative were replaced by researcher-defined themes. But the criteria used for this reassignment, any independent coding or verification procedures, and measures of coding trustworthiness should be described in greater detail.
I strongly suggest you reconsider the use of PCA. Since the objective is latent construct development rather than data reduction, you should either justify the use of PCA or consider reporting results from a common-factor Exploratory Factor Analysis, which is more consistent with contemporary psychometric recommendations.
Also, you should address the discriminant validity findings more thoroughly. In other words, the paper acknowledges that the proposed engagement scales were significantly correlated with the measure selected for discriminant validity. But rather than just offering a speculative explanation, you should discuss the implications of this finding for the construct validity of the new instrument.
You should consider strengthening the evidence supporting generational conclusions. My concern is that the conclusion that employee engagement does not differ across generations is based on a relatively limited qualitative sample, particularly for Generation Z. You should either temper this conclusion or provide additional quantitative analyses demonstrating measurement equivalence across generations.
Make sure to reduce repetition throughout the literature review. In many sections of the paper, you repeatedly conclude with similar statements regarding expected in vivo codes and interview questions. I'd encourage you to consolidate these discussions, which would substantially improve readability without sacrificing methodological transparency.
Consider also condensing the presentation of qualitative findings. For instance, Table 3.2 and the accompanying narrative could be summarized more effectively by emphasizing the principal themes and representative quotations rather than presenting extensive lists of AI-generated codes.
You should expand the discussion of your study limitations and future validation. For instance, your paper should more explicitly acknowledge that this represents an initial validation study using a single U.S.-based sample and recommend future validation across different industries, organizational contexts, cultures, and independent samples.
Finally, I'd strongly suggest you perform a final editorial revision. Make sure to conduct a careful language edit to improve conciseness, eliminate repetitive phrasing, standardize terminology, such as employee engagement versus work engagement, and improve the overall flow of your paper.

Comments on the Quality of English Language

The English language would benefit from professional editing to improve flow and readability.

Author Response

Thank you for the opportunity to review this manuscript.

The study addresses an important and timely topic: the measurement of employee engagement across generational cohorts, particularly Generations Z, Y, and X.

The manuscript has several strengths, including its engagement with both classical and contemporary literature on employee engagement, its attempt to reconsider existing measures in light of generational change, and its use of a multi-stage scale development process.

However, I recommend major revisions before the manuscript can be considered for publication.

First, the claim that there are no meaningful differences among Generations Z, Y, and X in their understanding of employee engagement should be reconsidered. This conclusion appears insufficiently supported by the qualitative evidence, particularly because only three (3) Generation Z participants were interviewed. The authors should either collect additional qualitative data from Generation Z participants or substantially soften this claim.

Response: Please see our response to Reviewer 1, who raised a similar concern

Second, the scale development and validation process requires further clarification.

The manuscript indicates that the study began with a broader theoretical framework but ultimately retained three factors: organizational engagement, community engagement, and supervisor engagement.

The rationale for retaining these three factors should be explained more clearly.

Response: The retention of the three factors was based on the factor analysis results. Please see our response to Reviewer 1 about the revised study’s use of Principal Axis Factoring and the new factors that emerged.

The authors should also clarify how the final empirical factor structure relates to the original theoretical framework.

Response: Please see our response to Reviewer 1 where we review the alignment of the literature to the interview questions, to the scale development items, to the factor analysis and new results of using Principal Axis Factoring;

Third, the CFA results need more careful interpretation.

Response: In the revised study, we ran two CFAs, one with all six of the new factors and one with the three factors that explained a minimum of 60% of the variance.

The manuscript states that the models show good fit, but some of the reported fit indices, especially RMSEA values, appear problematic. The authors should revisit the CFA interpretation, correct any inaccurate statements about acceptable fit thresholds, and discuss the limitations of the model fit more transparently.

Response: Please see the revised study and note the fit indices and our comments about each, along with three articles to support our claims/

Fourth, discriminant validity is a serious concern.

The manuscript itself indicates that discriminant validity was not clearly established. In addition, the very high correlation between some of the proposed dimensions suggests that the scales may not be empirically distinct. This issue is especially important because the manuscript proposes a new measurement instrument. The authors should provide stronger evidence that the dimensions capture distinct aspects of employee engagement.

Response: We plan to conduct a follow-up study with one or more discriminant validity variables.

Fifth, the conceptual status of “community engagement” requires further development. Some items appear to measure attachment to or satisfaction with the local community rather than employee engagement itself. While community context may influence engagement, it is not yet clear why it should be treated as a core dimension of employee engagement. The authors should provide a stronger theoretical justification for including this dimension.

Response: In the revised analysis, the factor is person-colleague rather than person-community; thus, this issue is resolved.

Sixth, the role of AI tools in the research process should be clarified. The manuscript notes that part of the operational definition was reworded using Copilot and that the qualitative analysis involved AI-assisted coding. This is not necessarily problematic, but it requires greater transparency. The authors should explain how AI-generated or AI-assisted outputs were checked, revised, and validated by the researchers.

Response: We used Copilot because we wanted to have an operationalized definition that would be in a compatible voice, comfortable with employees with a bachelor’s degree rather than an academic/scholar voice

Finally, the manuscript requires careful editorial revision. There are inconsistencies between the title, abstract, and methods regarding the number of scale development steps and theoretical dimensions. Placeholder references and template remnants should also be removed before further consideration.

Response: We reviewed the revised study for continuity of terminology.

The manuscript has promise and addresses a relevant topic, but it currently requires substantial conceptual, methodological, statistical, and editorial revision.

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for the opportunity to review this manuscript.

The study addresses an important and timely topic: the measurement of employee engagement across generational cohorts, particularly Generations Z, Y, and X.

However, I recommend major revisions before the manuscript can be considered for publication.

Second, the scale development and validation process requires further clarification.

The manuscript indicates that the study began with a broader theoretical framework but ultimately retained three factors: organizational engagement, community engagement, and supervisor engagement.

The rationale for retaining these three factors should be explained more clearly.

The authors should also clarify how the final empirical factor structure relates to the original theoretical framework.

Third, the CFA results need more careful interpretation.

Fourth, discriminant validity is a serious concern.

The manuscript has promise and addresses a relevant topic, but it currently requires substantial conceptual, methodological, statistical, and editorial revision.

All the best,

the reviewer

Comments for author File: Comments.pdf

Author Response

This paper offers a thoughtful and systematic effort to develop a new employee engagement measurement instrument using a mixed-methods approach grounded in an established scale development framework. It addresses an important topic, and the resulting instrument demonstrates promising preliminary psychometric properties. But some improvements need attention, as outlined below:

You should strengthen your justification for developing a new engagement scale. The argument is interesting, as the paper argues that existing instruments may not adequately measure engagement among Generation Z employees. However, the qualitative findings ultimately indicate no meaningful differences across generations. You should reconcile this apparent inconsistency and more clearly explain why a new instrument remains necessary despite this finding.

Response: in the revised study, we added more content to the introduction to show the finding about Generation Z and Generation Y’s similarity to Generation X’s understanding of what employee engagement is; but there is a difference in reasons to be engaged because of the impact of higher-order values and the importance of flexibility in the workplace.

It is important that you also expand the discussion of the transition from six themes to three factors. The qualitative phase identifies six conceptual themes, whereas the exploratory factor analysis retains only three factors. Your paper should provide a stronger theoretical explanation of why Person-Job Fit, Person-Group/Colleague Fit, and Personal Wellbeing were eliminated or merged into the final factor structure.

Response: Please see our responses to Reviewers 1 and 2 about the revised analysis using Principal Axis Factoring and the subsequent six factors.

I strongly encourage you to provide better transparency regarding the qualitative coding process. Your paper does a good job of explaining that AI-generated themes produced by Intellectus Qualitative were replaced by researcher-defined themes. But the criteria used for this reassignment, any independent coding or verification procedures, and measures of coding trustworthiness should be described in greater detail.

Response: We added more content supporting our decision to replace Intellectus Qualitative themes with our own.

I strongly suggest you reconsider the use of PCA. Since the objective is latent construct development rather than data reduction, you should either justify the use of PCA or consider reporting results from a common-factor Exploratory Factor Analysis, which is more consistent with contemporary psychometric recommendations.

Response: Your suggestion led us to use Principal Axis Factoring, which yielded new factors. Please see the revised study.

Also, you should address the discriminant validity findings more thoroughly. In other words, the paper acknowledges that the proposed engagement scales were significantly correlated with the measure selected for discriminant validity. But rather than just offering a speculative explanation, you should discuss the implications of this finding for the construct validity of the new instrument.

Response: We added more content on the correlation of the factors in the revised study. We recommend conducting future studies with different discriminant validity variables.

You should consider strengthening the evidence supporting generational conclusions. My concern is that the conclusion that employee engagement does not differ across generations is based on a relatively limited qualitative sample, particularly for Generation Z. You should either temper this conclusion or provide additional quantitative analyses demonstrating measurement equivalence across generations.

Response: We added additional content about a future qualitative study where we can explore the responses of Generation Z, Generation Y, and Generation X members about similarity among the generations, as well as explore the differences shown in the ANOVA with Bonferroni post-hoc test results.

Make sure to reduce repetition throughout the literature review. In many sections of the paper, you repeatedly conclude with similar statements regarding expected in vivo codes and interview questions. I’d encourage you to consolidate these discussions, which would substantially improve readability without sacrificing methodological transparency.

Response: We made content edits to reduce duplication, but we want to show a priori codes that support alignment of the final factors with the theoretical literature.

Consider also condensing the presentation of qualitative findings. For instance, Table 3.2 and the accompanying narrative could be summarized more effectively by emphasizing the principal themes and representative quotations rather than presenting extensive lists of AI-generated codes.

Response: We edited the tables, but still want to show the codes that led us to see the theoretical connection with the person-environment fit group of theories

You should expand the discussion of your study limitations and future validation. For instance, your paper should more explicitly acknowledge that this represents an initial validation study using a single U.S.-based sample and recommend future validation across different industries, organizational contexts, cultures, and independent samples.

Response: We edited the discussion section to be more explicit.

Finally, I’d strongly suggest you perform a final editorial revision. Make sure to conduct a careful language edit to improve conciseness, eliminate repetitive phrasing, standardize terminology, such as employee engagement versus work engagement, and improve the overall flow of your paper.

Response: We edited the revised manuscript for better flow.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have made substantial revisions. However, there are minor corrections required for acceptance:

*Missing citations in the reference list must be added.

*The repetitive sentence at the end of the Discussion section must be corrected.

Author Response

Reviewer 1:

The authors have made substantial revisions. However, there are minor corrections required for acceptance:
*Missing citations in the reference list must be added.
*The repetitive sentence at the end of the Discussion section must be corrected.
Response:
- We edited the article throughout to reduce redundancy. adding additional support for the use of Intellectus Qualitative and added additional clarification of the two models.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors,

The manuscript is generally understandable; however, numerous long sentences, awkward phrasing, and occasional grammatical inconsistencies remain. Careful language editing by a native English speaker or professional editing service would improve readability and enhance the overall quality of the manuscript.

Comments for author File: Comments.pdf

Author Response

The manuscript is generally understandable; however, numerous long sentences, awkward phrasing, and occasional grammatical inconsistencies remain. Careful language editing by a native English speaker or professional editing service would improve readability and enhance the overall quality of the manuscript.
Reviewer Report (Revised Manuscript)
Thank you for the opportunity to review the revised version of this manuscript. I appreciate the considerable effort the authors have invested in addressing the comments raised during the first round of review. The manuscript has improved substantially in terms of its conceptual framing, methodological presentation, and overall organization. The revised title better reflects the study's actual contribution, the literature review has been updated with recent references, and the rationale for developing a new employee engagement instrument has become much clearer. I also appreciate the shift from a three-factor model to a theoretically grounded multidimensional framework based on person–environment fit. Overall, the manuscript is considerably stronger than its previous version. Nevertheless, several important issues remain unresolved and should be addressed before the manuscript can be considered for publication.
Major comments
- Qualitative evidence and generational comparisons
  - My previous concern regarding the qualitative phase has only been partially addressed. The authors have moderated their conclusions and now refer to "substantial thematic similarity" rather than claiming the absence of generational differences. This is an improvement. However, the qualitative sample still contains only a very limited number of Generation Z participants, making it difficult to draw broader conclusions regarding similarities across generations. The conclusions should therefore remain appropriately cautious and explicitly acknowledge this limitation.
  - Response:
    - In the Qualitative study, we found no difference in understanding of the employee engagement concept across the three generations. In the Quantitative section, we show differences between Generation Z and Y and Baby Boomers. We acknowledge this as a topic for future research.
  - Discriminant validity
    - The authors have expanded the validation analyses; however, discriminant validity remains one of the weaker aspects of the instrument. Given that this manuscript proposes a completely new multidimensional measure, stronger evidence demonstrating that the six dimensions are empirically distinct would strengthen the study considerably. If not already reported, the manuscript would benefit from additional evidence such as the HTMT criterion or a more detailed discussion of discriminant validity.
    - Response:
      - We acknowledge this concern as a reason for future research.
    - CFA interpretation
      - The confirmatory factor analysis has been improved and supported with additional references. Nevertheless, the interpretation of the model fit should remain appropriately conservative. Rather than describing the instrument as fully validated, I recommend emphasizing that this study provides initial validation evidence, consistent with the title of the manuscript.
      - Response:
        
        We changed the title and we note the need for additional research.
      - AI-assisted qualitative analysis
        
        The manuscript still lacks sufficient transparency regarding the use of artificial intelligence during the qualitative phase. While AI-assisted coding is increasingly accepted, the authors should clearly explain how AI-generated coding was verified by the researchers, how disagreements were resolved, and what procedures ensured the credibility of the qualitative analysis.
        
        Response:
        
        We added content about using Intellectus Qualitative for the qualitative analysis.
        
        Minor comments
        
        The abstract has improved considerably but still contains several awkward sentences that should be edited for clarity and readability. Several long sentences throughout the manuscript would benefit from stylistic revision. A few grammatical inconsistencies remain and should be corrected during copy-editing. The item reduction process could be described more explicitly/
        
        Response:
        
        We used a copy editor to improve grammar, flow, and conciseness.

Author Response File: Author Response.pdf