Development and Validation of AI Help-Seeking Behavior Scale Among Undergraduate University Students
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsPage 3: Need more information regarding recruitment of the students – were they fluent in english or arabian; they were age more than 18 years, etc..?
Limitations in methods: One of the major limitations of this study is that the students are from a single university. Does this university reflect the university population of the United Arab Emirates? Are all of them residents of the United Arab Emirates?
Another limitation is that most of the participating students are from the humanities and sciences; could this have influenced the results in any way?
Methods: As this is a methodological study, the authors conducted exploratory factor analysis followed by confirmatory factor analysis.
However, the section on reliability analysis mentioned in section 2.9, regarding internal consistency, should follow the section on exploratory factor analysis (EFA). In the psychometric analysis of instruments following EFA, reliability must be assessed. If a measurement instrument is not reliable, then it cannot be valid. This should be revised to include a proper methodology section.
Author Response
Dear Editor,
Dr. Joy Li,
We would like to express our gratitude to you and the reviewers for their valuable comments and constructive feedback. Please find below our detailed point-by-point response to the reviewers.
------------------------------------------------------------------------------------------------------------
Reviewer comments
Reviewer #1:
Comment 1: Page 3: Need more information regarding recruitment of the students – were they fluent in english or arabian; they were age more than 18 years, etc..?
Response 1: Thank you for your thoughtful and positive feedback on our manuscript. We have revised in page 3 to provide details regarding participant recruitment and eligibility criteria. We clarified that participants were university students aged 18 years or older and were proficient in either Arabic or English, as the questionnaire was available in both languages.
Comment 2: Limitations in methods: One of the major limitations of this study is that the students are from a single university. Does this university reflect the university population of the United Arab Emirates? Are all of them residents of the United Arab Emirates?
Response 2: Thank you for this valuable comment. We acknowledge that recruiting participants from a single university may limit the generalizability of the findings in our limitation section. Please see our revised manuscript. However, Ajman University at UAE enrolls students from a wide range of nationalities and cultural backgrounds, reflecting the multicultural nature of higher education in the UAE. Thus, the study sample included participants from diverse demographic and national backgrounds rather than a single homogeneous population. We have also clarified that the study was conducted among students enrolled at a university located in the UAE, representing both UAE nationals and expatriate residents from various countries.
Comment 3: Another limitation is that most of the participating students are from the humanities and sciences; could this have influenced the results in any way?
Response 3: Thank you for this important comment. We do not believe this influenced the psychometric findings of the study. The aim was to evaluate the validity and reliability of the AI-HSB scale rather than to compare levels of AI-HSB across academic disciplines. Furthermore, participants were recruited from multiple colleges within the university, and the percentage representing the humanities and sciences students was 26.4% of the total sample, and if so, maybe due to accessibility and availability during the data collection period. Nevertheless, we acknowledge that future studies should examine the scale across more balanced disciplinary groups to further support its generalizability.
Comment 4: Methods: As this is a methodological study, the authors conducted exploratory factor analysis followed by confirmatory factor analysis. However, the section on reliability analysis mentioned in section 2.9, regarding internal consistency, should follow the section on exploratory factor analysis (EFA). In the psychometric analysis of instruments following EFA, reliability must be assessed. If a measurement instrument is not reliable, then it cannot be valid. This should be revised to include a proper methodology section.
Response 4: Thank you for this thoughtful comment. We agree that reliability is an essential component of psychometric evaluation. However, in the present study, the sample was randomly divided into two independent subsamples, with EFA conducted on the first subsample and CFA conducted on the second subsample. Following current psychometric recommendations, reliability was assessed using the CFA sample because it reflects the final confirmed factor structure of the instrument. Reliability is commonly evaluated after establishing an acceptable measurement model through CFA (Hair et al., 2019; Kline, 2016). Therefore, we retained the reliability section after the CFA section to reflect the sequence of analyses conducted on the independent validation sample.
We sincerely hope that the revised version of our manuscript meets the standards of your esteemed journal and is now deemed suitable for publication.
*Corresponding author:
Othman A. Alfuqaha, PhD
Department of Psychology, College of Humanities and Sciences, Ajman University.
Ajman, United Arab Emirates.
Tel: +971559011771
E-mail: o.alfuqaha@ajman.ac.ae
Reviewer 2 Report
Comments and Suggestions for AuthorsThank you for the opportunity to review this study entitled “Development and Validation of AI Help-Seeking Behavior Scale Among Undergraduate University Students” (ejihpe-4345575).
The study aimed to develop and validate a bilingual Artificial Intelligence Help-Seeking Behavior Scale (AI-HSB) among undergraduate university students. A sample of 416 students from Ajman University in the United Arab Emirates was involved in the research.
The manuscript is generally well structured and addresses a timely and potentially relevant topic. The development of a measure assessing students’ use of AI tools for psychological help-seeking is an interesting contribution, particularly given the growing integration of AI-based systems in higher education and mental health-related contexts. The bilingual development of the scale is also a strength of the study. At the same time, some issues should be addressed before the paper is suitable for publication.
- Keywords: Please list the keywords in alphabetical order.
- Keywords: “Ajman University Students” may be too specific as a keyword; the authors could consider using a broader term such as “undergraduate students” or “university students.”
- Introduction: The rationale for conceptualizing AI-based help-seeking as a construct distinct from traditional help-seeking should be strengthened.
- Introduction: The manuscript sometimes moves between academic help-seeking, psychological help-seeking, emotional coping, and AI use in general. The authors should define more precisely whether the scale is intended to measure psychological help-seeking through AI, general reliance on AI, attitudes toward AI mental health tools, or a combination of these constructs.
- Method: The authors clarified where recruitment took place, but they should also explain how participants were recruited and whether any compensation or incentive was offered for participation.
- Method/Item Generation: The item generation procedure should be described more transparently. It would be useful to specify how many experts contributed to the initial item development, how disagreements were resolved, and whether any initial items were removed or revised before arriving at the final 13-item version.
- Method/Scoring: The proposed cut-off scores for low, moderate, and high reliance on AI-HSB require further justification. The categories appear to be based on an arbitrary division of the Likert-scale mean range rather than on empirical validation, normative data, criterion-related evidence, or clinically meaningful thresholds. The authors should either provide a clear rationale for these cut-offs or describe them as descriptive score categories rather than validated cut-off points.
- Method: The statistical analyses should be described in a separate section entitled “Data Analysis.”
- Results: The second factor includes only three items. Although this is not necessarily problematic, the authors should discuss the potential limitations of such a short factor, especially considering the Cronbach’s alpha value of 0.67.
- Limitations: The authors report convergent and discriminant validity using AVE and the square root of AVE. However, criterion validity is not examined. The authors should acknowledge this more explicitly and avoid presenting the scale as fully validated without external validation against related constructs, such as traditional help-seeking attitudes, AI attitudes, psychological distress, or digital literacy.
Best wishes
Author Response
Dear Editor,
Dr. Joy Li,
We would like to express our gratitude to you and the reviewers for their valuable comments and constructive feedback. Please find below our detailed point-by-point response to the reviewers.
------------------------------------------------------------------------------------------------------------
Reviewer # 2:
Comment 1: Thank you for the opportunity to review this study entitled “Development and Validation of AI Help-Seeking Behavior Scale Among Undergraduate University Students” (ejihpe-4345575).
The study aimed to develop and validate a bilingual Artificial Intelligence Help-Seeking Behavior Scale (AI-HSB) among undergraduate university students. A sample of 416 students from Ajman University in the United Arab Emirates was involved in the research.
The manuscript is generally well structured and addresses a timely and potentially relevant topic. The development of a measure assessing students’ use of AI tools for psychological help-seeking is an interesting contribution, particularly given the growing integration of AI-based systems in higher education and mental health-related contexts. The bilingual development of the scale is also a strength of the study. At the same time, some issues should be addressed before the paper is suitable for publication.
Response 1: We sincerely thank the reviewer for the careful evaluation of our manuscript and for the positive comments regarding the relevance, timeliness, and potential contribution of this study. We are pleased that the reviewer recognized the importance of developing a bilingual measure to assess AI-HSB among university students.
Comment 2: Keywords: Please list the keywords in alphabetical order.
Keywords: “Ajman University Students” may be too specific as a keyword; the authors could consider using a broader term such as “undergraduate students” or “university students.”
Response 2: Thank you for this helpful suggestion. We have revised the keywords section accordingly.
Comment 3: Introduction: The rationale for conceptualizing AI-based help-seeking as a construct distinct from traditional help-seeking should be strengthened.
Introduction: The manuscript sometimes moves between academic help-seeking, psychological help-seeking, emotional coping, and AI use in general. The authors should define more precisely whether the scale is intended to measure psychological help-seeking through AI, general reliance on AI, attitudes toward AI mental health tools, or a combination of these constructs.
Response 3: Thank you for this important observation. We agree that the distinction between AI-based help-seeking and related constructs should be clarified. The purpose of the AI-HSB Scale is not to assess general AI use, academic help-seeking, or overall reliance on AI technologies. Rather, it is intended to measure psychological help-seeking behavior through AI-based systems, including individuals' willingness to seek psychological support from AI tools, their perceived usefulness of such tools, trust in AI-based support, and concerns regarding their use for mental health-related issues. We have provided a conceptual definition of AI-HSB and distinguished it from traditional help-seeking behaviors, general AI use, and attitudes toward technology. Please see our revised manuscript under the Introduction Section.
Comment 4: Method: The authors clarified where recruitment took place, but they should also explain how participants were recruited and whether any compensation or incentive was offered for participation.
Response 4: Thank you for this comment. Additional details regarding participant recruitment was added. Participants were recruited through announcements distributed via university communication channels and were invited to complete an online questionnaire voluntarily. Participation was entirely voluntary, and no financial compensation, academic credit, or other incentives were offered for participation. Please see our revised manuscript under the Method Section.
Comment 5: Method/Item Generation: The item generation procedure should be described more transparently. It would be useful to specify how many experts contributed to the initial item development, how disagreements were resolved, and whether any initial items were removed or revised before arriving at the final 13-item version.
Response 5: Thank you for this valuable suggestion. We have expanded the description of the item-generation process in the Methods section. We clarified that the initial item pool was developed based on a comprehensive review of the literature on help-seeking behavior, mental health support, and the emerging use of AI in psychological services. The initial items were then evaluated by an expert panel consisting of specialists in psychology, counseling, AI, and mental health (8 arbitrators). Experts assessed the relevance, clarity, and appropriateness of each item, and their feedback was used to refine item wording and improve clarity, and content validity. Discrepancies in expert opinions were resolved through discussion and consensus among the research team. The revised process is now described in greater detail in the revised manuscript.
Comment 6: Method/Scoring: The proposed cut-off scores for low, moderate, and high reliance on AI-HSB require further justification. The categories appear to be based on an arbitrary division of the Likert-scale mean range rather than on empirical validation, normative data, criterion-related evidence, or clinically meaningful thresholds. The authors should either provide a clear rationale for these cut-offs or describe them as descriptive score categories rather than validated cut-off points.
Response 6: Thank you for this important observation. We agree that the proposed score categories were not established through empirical validation, normative data, or criterion-related analyses. Our intention was not to establish meaningful cut-off scores but rather to provide a descriptive interpretation of the mean scores obtained from the four-point Likert scale. Thus, we omitted the cut-off point and replace it with descriptive score.
Comment 7: Method: The statistical analyses should be described in a separate section entitled “Data Analysis.”
Response 7: Thank you for this suggestion. We agree that presenting the statistical procedures in a dedicated section improves the clarity and organization of the manuscript. Accordingly, we have added a separate section entitled “Data Analysis” as of your recommendation.
Comment 8: Results: The second factor includes only three items. Although this is not necessarily problematic, the authors should discuss the potential limitations of such a short factor, especially considering the Cronbach’s alpha value of 0.67.
Response 8: Thank you for this important observation. We acknowledge that the second factor consists of only three items and demonstrated a Cronbach's alpha coefficient of 0.67. However, it is well established that Cronbach's alpha is influenced by the number of items in a scale, and shorter subscales often yield lower reliability estimates despite adequate item homogeneity. Given that this factor contains only three items, the observed reliability coefficient falls within the acceptable range for newly developed scales and exploratory research. Furthermore, all three items demonstrated satisfactory factor loadings and contributed meaningfully to the conceptual domain of concerns regarding AI-HSB. Nevertheless, we have added a statement to the Discussion section acknowledging this limitation and recommending that future research further examine and potentially expand this dimension.
Comment 9: Limitations: The authors report convergent and discriminant validity using AVE and the square root of AVE. However, criterion validity is not examined. The authors should acknowledge this more explicitly and avoid presenting the scale as fully validated without external validation against related constructs, such as traditional help-seeking attitudes, AI attitudes, psychological distress, or digital literacy.
Response 9: Thank you for this important observation. We agree that criterion-related validity was not examined in the present study. The main aim was to establish the initial psychometric properties of the AI-HSB Scale, including content validity, construct validity, convergent validity, discriminant validity, and internal consistency reliability. We acknowledge that external validation against theoretically related constructs, such as traditional help-seeking attitudes, attitudes toward AI, psychological distress, digital literacy, or actual AI use behaviors, would provide additional evidence regarding the scale's validity. Accordingly, we have revised the Limitations section to explicitly acknowledge the absence of criterion-related validity testing and recommend that future studies examine these relationships to further strengthen the validation of the AI-HSB Scale.
We sincerely hope that the revised version of our manuscript meets the standards of your esteemed journal and is now deemed suitable for publication.
*Corresponding author:
Othman A. Alfuqaha, PhD
Department of Psychology, College of Humanities and Sciences, Ajman University.
Ajman, United Arab Emirates.
Tel: +971559011771
E-mail: o.alfuqaha@ajman.ac.ae
