Acceptance of Artificial Intelligence (ChatGPT) Among Trainee Teachers in Higher Education
Cristina Maria Ribeiro Caridade
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsI appreciate the opportunity to review this paper. The topic is timely and potentially impactful within its field. However, the manuscript would benefit from further development, particularly with respect to clearly articulating the research questions. Establishing explicit research questions would help guide the reader through the paper and strengthen the argument being presented.
In the following section, I provide some suggestions and pose several questions that may help clarify certain aspects and enhance the overall quality of the work.
- Research questions are missing. I suggest adding them after stating the research aim at the end of the introduction.
See line 37: you do not need to introduce the abbreviation “AI” at this point again as you already have introduced it in line 23.
- “1.1 Chat GPT” I suggest to be consistent with the wording ChatGPT (see line 48, p. 2) and use no space here.
- I am wondering in which previous studies the used questionnaire was validated. Please add the citations (see line 115, p. 3).
- Please provide more information regarding the translation of the questionnaire. Did you translate it yourself? Did you use a back-translation approach?
- See line 134: Why did you adjust the scale?
- Regarding Figure 2: Please revise the axis labling since they are overlapping.
- “gAI” (see line 224, p. 8) please introduce this abbreviation properly and use it consistently (see “gIA” in line 244).
- Please describe Figure 1 in the text. I am confused by the axis labling (why do you distinguish between “AI”, “gAI” & “AI for text”?).
- Regarding the analysis: Why didn't you discuss the standard deviation of the items?
- See Table 3 & 4: Where do these free text answers come from? I think the questionnaire does not include questions with free text answers. How did you analyse these free text answers?
- Do you have any plans for further research?
Comments on the Quality of English LanguageI think this paper should be proofread. Here are some suggestions for improvement:
- OpenIA should be OpenAI (see line 50, p. 2)
- A space between “.In” is missing (see line 71, p. 2)
- “High Education students” (see line 107, p. 2) sounds strange to me.
- “anomymous” (see line 123, p. 2) should be “anonymous”
- “generative IA” should be “generative AI” (see line 192, p. 7)
Author Response
Thank you very much for taking the time to review this manuscript. We are specially grateful for the constructive tone and the very precise suggestions that will, for sure, improve the overall quality of the paper. All the comments referring to spelling or wording have been directly corrected. Please find the detailed responses below to the rest of the comments and the corresponding revisions and corrections in track changes in the re-submitted files.
Comments 1: The manuscript would benefit from further development, particularly with respect to clearly articulating the research questions. Establishing explicit research questions would help guide the reader through the paper and strengthen the argument being presented […] Research questions are missing. I suggest adding them after stating the research aim at the end of the introduction.
Response 1: Thank you for pointing this out. We agree with this comment. Therefore, we have introduced the research aim and the research questions at the end of the introduction (lines 139 – 144).
Comments 2: I am wondering in which previous studies the used questionnaire was validated. Please add the citations (see line 115, p. 3).
Response 2: Thank you for mentioning it. We didn’t deem it necessary to cite again the studies to which we had just referred in the introduction, but we agree that it’s much more transparent to include the references. We’ve added two references ([11, 20]) to line 138.
Comments 3: Please provide more information regarding the translation of the questionnaire. Did you translate it yourself? Did you use a back-translation approach?
Response 3: We translated the questionnaire ourselves. Then, to ensure the fidelity of the translation, we got the questionnaire translated back from both vernacular languages into English. There was a good correspondence. We’ve mentioned it in the text.
Comments 4: Why did you adjust the scale?
Response 3: Probably there is a problem with the way of phrasing the idea. We just used a 7-point Likert scale, as in the work of Strzelski [11], with no further adjustments. We’ve provided an alternative phrasing (line 136).
Comments 4: Regarding Figure 2: Please revise the axis labling since they are overlapping.
Response 4: Thanks for noticing. We’ve modified the axis to avoid the overlapping. It has not been recorded as a change.
Comments 5: Please describe Figure 1 in the text. I am confused by the axis labling (why do you distinguish between “AI”, “gAI” & “AI for text”?).
Response 5: Thanks for noticing, we had removed unadvertedly the reference to Figure 1 in the text, but following your comment we’ve restored it. See line 190.
As these items were intended to test the practical and conceptual knowledge of AI among the respondents of the questionnaire, we preferred to keep these three concepts separated. Although there are plenty of references to AI in the user’s daily context, they might not be conscious that the concept is wider and goes beyond generative AI tools, such as ChatGPT. Or, in other words, they might not be aware that ChatGPT represents a specific type of AI; this that is able to generate new content in natural language. Moreover, we were interested in the familiarity of the users not only in AI tools for producing text (such as ChatGPT that is, most probably, the single best-known tool), but also with not so popular tools for producing artifacts in other formats.
Comment 6: Regarding the analysis: Why didn't you discuss the standard deviation of the items?
Response 6: For the sake of clarity and considering the high number of questions involved and domains being analyzed, we just focused on the mean values. However, we agree with this comment, and have included a couple of sentences to discuss the standard deviation (lines 174 – 175 (results) and 220 – 221 (discussion)).
Comment 7: See Table 3 & 4: Where do these free text answers come from? I think the questionnaire does not include questions with free text answers. How did you analyse these free text answers?
Response 7: There was a mention to the free text answers in the methodology, but so short that apparently it went unnoticed. To address this comment and also comment #5, we’ve introduced a new table (Table 2) containing the statements.
The free text answers were categorized using an inductive approach; i.e., emerging themes were identified in the answers provided, and grouped into coherent categories. Both researchers revised and categorized jointly the answers. It has been explained in lines 174 – 178.
Comment 8: Do you have any plans for further research?
Response 8: Thank your for your comment. Plans for further research have been added at the end of the discussion (lines 301 – 305).
Comments on the quality of the English:
The paper has been proofread, and the suggested changes implemented.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe paper describe a study that analyses the knowledge, acceptance and perception of the use of ChatGPT among Secondary Teacheing master's degree in 2 Spain universities, by using a questionnaire besed on the Unified Theory of Acceptance and Use of Technology (UTAUT2) model.
The introduction of the paper describes the objectives of the study based on references from articles on the subject in question. Perhaps it would be interesting to research the state of the art, especially the use of ChatGPT in the context of higher education education.
In materials and methods, they are the participants and the questionnaire carried out through the 7 dimensions defined in UTAUT2. The 30 questions were not described, nor was the questionnaire attached. Only the number of questions in each of the 7 dimensions is mentioned in table 2.
In the results, a complete review is needed. From the text, comments and explanations as well as the figures and tables presented. There is a lot of confusion with definitions, for example Facilitating Conditions is sometimes defined with CF and sometimes as FC. The results of the study are not presented clearly and are difficult to understand. Acronyms are used in the figures that are not defined in the text, other acronyms are interchanged and others are not defined. It is difficult to understand what the authors mean by PE1, PE2, PE3, PE4 and PE5. I think these are 5 questions in the PE dimension, but none of this is said or explained. Just as it is not mentioned what these questions are, which seems strange to me since the study deals with the analysis of the questionnaire.
Here too, percentages are given to open-ended responses. How are these percentages calculated? How were they quantified?
In the discussion and conclusions, I think that after changing the results chapter, this chapter will also have to be changed.
-Removed for peer-review-
Author Response
Thank you very much for taking the time to review this manuscript and for your comments that will, for sure, improve the overall quality of the paper. Please find below the detailed responses to your comments and the corresponding revisions; the requested changes and corrections have been done with track changes in the re-submitted files.
Comments 1: The introduction of the paper describes the objectives of the study based on references from articles on the subject in question. Perhaps it would be interesting to research the state of the art, especially the use of ChatGPT in the context of higher education education.
Response 1: The whole section 1.2 ChatGPT in education revolves around the use of ChatGPT in education, but mostly in Higher Education, where the vast majority of the studies have been conducted so far. The section provides some insights on the penetration of the tool, potential benefits and drawbacks, as based on works published in the last years or months. However, we have taken into account the suggestion of the reviewer, and have developed further this section, to provide a better description of the state of the art.
Comments 2: In materials and methods, they are the participants and the questionnaire carried out through the 7 dimensions defined in UTAUT2. The 30 questions were not described, nor was the questionnaire attached. Only the number of questions in each of the 7 dimensions is mentioned in table 2.
Response 2: The 30 questions were not included originally because they are the same in the original sources ([20] as modified by [11]), but translated into Basque and Spanish. Of course, there is no problem to make both versions of the questionnaire available as supplementary material, if deemed necessary by the reviewer or the editor. (See also comment 3 for the definition of each item).
Additionally, we have included further details on the 5 questions on knowledge of the technology and on the three free text questions (Table 2; line 149).
Comments 3: In the results, a complete review is needed. From the text, comments and explanations as well as the figures and tables presented. There is a lot of confusion with definitions, for example Facilitating Conditions is sometimes defined with CF and sometimes as FC. The results of the study are not presented clearly and are difficult to understand. Acronyms are used in the figures that are not defined in the text, other acronyms are interchanged and others are not defined. It is difficult to understand what the authors mean by PE1, PE2, PE3, PE4 and PE5. I think these are 5 questions in the PE dimension, but none of this is said or explained. Just as it is not mentioned what these questions are, which seems strange to me since the study deals with the analysis of the questionnaire.
Response 3: The reversed order of some of the acronyms (CF/FC) comes from the translation, where we apparently overlooked some of them, that remained in the original order that reflects the Spanish wording. We have double-checked all the initials, changed two instances of reversed or unconsistent acronyms (l. 175 and 183) and apologize for this inconvenience.
For the sake of clarity, we have replaced the code assigned to each of the items, and composed by the acronym summarizing the dimension + number of the item), by a short description of the content of the question (see Table 3).
We would be grateful if the reviewer could explain which else needs to be revised in the text, comments and explanations.
Comment 4: Here too, percentages are given to open-ended responses. How are these percentages calculated? How were they quantified?
Response 4: The free text answers were categorized using an inductive approach; i.e., emerging themes were identified in the answers provided, and grouped into coherent categories. Both researchers revised and categorized jointly the answers. It has been explained in lines 152 – 156.
Comment 5: In the discussion and conclusions, I think that after changing the results chapter, this chapter will also have to be changed.
Response 5: To the best of our understanding, all the requested changes had to do with formal questions (reversed acronyms and lack of precision in the definition of each of the items). However, we don’t feel that these have radically changed the content of the results, so we don’t know what to modify in the discussion section. As stated before, if the reviewer could provide more information on required changes, we would be more than happy to implement them.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe work is written in a scientific manner and within the framework of an emerging theme.
It is rather limited in its objectives. It only has one general objective. I think it would be important to break it down into specific objectives and respond to these. It would give the work a better dimension.
The graph in figure 2 doesn't make sense as a line graph, but as a bar graph. This is because the Likert scale categories are ordinal and the bar graph makes it easier to compare the frequencies or averages of each category in a clear and visually accessible way. Line graph is more suitable for continuous data or to show trends over time.
- The conclusion is very long and reads like a literature review. I think the authors should move some of this information to the introduction and, in the conclusion, respond more specifically to the objectives.
Author Response
Thank you very much for taking the time to review this manuscript and for your valuable suggestions that will contribute to improve the overall quality of the paper. Please find the detailed responses below to ñyour comments below; the corresponding revisions and corrections have been done with track changes in the re-submitted files.
Comments 1: It is rather limited in its objectives. It only has one general objective. I think it would be important to break it down into specific objectives and respond to these. It would give the work a better dimension.
Response 1: Thank you for pointing this out. We agree with this comment, and we have introduced three research questions at the end of the introduction (lines 117 – 121).
Comments 2: The graph in figure 2 doesn't make sense as a line graph, but as a bar graph. This is because the Likert scale categories are ordinal and the bar graph makes it easier to compare the frequencies or averages of each category in a clear and visually accessible way. Line graph is more suitable for continuous data or to show trends over time.
Response 2: Thank you for discussing this point. We agree with your comment, and we are aware that line graphs are specially suitable to show trends over time. However, it is not to noted that this is not the only use of this type of charts, and that they can be used to depict other types of trends. Just to give an example, Statistics Canada (https://www150.statcan.gc.ca) in their define a line chart as “a visual comparison of how two variables – shown on the x- and y- axes are related or vary with each other […] The y-axis in a line chart usually indicates quantity1 (e.g. dollars, litres) or percentage, while the x-axis often2 measures units of time. As a result, the line chart is often viewed as a time series. Vertical bar charts and line charts share a similar purpose. The vertical bar chart, however, reveals a change in magnitude, whereas the line chart reveals a change in direction. In summary, line charts: show specific values of data well, reveals trends and relationship between data3 and compare trends among groups.”.
The superscripts, from the authors, allow us to point the attention to different details of the definition:
- The data must be quantitative, but not necessarily continuous.
- The use of the adverb “often” reveals that, albeit being the most common use, there would be other alternatives.
- In line with the previous, other relationships than time series can be depicted using line charts.
Indeed, the reason for using a line chart in this case is to attract the attention to the decreasing trend in percentage of answers with classes of frequency use, which is also ordinal. That is to say: most users make infrequent use of ChatGPT, and the higher the frequency, the less the answers in this category.
Nevertherless, and after having explained our reasons to choose a line chart over a barplot, if the referee still considers it necessary, we don’t have any objection to change the type of the chart. In the attached document we have included an for figure 2, is the reviewer thinks that it would be a better option.
Comments 3: The conclusion is very long and reads like a literature review. I think the authors should move some of this information to the introduction and, in the conclusion, respond more specifically to the objectives.
Response 3: Thank you for this comment. The reason why this section is rather long as compared to the typical conclusions section is that it combines discussion and conclusions. To avoid creating misleading expectations, we removed the word “conclusions” from the title, and retained just the part of “discussion”, which we think describe well the content of the section.
Author Response File: 
Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsI would like to express my appreciation for the thorough consideration of my suggestions and the substantial effort made to enhance the quality of the manuscript. The revisions have significantly improved the clarity, depth, and overall presentation of the paper. I am satisfied with the changes implemented and have no further suggestions for improvement.
Author Response
Dear reviewer,
Thanks for your comments on this final version of the manuscript. We're glad we correctly addressed your concerns and agree that the quality of the manuscript has substantially improved.
Thanks for your time,
Reviewer 3 Report
Comments and Suggestions for AuthorsI would prefer the graph in Figure 2 to be columns, since the variable is discontinuous, but I recognize that very often, in scientific articles, they are used as if the variable were continuous. I leave it to the editors for consideration.
Author Response
Dear reviewer,
Thanks for your answer and your last recommendation on the figure. Indeed, we just tried to explain why we had chosen a line graph, but we would not object to using the columns. We'll submit both versions and leave it to the editor's decision.
Sincerely yours,
