The Role of Artificial Intelligence in Project-Based Learning: Teacher Perceptions and Pedagogical Implications

Round 1

Reviewer 1 Report (Previous Reviewer 3)

Comments and Suggestions for Authors

This work has improved with the additions and wording changes that were introduced in this last version. 

Still, there are two main things that I suggest must be worked further

1) Statistical method. I want to clarify that I've never said that using Likert scale was wrong. I said, and I'm still saying with this version, that a t-test is not the appropriate tool for the analysis given that a Likert scale is not continuous and t-test is made for continuous variables. Instead, I strongly suggest to use a Mann-Whitney U-Test (see https://www.uv.es/prodat/ViSta/vista-frames/help/lecturenotes/lecture09/lec9part4.html). It is the same process (and you will surely obtain similar results), but with the appropriate statistical tool.

2) I notice that you reoriented the Discussion section in order to point to the teachers' perspectives (variable that you are measuring) instead of students' learning gains (variable that you are not measuring). There are two things in this:

a) Check your English. Some things cannot be understood. For example: "In the literature, some of the generally encountered hurdles are highlighted, namely

that the infrastructure and technical training needed to adopt advanced technologies is lacking [21], the first systematic and collective review of the deployment of artificial intel-ligence (AI) and learning analytics on teacher education, concludes that these educational data-mining methods should only be introduced if teachers have already acquired ad-vanced digital competencies, thought to be key to ensure the successful application of AI in the teaching space."

b) Your hypothesis says "H1: The introduction of Artificial Intelligence (AI) into Project-Based Learning (PBL) greatly improves learning personalization, student motivation, and assess-ment efficiency, when compared with traditional PBL without AI." But you cannot prove this because you are not using grades or other things that show learning gains. Neither you are using variables to prove motivation or personalization. The hypothesis must be rewritten to reflect the teachers' perception (which is the variable you analyze).

Comments on the Quality of English Language

Everything's fine. Only the discussion section should be revised and rewritten to make good use of English.

Author Response

Thank you for reading our work and for your comments.

1. We completely agree and have incorporated the Mann-Whitney U test. Thank you very much for the suggestion. After analysis, this test provides similar results.
2. Changes have been made to the phrasing of some sentences, as well as to the hypothesis, to better align with what is presented in the manuscript.

Attached is the manuscript with the minor changes highlighted.

Thank you again for the feedback.

Author Response File: Author Response.pdf

Reviewer 2 Report (Previous Reviewer 4)

Comments and Suggestions for Authors

The article under review is a version of the article "Future of Project-Based Learning: Teacher perceptions on AI and PBL", which the authors submitted for review earlier. The new title is much better than the previous version. However, questions about the text of the article remain.

Let's start with an important remark.

Page 4, Position "Student motivation". Questions for this position relate more to students than to teachers. However, the authors have not previously mentioned student surveys. The questionnaire was conducted only among teachers.

Now let's move on to editorial comments.

Page 2, 20th line from the top. Repeat of the text "AI integrated PBL";

Page 3, 10th line from the top, it says "In addition to teacher training". Previously, teacher training was not mentioned.

Page 3 After the description of the second phase, a fairly detailed description of the teacher training process is given. The question arises: when did this training take place? During the second phase of the experiment or earlier?

Page 9, 7th line from the bottom "as I have observed"

Page 10, First paragraph from the top. Here authors need to more clearly separate the results from the article [18] and their article.

Pg. 10, 15th line from the top. The term "deep learning" is more typical for machine learning.

Pg. 10, 9th line from the bottom. Note the end of the sentence.

Author Response

Thank you for reading our work and for your comments.

• The section on student motivation has been clarified. This was addressed by adding explanations to the questions to make it clearer. What we are evaluating is the teachers' perception regarding what they observe and perceive from their students.
• The repetition of "AI integrated PBL" has been removed.
• The reference to teacher training has been clarified.
• The timing of the teacher training has been specified.
• The writing on page 9, 7th line, has been improved.
• The results of the article have been clearly distinguished from those we collected.
• Deep learning" has been added. We agree.
• The wording has been modified.

Attached is the manuscript with the minor changes highlighted.

Thank you again for the feedback.

Author Response File: Author Response.pdf

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors put all observations that I supposed. I agree now with publication this paper.

Author Response

Dear reviewer. Thank you for your comments to improve our work. A cordial greetings.

Reviewer 2 Report

Comments and Suggestions for Authors

The document presents improvements concerning the one that received the previous comments, both in form and content.

Regarding the form, I suggest the following action: Frame both the figures and the tables so that the beginning and end are explicit.

Although the biases in favor of the use of AI in PBL were reduced, some could still prevail in a hidden way within the document due to lack of information, for example:

- It mentions the training offered to teachers, in which they introduce AI tools. It would be very valuable to know more about this training offered (especially for a potential reader who wants to know more about the subject. This well-presented topic could already be a publishable topic in education) along with the suggested tools (there are a large number of them). All this would give a better overview of the context and conditions under which AI was used.

- There are no known results about this preparation, that is, whether the teachers considered that this training was sufficient for the study or if they would have preferred another type of AI tool.

- Although some questions from the questionnaire are mentioned in this version, we still do not have access to all of them, which would be valuable, since, for example, there are no questions related to how the teachers felt working with AI, if there was a greater workload or, on the contrary, it was less. If they agreed with the answers offered by the AI ​​and if they consider that the AI ​​did a better job than they could have done due to their resource limitations, especially time. And so on.

 

- There is also mention about the anxiety of the students and how this is reduced by the real-time response of the AI, however, this aspect is not directly evaluated, and no type of evaluation is made regarding the perceived usefulness by the students of the feedback obtained by the AI.

 

The discussion section is interesting and as I mentioned it reduces the bias in favor of AI, although not completely.

 

 

Finally, I understand the comparison made between PBL with and without the use of AI, however, this work still seems far from the proposed objective which is “to contribute to the design of an AI-optimized PBL pedagogical model”. And if so, I would like to know in detail the design and the pedagogical model, and how it was modified and contributed due to this study.

Author Response

Dear reviewer. Thanks for the comments to improve our article. It has enriched us a lot and below I highlight the modifications:

1. Border and frame have been added to tables and figures to establish their limits.
2. The training and tools offered to teachers have been clarified more specifically, as well as their degree of satisfaction with training.
3. Questions have been explicitly collected and incorporated into the questionnaire.
4. We have included the importance of assessing the reduction in anxiety that students experience due to the use of AI in PBL, as well as future steps to continue the development of this method.

Attached I send you the article with the changes in blue also in accordance with reviewers' suggestions.

Thanks again for your time and dedication.

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you for your response to comments in previous review. Several things have been improved. However, some important things remain in this new text. 

It is great you described your experiment in lines 95 to 101; however, it is important for you to extract from there a general methodology that can be adapted in any course by any professor and then, show that it works by means of your results. That is, first, you have to describe a general list of steps that will help any professor to insert PBL to their courses. Then, you say that, to test it, you build the experiment described in lines 95 to 101. And then, you prove it right by means of your statistical analysis. In this way, your contribution to the state of art is this series of general steps that anyone can deploy in their courses. 

Specify things in lines 173 to 179. For example, how many experts were involved? What do you mean by minor revisions? Only wording or question ordering?

Regarding the statistical test, even if your distribution seems to be close to the normal distribution, you cannot use t test because it is used for continuous variables and a Likert scale is not continuous. You can use a Mann-Whitney U-Test (see https://www.uv.es/prodat/ViSta/vista-frames/help/lecturenotes/lecture09/lec9part4.html). It is very likely that you will get similar and positive results.

Additionally to this, your normality test fails for one of the samples. If you want to explore further in this path, you can change Figure 1 for a raincloud plot to see the shape of the distribution. 

Furthermore, there is a need to reorient the discussion section. Your work is about the perception of professors when using AI in PBL. In order to know if the general methodology I mention before is better in front of other methodologies, you have to make a comparison of your methodology to other works that have implemented a similar methodology and that have asked professor about their perceptions of using AI. You cannot compare your methodology to the results on students (learning gain) when using AI because you are not measuring this. The current discussion section can be useful as a review of the state of the art to say that inserting AI in PBL works for students and to state the research questions you have in lines 64 to 75. That is, to say: "AI in PBL offers better learning gains than traditional methodologies, but what about teachers? Do they like AI in PBL?". In short: your discussion section is not centered in answering your research questions about professors perceptions, nor can your hypothesis, about the improvement of learning, be proven based on statistical results that focus on teachers' perceptions because learning is not being measured. 

You have a good work, the only thing missing is organization to show that what you found is that professors see positive characteristics in AI in PBL, which is not the same as saying that AI in PBL has positive results in learning (your hypothesis), you did not test nor find this last thing.

Author Response

Dear reviewer. Thank you for your comments to improve our article. It has enriched us a lot and below I highlight the modifications:

1. The steps that we indicate to teachers have been included to be able to begin the application of the AI-based method regardless of the teaching stage.
2. The number of experts consulted has been incorporated with parentheses.
3. By minor changes we mean that they involved modifications in the style or wording of some of the questions given their experience.
4. Considering the feedback from other reviewers who highligthed the Likert scale as a positive aspect, along with our own criteria, we consider maintaining it as a it form the basis of the research.
5. References have been added to the discussion to teachers' perception of AI, also aligning it with the proposed hypothesis.

Attached I send you the article with the changes in blue also in accordance with reviewers' suggestions.

Thank you again for your time and dedication.

Reviewer 4 Report

Comments and Suggestions for Authors

The article under review is a revised version of the article under the same title. The main changes affected Part 2 "Materials and Methods". This version contains new information about the participants. In particular, it is indicated that there was a new phase "Intro phase", in which teachers were trained. In addition, it is indicated that the teachers received continuous support from the researchers. This, of course, significantly reduces the risks of incompetence on the part of the teachers and makes the experiment quite evidential.

However, some questions arise.

1. What does the letter T mean for the paragraphs on page 4. If these are new versions of the text fragments from page 3, then the authors should not leave the old version.

2. Why is the fact that the experiment took place in two periods of five months mentioned several times?

3. What young people are mentioned in line 21 from the top?

There are many such minor questions and repetitions in the text and the authors could have avoided this if they had carefully approached the editing of their text.

Conclusion. The article is quite suitable for publication, but needs editorial revision, at least in part 2.

Author Response

Dear reviewer. Thank you for your comments to improve our article. It has enriched us a lot and below I highlight the modifications:

1. The lettter T was a writing and editing error. It has been corrected.
2. A final repetition that each phase of the study lasted five months has been removed. The objective is to transfer the number of months that each phase has lasted.

Attached I send you the article with the changes in blue also in accordance with reviewers' suggestions.

Thank you again for your time and dedication.