Educational Impacts on Robotic Engineering Students of an International Online Project-Based Learning Course

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

 

Review

The topic of the article, the impact of project-based online learning on the competencies of robotics engineering students, is timely and interesting. This article examines the impact of project-based learning (PBL) methodology on acquiring generic and specific competencies in a group of second-year Robotics Engineering students. The study investigates an international online PBL course involving students from several European universities. The researchers conducted a qualitative and quantitative analysis comparing students who completed the course with those who did not. The findings indicate a significant improvement in both generic and specific competencies among students completing the online PBL course, supporting the effectiveness of this methodology in enhancing engineering education. The study also leverages existing research on PBL, online learning, and international collaborations in engineering education to contextualise its findings.

The abstract and introduction clearly situate the research within the existing literature on project-based learning (PBL) in engineering education, highlighting its benefits for developing both generic and specific competencies. The results section presents both qualitative and quantitative data supporting the positive impact of the PBL course on student competencies.  The discussion section further contextualises the results by comparing them to findings from previous studies on online and blended learning in engineering.  Therefore, the content is succinctly described and well-contextualized with respect to the relevant theoretical and empirical background.

The research has potentially significant practical value as it can provide information for developing effective online engineering education methods. However, the presentation of the results could be improved in several respects:

1.  The title is a reasonable length of 14 words. Generally, the title should be about 15 to 20 words so the reader can easily identify the topic. The abbreviation in the title may be misleading because the abbreviation PBL usually stands for "problem-based learning", but in this article, the author(s) use it for project-based learning. I propose that the title be amended.

2.  The abstract is missing key details needed for readers to grasp the overall content. It should clearly summarise the methods and steps involved. The author(s) should be much more specific. The results are not described in detail. The description of the results is vague and could be more specific to better fit the objective.

3. Keywords are a tool to help indexers and search engines find relevant articles. If the search engines in the databases can find the paper, readers will find it, too. This will increase the number of people reading your manuscript and is likely to result in more citations. The authors present five keywords. I think the number is appropriate and fits well with the text of the document. However, the order of presentation of keywords should move from the most general to the most specific, which is not the case in the document. I also recommend the use of small initials.

4. The Introduction and the Literature Review should be separated. The Introduction should be more concise and focus on the research problem, the importance of the study and the general context.

5. The Methodology section is misleading. It describes the pedagogical approach used in the online course (i.e., the teaching method employed), not the research methodology used to investigate the course's effectiveness. The research methodology should detail the statistical methods used for data analysis, participant selection (sampling methods), and other aspects of the research design.  The course methodology and the research methodology have to be distinct and should not be conflated. The methodology section should be rewritten.

6.  The article does not contain research questions. Despite its strengths, the explicit formulation of research questions would improve the quality of the article.  Currently, the objectives implicitly define the research questions, which is a less structured and less clear approach.

 7.   I am concerned that the article contains only descriptive statistics. Descriptive statistics are useful in themselves, as they present the data but do not provide information on whether the observed differences are statistically significant.  Thus, it is not possible to determine whether the observed differences are the result of random fluctuations or whether they are due to the true effect of the PBL course. I propose a hypothesis formulation. After that, the significance of the differences between the experimental and control groups can be analysed using statistical methods. This would show whether the differences are statistically supported and not just random fluctuations.

 8.   The Results and Discussion section should deal with a detailed presentation of the main results of the research and their critical analysis. In this section, student work presentation with photos is irrelevant. Moreover, the Results and Discussion section of the paper does not present an explicit and detailed comparison with the results of other researchers.

 9.   The limitations of the study, in particular the quasi-experimental design and the possibility of limited generalisability, should be more emphasised in the conclusions. A more thorough integration of the literature could also improve the quality of the article. Sample size and the specific characteristics of the participating university and programme may limit generalisability. Further clarification of the generalisability of the conclusions would be useful.

10.   The article introduces the concept of the PBL course, but does not include the detailed descriptions needed to reproduce the experiment in full.

11.   The quality of the English in the article is generally good, with a few minor errors. The writing is mostly clear, concise and grammatically correct. The scientific vocabulary is appropriate to the topic. Some sentences could be slightly improved for clarity or continuity, perhaps by breaking long, complex sentences into shorter, more digestible sentences.

 

Comments for author File: Comments.pdf

Author Response

We would like to express our sincere gratitude for the comments and suggestions provided. We believe they have been very accurate and are confident that they have significantly improved the quality and clarity of our article. Below, we present our responses to each of the points raised:

It is worth mentioning a typographical error found in the manuscript, which has been corrected in this revision. Where it says 34 students who did not take the course, it should have said 43.

Comments 1:

The title is a reasonable length of 14 words. Generally, the title should be about 15 to 20 words so the reader can easily identify the topic. The abbreviation in the title may be misleading because the abbreviation PBL usually stands for "problem-based learning", but in this article, the author(s) use it for project-based learning. I propose that the title be amended.

Response 1:

Thank you for pointing this out. We agree with this comment. Therefore, we have made the necessary changes. The title has been expanded and clarified according to your recommendations to more clearly identify the topic.

New title: “Educational impact in Robotic Engineering students by means of an international on-line Project Based Learning course”.

Comments 2:

The abstract is missing key details needed for readers to grasp the overall content. It should clearly summarise the methods and steps involved. The author(s) should be much more specific. The results are not described in detail. The description of the results is vague and could be more specific to better fit the objective.

Response 2:

Thank you for your valuable feedback. We have, accordingly, revised and rewritten the abstract according to your suggestions. It now includes detailed information on the methods, a clear description of the procedure used, and a more specific presentation of the results.

Comments 3:

Keywords are a tool to help indexers and search engines find relevant articles. If the search engines in the databases can find the paper, readers will find it, too. This will increase the number of people reading your manuscript and is likely to result in more citations. The authors present five keywords. I think the number is appropriate and fits well with the text of the document. However, the order of presentation of keywords should move from the most general to the most specific, which is not the case in the document. I also recommend the use of small initials.

Response 3:

Thank you for pointing this out. We agree with this comment. Therefore, we have changed the order of the keywords according to the given recommendations.

Keywords: online education, collaborative learning, cross-cultural projects, project based learning, robotics engineering

Comments 4:

The Introduction and the Literature Review should be separated. The Introduction should be more concise and focus on the research problem, the importance of the study and the general context.

Response 4:

Agree. We have, accordingly, revised and modified the Introduction. The paragraphs between lines 48 and 58 have been rewritten to be more concise and to focus on the research problem.

A new section (Section 2) has been created specifically for the literature review (line 66).

The paragraphs that were between lines 135 and 155 in the original paper have been synthesized and relocated to Section 4.1, 'Online Course Methodology,' forming part of the first two paragraphs of this section. These changes have improved the clarity of the text.

Comments 5:

The Methodology section is misleading. It describes the pedagogical approach used in the online course (i.e., the teaching method employed), not the research methodology used to investigate the course's effectiveness. The research methodology should detail the statistical methods used for data analysis, participant selection (sampling methods), and other aspects of the research design.  The course methodology and the research methodology have to be distinct and should not be conflated. The methodology section should be rewritten.

Response 5:

Agree. We have, accordingly, rewritten this point. The methodology used in the research methodology section to evaluate the course's effectiveness has been revised (lines 215-281). It describes the qualitative and quantitative analyses, the statistical methods used as well as the selection of participants and other aspects of the research, are also detailed. A separate and specific section, “4.1 Online Course Methodology,” has been created to describe the pedagogical approach separately (lines 283-348)

Comments 6:

The article does not contain research questions. Despite its strengths, the explicit formulation of research questions would improve the quality of the article.  Currently, the objectives implicitly define the research questions, which is a less structured and less clear approach.

Response 6:

Thank you for pointing this out. We agree with this comment. Therefore, we have reformulated the point 3 Educational objectives into questions according to the recommendations (lines 163-170).

Comments 7:

I am concerned that the article contains only descriptive statistics. Descriptive statistics are useful in themselves, as they present the data but do not provide information on whether the observed differences are statistically significant.  Thus, it is not possible to determine whether the observed differences are the result of random fluctuations or whether they are due to the true effect of the PBL course. I propose a hypothesis formulation. After that, the significance of the differences between the experimental and control groups can be analysed using statistical methods. This would show whether the differences are statistically supported and not just random fluctuations.

Response 7:

Agree. We have, accordingly, included an inferential analysis to improve this point. A new table of results (Table 5) is included, and the most significant data are critically discussed (lines 514-547).

Comments 8:

The Results and Discussion section should deal with a detailed presentation of the main results of the research and their critical analysis. In this section, student work presentation with photos is irrelevant. Moreover, the Results and Discussion section of the paper does not present an explicit and detailed comparison with the results of other researchers.

Response 8:

Thank you for pointing this out. We agree with this comment. Consequently, we have removed the section on the student work presented in the online course. The text describing Section 6.1, 'Qualitative Analysis,' has been rewritten (lines 427-447). The graphs related to the qualitative analysis have been eliminated as they were redundant representing the same results as the tables.

The results of the qualitative analysis have been detailed more thoroughly (lines 427-447 and lines 497-501).

The title of Table 3 has been revised.

The paragraph describing Figure 1, has been rewritten to enhance the understanding of the graph (lines 508-513).

A new paragraph (lines 518-544) has been added, providing a critical analysis of the obtained results. Additionally, a paragraph has been added (lines 567-572) to clarify the conclusions in Section 5.2.

Furthermore, a bibliographic reference [30] has been included in the '7. Conclusions' section, citing a study titled 'Project-Based Learning (PBL) as an Experiential Pedagogical Methodology in Engineering Education: A Review of the Literature.' This study analyzes a large sample of research related to this field, and its conclusions align with the results of the presented study (lines 617-621).

Comments 9:

The limitations of the study, in particular the quasi-experimental design and the possibility of limited generalisability, should be more emphasised in the conclusions. A more thorough integration of the literature could also improve the quality of the article. Sample size and the specific characteristics of the participating university and programme may limit generalisability. Further clarification of the generalisability of the conclusions would be useful.

Response 9:

Thank you for pointing this out. We agree with this comment. Therefore, we have revised section “6.3 Limitations of the study” (lines 573-585) and added a bibliographic reference [30] that supports the results obtained in the presented study (lines 617-621).

Comments 10:

The article introduces the concept of the PBL course, but does not include the detailed descriptions needed to reproduce the experiment in full.

Response 10:

Thank you for pointing this out. The online course has been described in detail in section 4.1 Online Course Methodology, including the “Weekly Course Schedule.” We have provided the survey for replicating the qualitative study. The Data Availability Statement link (https://doi.org/10.17632/hnkz43sd58.1) leads to the written exam questions used for the quantitative analysis. While we acknowledge the limitations of the presented study, we believe that we have provided the necessary information for its reproduction.

Comments 11:

The quality of the English in the article is generally good, with a few minor errors. The writing is mostly clear, concise and grammatically correct. The scientific vocabulary is appropriate to the topic. Some sentences could be slightly improved for clarity or continuity, perhaps by breaking long, complex sentences into shorter, more digestible sentences.

Response 11:

Thank you for pointing this out. We agree with this comment. Therefore, we have revised to improve and clarify its readability. Some sentences in the introduction have been simplified into shorter phrases to enhance comprehension without altering the content. The new text has been carefully included following your instructions.

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

Thanks for your effort to this research. You may consider the following observations to improve its overall quality.

(1) The abstract should have a brief about the methodology of the research with a clear indication of sample size and sampling technique.

(2) On the whole work, the specific role and engagement of the students of UBI and GU were not clearly stated. The selection two types of student-groups (studied with and without PBL) from the universities (is it only UA?) was not clear.

(3) Lines 177 and 197: How the student competencies of three different universities under two subject areas were came into a single platform? The paper should clearly mention the process of selecting the GCs and the SCs. 

(4) Line 215: What is ECTS credit and how has it a relevance with the context?

(5) Lines 239-241: The paper should answer the questions: How many students were in the sample from UBI and GU? How these eight groups were involved under the PBL activity?

(6) Section 4.1: Qualitative analysis is actually describing the numerical findings from the likert responses, which is questionable to be considered as qualitative. Did the respondents have any open ended questions to answer?

(7) Line 263: CE1-CE4 might be a typo. If not, please elaborate with identifications.

(8) Table 2: SC1-4 questions should have been on specific competencies, not generic competencies.

(9) Lines 277-278: The paper should clearly state how the 34 students did study: it was not under PBl, but what was the method?

(10) Figure 3 and Lines 365-368: Was this activity included in this research? Its description was not justified.

(11) Figure 4 and Table 3 are same by the information. Also, Figure 5 and Table 4 are same by the information. Only one can be kept in each case, if not necessary at specific reasons.

At the end, best wishes for this attempt of publication.

Author Response

We would like to express our sincere gratitude for the comments and suggestions provided. We believe they have been very accurate and are confident that they have significantly improved the quality and clarity of our article. Below, we present our responses to each of the points raised:

It is worth mentioning a typographical error found in the manuscript, which has been corrected in this revision. Where it says 34 students who did not take the course, it should have said 43.

Comments 1: The abstract should have a brief about the methodology of the research with a clear indication of sample size and sampling technique.

Response 1:

Thank you for your valuable feedback. We have, accordingly, revised, rewritten and improved the abstract according to your suggestions. It now includes detailed information on the sample size and sampling technique.

Comments 2: On the whole work, the specific role and engagement of the students of UBI and GU were not clearly stated. The selection two types of student-groups (studied with and without PBL) from the universities (is it only UA?) was not clear.

Response 2:

Thank you for pointing this out. We agree with this comment. To clarify the specific role and engagement of the students from UBI and GU, as well as the selection of the two types of student groups (those who studied with and without PBL), we have rewritten section 4.1. This section provides a detailed explanation of the online course methodology. Additionally, we have addressed this issue in the newly written paragraphs found in lines 326-335.

Comments 3: Lines 177 and 197: How the student competencies of three different universities under two subject areas were came into a single platform? The paper should clearly mention the process of selecting the GCs and the SCs.

Response 3:

Thank you for pointing this out. As indicated above and as stated in the new abstract and introduction, the study focuses on the analysis of students at the University of Alicante. For this reason, the selection of competences was made only on the basis of all the competences defined in the Verification Report of the UA degree in Robotics Engineering, reference [28]. Among all of them, those that had some kind of relationship with the activities to be developed in the subject ‘Robot Mechanisms and Modelling’ and in the online course were selected. Some of the specific competences have also been selected because of their relation to the online course and correspond to competences linked to the subject ‘Graphic Expression’, taken by all the students the previous year.

Comments 4: Line 215: What is ECTS credit and how has it a relevance with the context?

Response 4:

We agree with this comment. Therefore, we have clarified the meaning. ECTS credit (European Credit Transfer and Accumulation System) (lines 294-295) is a standardized unit of measurement used in the European Higher Education Area (EHEA) to quantify the workload required to complete university courses and study programs. One ECTS credit corresponds approximately to a workload of 25-30 hours.

 Comments 5: Lines 239-241: The paper should answer the questions: How many students were in the sample from UBI and GU? How these eight groups were involved under the PBL activity?

Response 5:

Thank you for pointing this out. We agree with this comment. Therefore, we have mentioned this issue in section 4.1 Online Course Methodology, lines 326-332.

Comments 6: Section 4.1: Qualitative analysis is actually describing the numerical findings from the likert responses, which is questionable to be considered as qualitative. Did the respondents have any open ended questions to answer?

Response 6:

Thank you for pointing this out. We agree with this comment. The qualitative questionnaire was developed using only closed questions focused on qualitative aspects, specifically asking about the participants' perception of their competence acquisition. This approach was chosen to facilitate the processing and comparison of the results. The inclusion of some open-ended questions would likely have enriched the analysis. Additionally, the limitations of the study have been emphasized in section 6.3.

Comments 7: Line 263: CE1-CE4 might be a typo. If not, please elaborate with identifications.

Response 7:

Thank you for pointing this out. Yes, this is a typographical error. It should state: "(generic GC1-GC9 and specific SC1-SC4).

Comments 8: Table 2: SC1-4 questions should have been on specific competencies, not generic competencies.

Response 8:

Thank you for pointing this out. To clarify this point, the title of the table has been rewritten as “Questions asked in questionnaire 1 and their related generic and specific competencies”. Table 2 includes the two types of competencies: generic and specific. Through qualitative analysis, the perception of the degree of acquisition of both types of competencies is evaluated. Quantitative analysis provides information about the specific competencies.

Comments 9: Lines 277-278: The paper should clearly state how the 34 students did study: it was not under PBl, but what was the method?

Response 9:

We agree with this comment. Therefore, we have clarified this question in point 4.1 Online Course Methology, lines (303-305)

Comments 10: Figure 3 and Lines 365-368: Was this activity included in this research? Its description was not justified.

Response 10:

Agree. Based on your comment and the feedback from the other reviewer, we have decided to remove the images and their descriptions from the research article. These images did not address the main topic of the article.

Comments 11: Figure 4 and Table 3 are same by the information. Also, Figure 5 and Table 4 are same by the information. Only one can be kept in each case, if not necessary at specific reasons.

Response 11:

Thank you for pointing this out. Completely agree. We have removed the graphical images whose information was redundant with the tables.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The modifications made have significantly improved the quality of the paper.

Author Response

Thank you very much for your valuable comments and suggestions. The article has gained in clarity and quality.

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Author(s),

Thanks for addressing the comments and improve the paper's merit.

In the abstract, consider mentioning the teaching-learning approach of 'project based learning'.

Also rethink to include the keyword 'cross cultural project' in the list.

Author Response

Comments:

In the abstract, consider mentioning the teaching-learning approach of 'project based learning'.

Also rethink to include the keyword 'cross cultural project' in the list.

Response:

We greatly appreciate your comments and agree with your observations. We have mentioned the teaching-learning approach of 'project-based learning' in line 9 of the abstract. Additionally, we have reconsidered the keyword ‘cross cultural project’ and replaced it with 'student perception', which we have seen used in other articles as a keyword and believe is very appropriate for this study.