The Impact of Flipped Learning on Cognitive Knowledge Learning and Intrinsic Motivation in Norwegian Secondary Physical Education

Round 1

Reviewer 1 Report

Overall, this is a good study and the research questions are interesting. You do a nice job with background information in the introduction. There are a number of issues that I feel you must address prior to publication.

Writing

• As a whole, the paper is well-written. There are a few instances, however, where the wording choice makes it difficult to fully understand what is being said. As a result, reading the paper is not as fluid for the reader.

Methods

• Would you include a statement that describes the “play-like” activities that were done so that the reader has an understanding of the extent of change from the traditional sport-like activities? What is the difference between the two?
• In your procedures, state when the tests were administered (e.g., both at the first day of study and again on the last day of the study), especially to make it clear to the reader that it was repeated measures.
• The statistical test was incorrect. It was mentioned that a one-way ANOVA was used. It should be a mixed factorial ANOVA since there were two independent variables: sex (independent groups), learning method (independent groups), and time (repeated measures). Also state the number of ANOVAs run. Since you had multiple dependent variables (the different types of motivation as well as HRFK), you would have had to run it multiple times.

Results

• Please reference the figure numbers in the results section.
• In Table 1, please put an asterisk (*) with the values that were found to be statistically different.
• In Table 1, I’m not sure that the “Total” column is for. Is that evaluating the main effect of sex?
• Include the η2 in Table 1
• Include asterisks in the figures to show what changes were statistically significant.
• I am not sure why you ran the t-tests since the ANOVA would provide you with information about sex differences.
• The biggest issue in this section is that you did not report the means with the simple effect tests and what the effect was (e.g., boys’ scores increased significantly). Put these means in the body of the text, not just in the table, so that the reader can interpret the results as they are reading along.

Discussion

• In this section, leave the statistics jargon (e.g., “interaction effect”) out and discuss it in straight-forward terms. For example, simply state that “IM decreased for boys in the NFL group while IM remained the same for girls in the NFL group. For both boys and girls in the FL group, IM increased slightly, although this increase was not significant. This indicates that…”
• Be careful in your interpretation – or your explanation of your interpretation – for HRFK. There was not difference at T2 between both groups of girls. This does not necessarily mean that the girls in the FL group learned more; they in fact started out with lower HRFK scores at T1. The only real effect in teaching method appears to have happened for the boys: not only was the grown in the FL boys greater, their scores were higher at T2, even compared to the girls. How would you interpret this?
• This is just a suggestion: Without having done research on this myself, you may want to consider including a statement or two about self-efficacy when you are addressing future research, since it too relates strongly to behavior. This may also help explain the sex differences that you found.
• In your conclusion, you state, “The present study demonstrates that girls benefit from such an activity change, but that the IM of boys…” Looking at figure 1, however, it doesn’t appear that girls benefit from the activity change. They remain the same. If anything, they slightly benefit when FL is implemented. (Was this slight increase in IM significant?)

Author Response

Reviewer 1

Writing

• As a whole, the paper is well-written. There are a few instances, however, where the wording choice makes it difficult to fully understand what is being said. As a result, reading the paper is not as fluid for the reader.
  • As a sum of all changes made to the manuscript, we hope the reader will now perceive a better fluency.

Methods

• Would you include a statement that describes the “play-like” activities that were done so that the reader has an understanding of the extent of change from the traditional sport-like activities? What is the difference between the two?
  • We agree, and an elaboration on play-like, in form of an example that illustrates the difference regarding a more traditional approach is included (Line 213-215).
• In your procedures, state when the tests were administered (e.g., both at the first day of study and again on the last day of the study), especially to make it clear to the reader that it was repeated measures.
  • We have now included some text on this as suggested, including moving this text to the measurement-section as suggested by Reviewer 2 (Line 250-259)
• The statistical test was incorrect. It was mentioned that a one-way ANOVA was used. It should be a mixed factorial ANOVA since there were two independent variables: sex (independent groups), learning method (independent groups), and time (repeated measures). Also state the number of ANOVAs run. Since you had multiple dependent variables (the different types of motivation as well as HRFK), you would have had to run it multiple times.
  • One-way ANOVA was a typing-error. Have corrected to mixed factorial ANOVA. The number of ANOVAs was one per DV, totalling five tests. This is now included in the manuscript. We have also included benchmarks for partial eta squared; .0099 for small, .0588 for medium, .1379 for large. (Line 269-273)

Results

• Please reference the figure numbers in the results section.
  • The figures are now referred to in the text. (Line 284 and 320)
• In Table 1, please put an asterisk (*) with the values that were found to be statistically different.
  • To keep the clarity of table 1 we have marked significant differences from the main ANOVAs using superscript a, b, c for sig. a) main effect, b) interaction between time and instruction group, c) interaction between time and gender.
• In Table 1, I’m not sure that the “Total” column is for. Is that evaluating the main effect of sex?
  • Total column is intended to show descriptive statistics for the total sample and is used when evaluating the main effect of sex. Gender differences is an important aspect in the literature, so including means and SD of boys and girls in order to make comparisons across studies appears reasonable.
• Include the η2 in Table 1
  • According to reviewer 2 we should avoid presenting results twice. Also, there is an issue of clarity of the table. Introducing partial η2 in the table, with the many test conducted, would make the table more difficult to read. Please note that we have followed the good advice of including means from the table when presenting simple effects, making for a stronger connection with the table.
• Include asterisks in the figures to show what changes were statistically significant.
  • Tables show separate lines for boys and girls in the FL and NFL groups, but do not show lines for boys or girls total or FL and NFL- groups total. Providing separate lines with asterisks could therefore be misinterpreted, and we think the statistically significant changes are better described in the text as is. The main point of the figures is to provide the reader with a graphical presentation of changes related to the relative complexity of the interaction effects. We therefore argue to leave the figures as is, without including asterisks.
• I am not sure why you ran the t-tests since the ANOVA would provide you with information about sex differences.
  • T-tests were run to check for gender and instruction group differences at T1 since ANOVA does not provide this information without running simple effects. Have now deleted t-tests and checked for simple effects even when there are no interactions. Results remain the same.
• The biggest issue in this section is that you did not report the means with the simple effect tests and what the effect was (e.g., boys’ scores increased significantly). Put these means in the body of the text, not just in the table, so that the reader can interpret the results as they are reading along.
  • Have included means in brackets in the text when presenting simple effects. Think this is a good advice that works very well. (Line 292-306 and 323-328)

Discussion

• In this section, leave the statistics jargon (e.g., “interaction effect”) out and discuss it in straight-forward terms. For example, simply state that “IM decreased for boys in the NFL group while IM remained the same for girls in the NFL group. For both boys and girls in the FL group, IM increased slightly, although this increase was not significant. This indicates that…”
  • We have rephrased some text in the discussion section. (Line 339)
• Be careful in your interpretation – or your explanation of your interpretation – for HRFK. There was not difference at T2 between both groups of girls. This does not necessarily mean that the girls in the FL group learned more; they in fact started out with lower HRFK scores at T1. The only real effect in teaching method appears to have happened for the boys: not only was the grown in the FL boys greater, their scores were higher at T2, even compared to the girls. How would you interpret this?
  • We discuss the results not specific to ‘gender’*’instruction typ’ just because of the point made by the reviewer. The FL group increased their HRFK much more than the NFL group (and we do not distinct on gender), and this is what me mention in the discussion.
• This is just a suggestion: Without having done research on this myself, you may want to consider including a statement or two about self-efficacy when you are addressing future research, since it too relates strongly to behavior. This may also help explain the sex differences that you found.
  • We appreciate the comment, but at this time we chose not to follow the path suggested.
• In your conclusion, you state, “The present study demonstrates that girls benefit from such an activity change, but that the IM of boys…” Looking at figure 1, however, it doesn’t appear that girls benefit from the activity change. They remain the same. If anything, they slightly benefit when FL is implemented. (Was this slight increase in IM significant?)
  • We argue that holding a stable IM, when research shows that IM decreases with time, is a benefit. It is beneficial to uphold a stable level of IM in contrary to experience a decrease, as research demonstrates happens, and especially among girls.

Reviewer 2 Report

I recommend in the abstract to mention aspects referred to in the methods section such as: samples, tests, etc.
Please mention in brackets what ICT means.
Line 58, I think is a mistake in the numbering of the bibliographic index, 36 not 26, I recommend the verification and correction.
I think the study is outdated due to the period, 2016. I am convinced that during the last 4 years the data have changed considerably, which makes this study not relevant and up to date.
Line 172, I recommend specifying what VG1 means.
Line 227, I recommend replacing the word test with questionnaire or script test, because in PE, the test usually refers to the physical, practical part.
Line 237, I recommend that the section be called Results, the procedure is part of the measurements, there is a slight confusion of terms.
Table 1, until this table did not mention T1 and T2, I recommend clarification.
I recommend that in the participant section you mention the number of samples, they appear only in table 1.
I recommend that the title of table 1 be reformulated, specifically.
Line 254, repeat the table number, again tables 1. I recommend reformulating the name of this table.
Line 256 where is the table ?????
Figure 1, repeat the test mentioned in lines 260-266, I recommend deleting the figure so that the data is not duplicated, as in figure 2.

I recommend that the discussion section be focused on the results of this study in correlation with other studies, it is too long and bad ideas presented previously.

Author Response

Reviewer 2:

• I recommend in the abstract to mention aspects referred to in the methods section such as: samples, tests, etc.
  • Aspects concerning sample and type of intervention included in abstract. (Line 14-17)
• Please mention in brackets what ICT means.
  • Information communication technology (ICT) included in abstract (Line 16-17)
• Line 58, I think is a mistake in the numbering of the bibliographic index, 36 not 26, I recommend the verification and correction.
  • The reference number 26 is used two times in the Introduction part. That is why is appears misplaced (Line 52 and 67)
• I think the study is outdated due to the period, 2016. I am convinced that during the last 4 years the data have changed considerably, which makes this study not relevant and up to date.
  • We appreciate the comment, but we disagree. Our empirical data, which consists of measurements of student motivation and knowledge, is considered by us to me more stable than the reviewer expresses. In just 4 years, student motivation and knowledge regarding FL in PE will probably not have changed considerably. At least we consider this to be the case within the Norwegian context. We believe this is an important contribution for other studies to build upon, and studies in the near future can assess if there is a considerable change in student motivation and knowledge when applying a FL framework. Although the context would have to be similar.
• Line 172, I recommend specifying what VG1 means.
  • Have replaced VG1 with Year 11. This could also stand as first year of upper secondary education if preferred. (Line 186 and 253)
• Line 227, I recommend replacing the word test with questionnaire or script test, because in PE, the test usually refers to the physical, practical part.
  • Changed from test to questionnaire throughout the subsection “measures” (Line 246, 252 and 254)
• Line 237, I recommend that the section be called Results, the procedure is part of the measurements, there is a slight confusion of terms.
  • Have included this section into measures-section by deleting the section heading. Some mote text is included as requested by Reviewer 1. (Line 257-259)
• Table 1, until this table did not mention T1 and T2, I recommend clarification.
  • T1 and T2 is now included in the methods-section. (Line 259)
• I recommend that in the participant section you mention the number of samples, they appear only in table 1.
  • Have specified the number of participants in the FL and NFL groups, and provided percentage of girls in the two samples. (Line 189-191)
• I recommend that the title of table 1 be reformulated, specifically.
  • We consider the table title, together with the note, gives adequate and precise information, so no changes are made.
• Line 254, repeat the table number, again tables 1. I recommend reformulating the name of this table.
  • This is not the table heading
• Line 256 where is the table ?????
  • The table is not misplaced in our working-document, and we will make sure it stays in the right place when we upload this revised manuscript.
• Figure 1, repeat the test mentioned in lines 260-266, I recommend deleting the figure so that the data is not duplicated, as in figure 2.
  • Results in figure 2 are duplicated in the text in lines 320-328. This is done because of the interaction effects displayed in the case of internal motivation (IM) and health related fitness knowledge (HRFK). Interaction effects are often complexed to understand, and we believe visualizing the results are helpful for the reader. We have included a reference text to the figures in text to ease the reading flow.
• I recommend that the discussion section be focused on the results of this study in correlation with other studies, it is too long and bad ideas presented previously.
  • We have included two more studies in the introduction and the discussion.

Reviewer 3 Report

The study carried out is of interest to the scientific community. The use of flipped learning is currently booming in training processes, making this research a potential potential publication for readers.

However, for its approval, the following observations must be taken into account:

-The abstract must be expanded. It also contains abbreviations that have not been explained, although they are understood as ICT. But every abbreviation must be detailed for the first time.

-The introductory section should be expanded to include potential studies that have not been taken into account and that have studied aspects analyzed here such as motivation. It is suggested to analyze and add the following studies indexed in JCR and Scopus:

Hinojo F.J; López, J.; Fuentes, A.; Trujillo, J.M.; Pozo, S. Academic Effects of the Use of Flipped Learning in Physical Education. Int. J. Environ. Res. Public Health 2020, 17, 1-14. https://doi.org/10.3390/ijerph17010276

López, J.; Fuentes, A.; López, J.A.; Pozo, S. Formative Transcendence of Flipped Learning in Mathematics Students of Secondary Education. Mathematics 2019, 7, 1-14. https://doi.org/10.3390/math7121226

López, J.A.; López, J.; Moreno, A.J.; Pozo, S. Effectiveness of Innovate Educational Practices with Flipped Learning and Remote Sensing in Earth and Environmental Sciences—A Case Study. Remote Sens. 2020, 12, 1-14. https://doi.org/10.3390/rs12050897

Moreno-Guerrero, A.J.; Romero-Rodríguez, J.M.; López-Belmonte, J.; Alonso-García, S. Flipped Learning Approach as Educational Innovation in Water Literacy. Water 2020, 12, 1-13. https://doi.org/10.3390/w12020574

Pozo, S.; López, J.; Moreno, A.J.; López, J.A. Impact of Educational Stage in the Application of Flipped Learning: A Contrasting Analysis with Traditional Teaching. Sustainability 2019, 11, 1-15. https://doi.org/10.3390/su11215968

-All the rules of adaptation to the journal's template have not been respected. The subsections must follow the format indicated by the magazine.

-Table 1 appears in a position that is not correct.

-The discussion of the results is adequate but should include the new incorporated studies that are proposed to collect the entire state of the matter.

-The section titled Strengths and limitations should go after the conclusions.

-The references section does not comply with the standards established by the journal.

Author Response

Reviewer 3:

• The abstract must be expanded. It also contains abbreviations that have not been explained, although they are understood as ICT. But every abbreviation must be detailed for the first time.
  • The abstract is now expanded, and ICT is explained.
• The introductory section should be expanded to include potential studies that have not been taken into account and that have studied aspects analyzed here such as motivation. It is suggested to analyze and add the following studies indexed in JCR and Scopus:
  Hinojo F.J; López, J.; Fuentes, A.; Trujillo, J.M.; Pozo, S. Academic Effects of the Use of Flipped Learning in Physical Education. J. Environ. Res. Public Health 2020, 17, 1-14. https://doi.org/10.3390/ijerph17010276
  López, J.; Fuentes, A.; López, J.A.; Pozo, S. Formative Transcendence of Flipped Learning in Mathematics Students of Secondary Education. Mathematics 2019, 7, 1-14. https://doi.org/10.3390/math7121226
  López, J.A.; López, J.; Moreno, A.J.; Pozo, S. Effectiveness of Innovate Educational Practices with Flipped Learning and Remote Sensing in Earth and Environmental Sciences—A Case Study. Remote Sens. 2020, 12, 1-14. https://doi.org/10.3390/rs12050897
  Moreno-Guerrero, A.J.; Romero-Rodríguez, J.M.; López-Belmonte, J.; Alonso-García, S. Flipped Learning Approach as Educational Innovation in Water Literacy. Water 2020, 12, 1-13. https://doi.org/10.3390/w12020574
  Pozo, S.; López, J.; Moreno, A.J.; López, J.A. Impact of Educational Stage in the Application of Flipped Learning: A Contrasting Analysis with Traditional Teaching. Sustainability 2019, 11, 1-15. https://doi.org/10.3390/su11215968
  • We have included the first suggested research in our manuscript. Both as part of the introduction (Line 155) and discussion section (Line 346-450). The other suggestions are from our perspective not relevant regarding age-level, school subject or context, as our study were conducted in a Norwegian PE context among adolescents age 13-17.
  • In addition we have included another relevant study in the manuscript: Villalon, L.; Sebastiá Amat, S.; Pueo, B., El modelo pedagógico flipped classroom en la adquisición de conocimientos. In Inclusión, Tecnología y Sociedad: investigación e innovación en educación, Marín, J. A. M.;  García, G. G.;  Navas-Parejo, M. R.; Soto, M. N. C., Eds. Dykinson: Madrid, 2020; pp 574-584. Introduction (Line 162-164) and discussion (Line 394-396)
  • Our additions and some small corrections of the text in the Introduction section is seen in Line 149-167. We have made a short summary of the research previously done on FL in PE with the following text (Line 172-173) “These findings could suggest that the FL approach is beneficial across a range of school contexts and ages, and regarding several desired outcomes in PE.”
• All the rules of adaptation to the journal's template have not been respected. The subsections must follow the format indicated by the magazine.
  • This is revised. The subsections are now written in italic and positioned as suggested in the template.
• Table 1 appears in a position that is not correct.
  • The table is not misplaced in our working-document, and we will make sure it stays in the right place when we upload this revised manuscript.
• The discussion of the results is adequate but should include the new incorporated studies that are proposed to collect the entire state of the matter.
  • Two more studies are included in the discussion.
• The section titled Strengths and limitations should go after the conclusions.
  • Have moved “Limitations” (Renamed from Strengths and Limitations) after “Conclusions”.
• The references section does not comply with the standards established by the journal.
  • This is revised.

Round 2

Reviewer 1 Report

I have just a few more minor things for you:

In your results section in particular, you switch between the present tense and past tense. Go through the paper one more time and make sure everything is written in the past tense.

Also edit for punctuation. For example, I noticed in the results section that at one point you used a "," instead of a "." for a decimal. I know this varies from country to country - it should just be consistent.

Move the Limitations section so that it is before the Conclusion section.

For ease of reading, I would remove the wording that refers to the statistics, such as "interaction effect" and "simple effect" throughout the whole discussion. For example, at line 339 it could read, "However, the results of this study show that IM decreased for boys, whereas IM for girls remained stable from T1 to T2." This way the reader does not have to interpret what you are saying - it is more clear.

Author Response

Reviewer one

In your results section in particular, you switch between the present tense and past tense. Go through the paper one more time and make sure everything is written in the past tense.

• This is revised as suggested, and changes are highlighted in green.

Also edit for punctuation. For example, I noticed in the results section that at one point you used a "," instead of a "." for a decimal. I know this varies from country to country - it should just be consistent.

• This is revised as suggested, and changes are highlighted in green. Found only one though, at Line 303.

Move the Limitations section so that it is before the Conclusion section.

• This is revised as suggested, but the text is not marked in any way to show this. The numeration of these chapters are now: 5. Limitations, and 6. Conclusions.

For ease of reading, I would remove the wording that refers to the statistics, such as "interaction effect" and "simple effect" throughout the whole discussion. For example, at line 339 it could read, "However, the results of this study show that IM decreased for boys, whereas IM for girls remained stable from T1 to T2." This way the reader does not have to interpret what you are saying - it is more clear.

• We have now rephrased some of the text in the discussion according to the suggestions. The use of "interaction effect" and "simple effect" is revised throughout the whole discussion Edited text is marked with green.

Reviewer 2 Report

no comments

Author Response

Thank you for taking time to read and support our manuscript with helpful and constructive comments.