Learning and Emotional Outcomes after the Application of Invention Activities in a Sample of University Students

Round 1

Reviewer 1 Report

This paper examined the impact of Inventing to Prepare for Learning (IPL) in a university statistics course on performance and emotions. Findings showed that students in the IPL condition scored higher on a posttest measure of conceptual knowledge. Below I will describe some major and minor points to consider.

Major considerations

I believe that authors should recognize that this was not a test between IPL and direct instruction but instead between IPL and a worked example. In fact, both conditions received direct instruction in the final phase of instruction. Moreover, the entire duration of the experiment in terms of differentiation between conditions was 24 minutes, something that should be noted as a limitation. I think it is imperative to reframe the study not as a test between IPL and direct instruction but between IPL and a worked example. Please examine what an appropriate model of direct instruction contains (e.g., actual instruction, questioning, examples, practice, etc.).

I was surprised that the authors did not consider the implications of the flip side of their findings. What they found was a confirmation that students who come into a learning environment with more prior knowledge typically outperform their low-prior knowledge peers. So, what about the students who didn’t fare so well with IPL? Should they be provided with some form of brief direct instruction before the invention activity?

You mention critical thinking right out of the gate at the beginning of the paper but then this faded and was not well developed. I would be careful with the terms you use and only employ those that are central to your goals and what you are measuring.

You discuss cognitive load at some length but, in so doing, it might be beneficial to discuss the benefits of germane cognitive load and also how cognitive load might change over time.

Minor considerations

Two sentence titles are atypical. I would recommend a revision on the current title.

Research question #3 is a bit unclear and in general would be good to expound on the research questions along with hypotheses

I would refrain from emphasizing non-sig findings in the discussion.

Author Response

RESPONSES TO REVIEWER 1

This paper examined the impact of Inventing to Prepare for Learning (IPL) in a university statistics course on performance and emotions. Findings showed that students in the IPL condition scored higher on a posttest measure of conceptual knowledge. Below I will describe some major and minor points to consider.

Many thanks for your comments.

I believe that authors should recognize that this was not a test between IPL and direct instruction but instead between IPL and a worked example. In fact, both conditions received direct instruction in the final phase of instruction.. I think it is imperative to reframe the study not as a test between IPL and direct instruction but between IPL and a worked example.

The study has been reframed as comparing IPL with a “fully guided” strategy in which the concept is introduced to students with a worked example. To avoid confusion related to the fact that students in the IPL condition were also presented the worked example in a posterior phase, we have avoided naming the control condition as “worked example”. In cases where we could not specify that the difference between conditions refers to how the topic is introduced (invention activity versus worked example), we referred to the control condition as “fully guided” or used the term “guidance”. These small reframing changes have been done throughout the whole document (e.g., page 1, line 20; page 4, line 116; page 6, line 236; …).

Also, we have included a couple of sentences in the introduction to describe what a worked example is and how it was used in this study (page 4, line 134-141).

Moreover, the entire duration of the experiment in terms of differentiation between conditions was 24 minutes, something that should be noted as a limitation

The 24 minute duration of the experiment has been noted in the Limitation section as an important aspect to consider, adding that it is important for future studies to confirm the efficacy of invention activities of this short duration (page 15, lines 536-539).

I was surprised that the authors did not consider the implications of the flip side of their findings. What they found was a confirmation that students who come into a learning environment with more prior knowledge typically outperform their low-prior knowledge peers. So, what about the students who didn’t fare so well with IPL? Should they be provided with some form of brief direct instruction before the invention activity?

Different changes have been done in relation to this comment:

First, the different potential implications that previous knowledge can have on emotions have been briefly discussed in the introduction (page 5, lines 191-197). Second, we have done the following changes in the discussion:

   - Corrections have been made to accurately describe that the significant associations found referred to enjoyment with both previous knowledge and acquired knowledge in the invention condition (page 14, lines 491-494), but that the direct association between previous knowledge and acquired knowledge did not reach statistical significance (page 14, lines 498-500).

   - We have included the suggestion that further research should explore whether strategies including higher levels of guidance might work better in students who part from lower levels of previous knowledge (page 14, 503-505).

    - We have deleted the part which described the alternative scenario that we could expect (page 14, lines 500-502 ), because it has already been included in the introduction (as commented before, see page 5, lines 191-197).

You mention critical thinking right out of the gate at the beginning of the paper but then this faded and was not well developed. I would be careful with the terms you use and only employ those that are central to your goals and what you are measuring.

The concept of critical thinking has been taken away. The original discussion has been mostly conserved, but it has been put in relation to reflective learning processes, conceptual knowledge, or the ability to transfer knowledge to new situations, which are more in line with the scope of the study than critical thinking. These changes have been made in the Abstract (page 1, lines 13 and 26), the Introduction (page 1, lines 33-41), and in the Discussion (page 14, lines 512-517). Previous references match these transformations.

Additionally, in the Introduction, we have added definitions of the learning outcomes that are part of our goals: conceptual knowledge, transfer (page 2, lines 47-49), and procedural knowledge (page 4, lines 125-128). 

You discuss cognitive load at some length but, in so doing, it might be beneficial to discuss the benefits of germane cognitive load and also how cognitive load might change over time.

To address these recommendations, the discussion about cognitive load has been moved from its original paragraph (page 4, lines 147-152) to a new paragraph (page 3, lines 92-106), right after the description of processes that can be considered as “germane processes” (page 2, lines 85-87). In this new paragraph, we have made a description of extraneous cognitive load processes and germane cognitive load processes. Also, regarding the change over time, we hypothesized that what is initially perceived as extraneous cognitive load processes in the invention task can become germane cognitive load once students complement these processes with further instruction.

Two sentence titles are atypical. I would recommend a revision on the current title.

The title has been changed to: “Learning and Emotional Outcomes after the Application of Invention Activities in a Sample of University Students”

Research question #3 is a bit unclear and in general would be good to expound on the research questions along with hypotheses

This question has been simplified, and we have added hypotheses for this and the other three objectives that can help clarify them (page 8, lines 210-229).

I would refrain from emphasizing non-sig findings in the discussion

The non-significant statuses of the following have been clarified:

• The difference between conditions in transfer (page 13, lines 437-450).
• The weak association between enjoyment and general acquired knowledge in the control condition (Page 14, line 473).
• The associations between negative emotions and general acquired knowledge (Page 14, lines 480-482).

Reviewer 2 Report

Dear authors,

First of all, congratulations on your study.
The manuscript meets the scientific and academic quality criteria for its publication.
The research design, methodology, instruments and procedure are well described, the discussion and conclusions agreeing with results.

Author Response

RESPONSES TO REVIEWER 2

First of all, congratulations on your study.
The manuscript meets the scientific and academic quality criteria for its publication.
The research design, methodology, instruments and procedure are well described, the discussion and conclusions agreeing with results.

Many thanks for your comments and feedback.

Reviewer 3 Report

An interesting work about the introduction of invention activities in a university statistics class which highlights an effective technique to promote deep learning.

It would have been necessary to study other variables that can influence significative the learning process, such as students' learning potential, critical thinking or their intellectual level. The lack of studying the influences of cognitive and motivational variables on learning makes the interpretation of results to be done with more caution.

Author Response

RESPONSES TO REVIEWER 3

An interesting work about the introduction of invention activities in a university statistics class which highlights an effective technique to promote deep learning.

Many thanks for your comments and feedback.

It would have been necessary to study other variables that can influence significative the learning process, such as students' learning potential, critical thinking or their intellectual level. The lack of studying the influences of cognitive and motivational variables on learning makes the interpretation of results to be done with more caution.

Considering these variables is something of great utility for future investigations indeed. In the case of the present study, we have added the absence of these variables as a limitation and described how including them could increase internal and external validity of the study (page 15, lines 540-544).

Round 2

Reviewer 1 Report

Thanks for sending over the revised draft of the paper. I was quite impressed at the level of revisions by the authors and recommend going ahead with publication.