Which Epistemic Processes Occur When Pre-Service Teachers Reflect on Practitioners’ Misconceptions?

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Dear authors,

Thank you very much for this very interesting and easy-to-read paper. However, I noticed a few aspects that you might want to take another look at: 

1. There are minor typos - e.g. line 16: ? behind didactic, line 130: Cognition with a capital C, line 186: establish instead of establishes, line 262: participant - written in lower case at the beginning of the sentence, line 353 Epistemic Ideals, line 574f: check terms (2x PCK)
2. I find it irritating to dispute a study in which the theory is about anecdotal evidence with 11 participants and then with the contrast of three participants. In my opinion this is impossible. I am aware that a mixed-method approach with qualitative data that is then quantitatively evaluated is very time-consuming, but this would bring more methodological evidence into the paper. Then you could calculate an LCA instead of the contrast of three people and find out whether these people are suitable as examples for three groups. ALTERNATIVELY, it would also be clear to emphasize that this research is an exploratory approach - then this aspect would have to be highlighted at various points in the paper (e.g. abstract, method, discussion) and, if necessary, followed by another study with significantly more participants and a quantitative method.
3. You could include the type of school in the abstract and in the participant description - this is missing because I assume that the school was run in the German, structured school system and that the students may be studying for a specific type of school or age group.
4. In the first paragraph of the introduction one could write about the ongoing theory-practice shock, which continues to exist despite all the reflections in accompanying internships
5. The question in the further part is what role anecdotal evidence plays for the study itself and whether this part is relevant to the content. It wasn't entirely clear to me the connection between anecdotal evidence and misconceptions (approx. lines 99-116). Is anecdotal evidence being equated with misconceptions here? This is not the case.
6. By the end of this section (approx. lines 120-125), the expectation was raised when reading that this could also be an intervention study in which self-reflection is first taught and then experimentally examined. However, this is not the case over time. It might be helpful to get some clarity here in order to avoid this misunderstanding. 
7. Approx. lines 128-137. In the first section up to 132 and in the second section different components of epistemic cognition are mentioned. It is not clear at this point which ones are important later on and why these different divisions are mentioned or explained.
8. What is the difference between the AIR model and the apt-AIR model? Does that matter? Apt-AIR is mentioned once (line 162), the AIR model significantly more often.
   There is an overview article by Barzilai & Chinn (2004) doi: 10.1007/s10648-024-09927-5
9. Line 185: what type of influence (e.g. positive, negative, ...)
10. According to APA, statistical abbreviations should be italicized - M, SD, ...
    

In the methods section there are various questions, which are then reflected in the results and in the discussion:

1. Lines 241-257. Here an aspect in the material is presented in great detail and it is deduced that it can be both a CK and a PCK aspect. In the end there is one PPK aspect, one CK aspect, but two PCK aspects that can be criticized in the material. Can this be a reason why the participants comment differently on the different knowledge domains? (No one really recognized PCK & CK - see Table 2) I also wonder how obvious these aspects are to the participants and how much prior knowledge they have - are the participants even able to recognize and name the errors mentioned here? Was a prior knowledge test carried out, if necessary, or how was this aspect checked?
2. The participants focus on PPK. This is known from other empirical studies on coherence formation in teacher training courses. Students often perceive PPK as not relevant, but then, for whatever reason, they focus on this content when evaluating tasks. Perhaps these are more obvious than CK and PCK.
3. Don't think-aloud protocols require regular prompting to think aloud? This is mentioned in the limitations, but for me it represents an elementary part of the method. Here you could possibly research studies that did the same thing and found no differences to studies with repeated prompts for thinking out loud.

Part 3: „Coding & Results“

1. Was the material checked in advance to ensure that no other misconceptions apart from the aspects mentioned were contained or could be declared as such? Respectively to point 11: it was ensured that the same number of misconceptions were identified from all three knowledge domains - this is a complex point, but one that is not irrelevant to the interpretation of the results. 

2. From line 344 onwards, it is not immediately clear upon first reading which categories were derived deductively and which were derived inductively. Here, perhaps a numbering and then a direct reference to it could support the reader. Also to clarify which content is based on the publication of the authors (Authors B).

3. From line 360:
   1. Here it could be shown more clearly how the absolute numbers were relativized - that is, what they were relativized to. On the overall units or on the overall epistemic processes
   2. Were there actually units that weren't categorized because they didn't fit into the coding scheme for whatever reason?
   3. Here, inferential statistical methods could then be calculated with significantly more participants in order to test the differences statistically.
4. Line 389: 
   Please don't talk about significant differences. You haven't tested that.
5. In Table 5 the formatting is not always correct.  Examples:
   1. The font is sans serif, with serifs in the other tables.
   2. The “12” in TA11/Reference to knowledge…” is bold
   3. Please check the heading: Does this table really only contain relative values? To me that also looks like absolute values.
6. Table 5 & written results to this table:
   1. I sometimes don't understand the written results because some values ​​show more differences or similarities between the participants than described in the text. Here it would be worthwhile to emphasize again in the text that these are only exemplary examples, or why these results were described and not others, or to expand the results section.
   2. It might be helpful for the reader to know how much was written overall and what the % values ​​refer to.
7. Line 396ff (-532)
   Is there a way to display these results in a table to make them easier to understand? As a text, this is very comprehensive.
   1. OR: First describe what each of the three participants did and then contrast them with each other?
   2. In this part, some content/aspects are interpreted, some are not. How come?
   3. Here too, the topic of prior knowledge comes into play again and again: What if the participants simply didn't know this? E.g. line 504ff, 528ff
8. Line 408
   Is there a way to measure epistemic attitude?
   This might be helpful for future studies
9. Line 424:
   Does elaboration make sense or not? Here it is presented more negatively than anecdotal evidence. However, elaboration is very important in self-regulated learning and learning strategies because it anchors the content better in long-term memory. Please clarify why elaboration is viewed rather negatively here.

Discussion

1. Here, too, the question arises: Why is the anecdotal evidence so important for this paper when it is about misconceptions and whether these are noticed in the lesson plan.
   1. What is measured is an evaluation of the lesson plan and noticing or dealing with misconceptions contained therein
   2. How is anecdotal evidence measured?
2. Limitations
   1. Yes, they are all correct. But they are more serious than shown here in the limitations
3. Line 603
   In my opinion, it remains an exploratory approach that produces anecdotal evidence here due to the number of participants and the contrast of three participants pursued in RQ2
4. Line 634ff
   I strongly doubt the function of the think-aloud protocols without prompting. Something more needs to be said here about reliability and validity.
5. Line 656f
   What is described here is also part of the research on coherence in teacher training courses.
6. Line 668:
   It remains questionable until the end why you didn't notice the misconceptions or why you didn't mention them. Maybe it’s “simply” due to a lack of prior knowledge.

Bibliography
Please translate the German titles into English and put them in [ ] - see APA

Elaboration task
The task explicitly calls for elaboration. Why is this anecdotal evidence later interpreted negatively?

Lesson description
Can't the situation with the hairdryer give a wrong impression? The hairdryer not only melts the water, but also heats up the water and therefore expands. This would result in further distortion of the experimental results

Instructions Think Aloud

1. The task is very complex and long. How error-prone is the procedure? Are there comparative values ​​or previous studies on this?
2. How long did the participants sit on the task overall (M, SD, Min, Max)
3. The documentation is also very complex
4. Was it checked how the participants processed and followed the instructions?

Overall, it seems very complex to me and therefore a high cognitive load. How much capacity did the participants still have for the actual task if they were not reminded of the think-aloud protocol (even if the advantages of the asynchronous survey are very obvious).

I am sorry, but it keeps coming back to the same questions: Pre-knowledge, reliability, cognitive load, anecdotal evidence vs. missconceptions vs. elaboration, ...

Best regards!

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The article is well-crafted, with ideas presented in a clear and logically connected manner. The study addresses a pertinent and significant topic within the field of science education, specifically examining the reflections of prospective teachers as they engage with misconceptions held by practitioners. The research design is methodologically sound and executed without any identifiable errors or inconsistencies. However, my critique primarily pertains to the formatting aspects of the work:

• The structure of the article is complete, and although the conclusions section is missing, the journal indicates that it is not mandatory. The discussion is very complete.
• References and quotations are well written and relevant. There are some references that are not cited in the text, such as 41, all the references should be checked. Authors use the acronym of the name in the Richter citation (T. Richter & Maier, 2017).
• Tables are misnamed, authors have given Table 3 to two different tables, and of course the subsequent tables are numbered incorrectly.
• It is not clear to me why the abbreviation 'seg.' is used (I understand that it refers to the seconds in which the statement of one of the pre-service teachers is found in the think-aloud protocols). I wonder, if I'm not mistaken about the meaning of the ‘seg.’, readers will have access to the recordings via a link. If not, why do the "seg." appear?
• There is a question mark in the abstract that does not make sense in my opinion.
• There is a missing capital letter on line 262.
• Kappa (kn) according to Brennan and Prediger is used before been introduced. I don't know if it is well enough known to be used in this way.

Therefore, I consider this article suitable for publication, with minor modifications, as it is in line with the theme of this prestigious journal and will contribute to its strengthening.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Dear authors,
Thank you very much for the detailed revision and incorporation of the comments and suggestions. I think this makes the article much easier to understand.
With best regards
Reviewer