Developing and Validating an AI-TPACK Assessment Framework: Enhancing Teacher Educators’ Professional Practice Through Authentic Artifacts

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Overall, the study requires revision to provide greater clarity in the findings and enhance the transparency of the research. Check the input in the submitted file.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

The language needs to be developed to make it easier to read for native English speakers.

Author Response

Dear Reviewer 1,

Thank you for the time and effort you dedicated to reviewing my manuscript.

I have addressed all points raised in your review. My responses are detailed below, with all revisions marked in track changes in the manuscript.

Comment 1: Justification for the ADDIE Model

The use of the ADDIE model is mentioned in the literature review, expert validation, and artifact analysis. However, the methods section does not briefly explain the rationale for the method selection and how ADDIE is relevant to the instrument validation context. This can appear merely descriptive, rather than argumentative.

Response 1: Agree. I added a sentence to explicitly clarify why the iterative, multi-phased structure of the ADDIE model is optimally suited for instrument validation in this context, strengthening the rationale.

• Location: Methodology section.

Comment 2: Data Presentation (Tables and Graphs)

Tables of findings should be placed in the Appendix and replaced with graphs to more clearly illustrate the influence of the indicators.

Response 2: Yes. Thank you for this crucial suggestion. I have restructured the data presentation to enhance visual clarity and flow:

1. Tables Relocation and Condensation: I replaced the original lengthy tables (Table 1: Criteria from Literature Review, Table 3: Final Tool Structure) with compact summary tables in the main text. The full, detailed content of these tables has been moved to Appendix A.
2. Visual Enhancement (New Figures): I added four new figures (CVR, Distribution, ICC, and Mean Scores) to visually illustrate the core statistical findings, including correlations, thus enhancing transparency and clarity.

• Location: Findings section and Appendix A.

Comment 3: Absence of Limitations in Conclusions

The conclusion does not mention limitations which are crucial for demonstrating academic transparency and opening opportunities for further research. For example, consider sample size, cultural context, or limitations of the artifacts analyzed.

Response 3: Indeed, academic transparency requires a succinct summary of the study's limitations in the conclusion. I have added a sentence that summarizes the main limitations (sample composition and reliance on written artifacts), clearly directing future research.

• Location: Conclusions section.

Comment 4: Lack of Explicit Link to Research Questions

The conclusion emphasizes the contribution to the development and validation of AI-TPACK, but does not explicitly link it back to the research questions or objectives stated at the beginning of the study. This makes the conclusion feel more like a summary of the results than a conclusion that answers the research problem.

Response 4: Agree. I fully concur that a strong conclusion must explicitly link major findings back to the original research objectives. I have corrected this by completely revising the Conclusions section. A new introductory paragraph was added that directly states how the research has fulfilled its three primary objectives (RQ1, RQ2, and RQ3).

• Location: Conclusions section.

I believe that the revised manuscript has addressed all comments constructively and comprehensively. I hope that the paper is now suitable for publication in Education Sciences.

Sincerely,

Liat

 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

Thank you for the opportunity to review your manuscript titled “Developing and Validating an AI-TPACK Assessment Framework: Enhancing Teacher Educators’ Professional Practice through Authentic Artifacts.”

This paper offers a timely and valuable contribution to the field of teacher education and educational technology. The development of an artifact-based AI-TPACK assessment tool represents an original and much-needed advancement, addressing the limitations of self-reported instruments commonly used in this area. The manuscript is theoretically well-grounded, methodologically rigorous, and clearly aligned with current research priorities concerning AI integration in education.

The study’s structure is logical, the research design is well-executed, and the statistical validation is both transparent and convincing. The argumentation is coherent, and the implications for professional development and curriculum design are articulated clearly. Overall, the paper demonstrates strong scholarly merit and is very well suited for publication after minor revisions.

Minor Suggestions for Improvement

• Conciseness: The Introduction and Methodology sections could be slightly condensed to enhance readability and reduce redundancy (some ideas are repeated, especially regarding self-reporting limitations and the authenticity argument).

• Language polish: Minor stylistic edits are recommended to improve flow and consistency (for instance, uniform use of “AI-TPACK” and consistent punctuation).

• Table presentation: Consider simplifying or summarizing large tables (e.g., Table 1) to make key points more accessible to readers.

• Reference formatting: Review the reference list for typographical inconsistencies and ensure uniform citation style.

• Discussion focus: The Discussion could briefly highlight how this instrument could be adapted across different cultural or institutional contexts, which would strengthen its practical relevance.

With these minor adjustments, the manuscript will be both more concise and more accessible to an international readership.

 

Author Response

Dear Reviewer 2,

Thank you for the time and effort you dedicated to reviewing manuscript: "Developing and Validating an AI-TPACK Assessment Framework: Enhancing Teacher Educators' Professional Practice through Authentic Artifacts."
I have addressed all points raised in your review. The responses are detailed below, with all revisions marked in red in the manuscript.

Suggestion 1 & 2: Conciseness and Language polish

The Introduction and Methodology sections could be slightly condensed to enhance readability... Minor stylistic edits are recommended to improve flow and consistency...

Response S1 & S2 (Combined): Agree. I performed an additional comprehensive language and stylistic edit across the entire manuscript. This process involved reducing redundancy, highlighting core arguments, minimizing repetition, and ensuring stylistic consistency (e.g., uniform use of the term 'AI-TPACK' and punctuation), thereby enhancing overall readability and conciseness in the Introduction and Methodology.

• Location: Introduction, Methodology, and Throughout the manuscript.

Suggestion 3: Table presentation

Consider simplifying or summarizing large tables (e.g., Table 1) to make key points more accessible to readers.

Response S3: Agree. In line with the comprehensive revision of the Findings section, all large tables Ire converted into figures and moved to Appendix A to improve accessibility and flow in the main text.

• Location: Findings section and Appendix A.

Suggestion 4: Reference formatting

Review the reference list for typographical inconsistencies and ensure uniform citation style.

Response S4: Agree. I performed a detailed review and standardization of the entire reference list to ensure uniform citation style and correct any minor typographical inconsistencies.

• Location: References section.

Suggestion 5: Discussion focus

The Discussion could briefly highlight how this instrument could be adapted across different cultural or institutional contexts, which would strengthen its practical relevance.

Response S5: Thank you. I added a sentence to the Discussion section emphasizing the potential for this validated artifact-based tool to be adapted across diverse cultural and institutional settings, thereby strengthening its practical implications.

• Location: Discussion section.

I believe that the revised manuscript has addressed all comments constructively and comprehensively. Hope that the paper is now suitable for publication in Education Sciences.

Sincerely,

Liat

 

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript develops an assessment method for the AI-TPACK framework. Overall, the research is well-designed and the manuscript is clearly written. The proposed AI-TPACK assessment method offers valuable contributions to both the education system and the research field. The content is highly aligned to the major theme of the journal.

Some minor comments and suggestions are listed in the following.

1. Two systematic reviews are conducted and presented in the manuscript. It is suggested that all the academic databases used for the review process can be described.

2. Please adjust the size of Figure 3, as part of the figure is currently cropped.

3. Please provide more information (e.g., educational background, academic degree, current position, professional experience, and so on) about the five experts. A table might be used to better present the information. 

4. Based on Lawshe's method, the CVR threshold for five experts seems to be 0.99. However, a value of 0.62 is selected and used in the manuscript. Please provide more explanations for readers about this design choice.

5. Please provide summary information or general characteristics of the 60 teacher educators who participated in the field testing.

6. Based on the manuscript, teacher educators provide 60 pedagogical artifacts for analyzing their skill levels and patterns of AI integration into their teaching. It is recommended that the pedagogical artifacts can be introduced in more detail. Examples of these artifacts and the associated AI-TPACK assessments can be presented in the manuscript as well. Thus, readers can get insights into the important artifacts which can be used for evaluation in the future.

Author Response

Dear Reviewer 3,

Thank you for the time and effort you dedicated to reviewing manuscript: "Developing and Validating an AI-TPACK Assessment Framework: Enhancing Teacher Educators' Professional Practice through Authentic Artifacts."

I have addressed all six minor suggestions raised in your review. My responses are detailed below, with all corresponding revisions implemented in the manuscript.

Comment 1: Description of Academic Databases

It is suggested that all the academic databases used for the review process can be described.

Response: Agree. I have revised the Methodology section (Phase 2) to explicitly list the specific academic databases used for both systematic reviews (Ib of Science, Scopus, ERIC, and Google Scholar), enhancing the transparency of the search process.

• Location: Methodology section (Phase 2).

Comment 2: Figure 3 Size

Please adjust the size of Figure 3, as part of the figure is currently cropped.

Response: OK, I have optimized Figure 3 (PRISMA flow chart) but editors will ensure it is resized appropriately and fully legible in the final manuscript layout.

• Location: Methodology section (Figure 3).

Comment 3: Information about the Experts

Please provide more information (e.g., educational background, academic degree, current position, professional experience, and so on) about the five experts. A table might be used to better present the information.

Response: Thank you for this comment. I have added a new table (Table 1) in the Methodology section (Phase 3) that summarizes the key professional attributes of the five expert validators, thereby enhancing the transparency and credibility of the content validation process.

• Location: Methodology section (Phase 3).

Comment 4: CVR Threshold Justification

Based on Lawshe's method, the CVR threshold for five experts seems to be 0.99... Please provide more explanations for readers about this design choice.

Response: Thank you for this insightful comment, which highlights an important nuance in applying Lawshe's method. Upon revisiting Lawshe's original table (1975), the minimum CVR threshold for a 5-member expert panel is indeed 0.99 at the one-tailed significance level (p < 0.05), requiring near-unanimous agreement to exceed chance levels. The value of 0.62, which I initially referenced, corresponds to a 10-member panel and was a misapplication on my part—thank you for pointing this out. Recent recalculations by Ayre and Scally (2014) using exact binomial probabilities adjust this to 1.00 for panels of 5-7 experts, emphasizing the stringency for small panels. In our study, while the overall CVR (0.86) and CVI (0.91) exceed common benchmarks, some items fell slightly below the strict threshold (e.g., CVR=0.60). These were retained based on their high mean relevance ratings (M=3.6 on a 1-4 scale), theoretical importance to the AI-TPACK construct, and expert consensus during the group discussion, as is common in practical applications of the method (e.g., when combining with qualitative feedback). To enhance clarity, I have added an explanatory note in the manuscript (Phase 3 and Figure 4 caption) detailing this decision and citing the relevant sources. This ensures transparency while maintaining the instrument's robustness.

Comment 5: Teacher Educators' Characteristics

Please provide summary information or general characteristics of the 60-teacher educators who participated in the field testing.

Response: Yes, I have added a summary in the Methodology section (Phase 4) describing the general characteristics of the 60 teacher educators, focusing on their institutional roles and the diverse contexts from which the artifacts are drawn.

• Location: Methodology section (Phase 4).

Comment 6: Artifact Detail and Examples

It is recommended that the pedagogical artifacts can be introduced in more detail... Examples of these artifacts... can be presented in the manuscript as Ill.

Response:  I acknowledge the importance of providing insight into the artifacts. I have significantly enhanced the detailed description of the 60 artifacts in the Methodology section (Phase 4). To illustrate the application and diversity, I have also integrated brief, illustrative examples (e.g., specific AI tool usage) directly into the text. I have chosen not to include lengthy, full-page artifact examples in the Appendix due to concerns regarding manuscript length and the difficulty of representing the full diversity of 60 unique artifacts with only one or two examples. Hope its acceptable.

• Location: Methodology section (Phase 4)

I believe that the revised manuscript has addressed all comments constructively and comprehensively. I hope that the paper is now suitable for publication in Education Sciences.

Sincerely,

Liat

 

Reviewer 4 Report

Comments and Suggestions for Authors

This manuscript presents a well-structured and relevant study focused on the development and validation of an AI-TPACK assessment instrument. Overall, the paper demonstrates strong theoretical grounding and methodological rigor, but there are a few areas where clarity and alignment between methods and analyses could be improved.

Introduction and Literature Review

• Good literature review grounded in TPACK and AI literature. Good job establishing a gap for this study.

Methods

• Good method description overall. In particular, the detailed process documentation for the systematic literature reviews is good and establishes a strong foundation to identify and develop the proposed AI-TPACK assessment instrument.

• However, in Phase 5 of the methods section, I’m a bit confused by the tests performed to assess validity. The authors describe tests for normality, linearity, homoscedasticity, and multicollinearity, as well as Cohen's d and eta-squared for effect size, all of which imply use of parametric regression or ANOVA. However, this section (and the findings) only describes reliability and validity testing using inter-rater reliability (and qualitative analysis), not predictive modeling. If no parametric tests are used, these assumption checks and effect size calculations are irrelevant.

Findings

• In the findings, what test was used to generate Table 6 and the related analysis? From the results, it appears to be a Pearson’s correlation, but this is never stated, and the procedures and justifications for using Pearson’s r are not specified. Importantly, Pearson’s r assumes interval-level data, but the previous sections imply the ratings are ordinal. How the data were treated for this analysis should be described and justified.

Discussion

• Good discussion related back to the research questions and literature.

Writing and Style

• Some typos (e.g., Line 126, “AI-TRACK”).

• Some repetitive phrasing (e.g., Line 499: “substantial agreement among raters, with Cohen’s Kappa = 0.704, indicating substantial agreement”).

Author Response

Dear Reviewer 4,

Thank you for your constructive and insightful comments regarding the manuscript. I appreciate your positive assessment of the study's theoretical grounding and methodological rigor. I have carefully addressed your suggestions to improve clarity and alignment between the methods and analysis, as detailed below.

Reviewer Comment	Agreement and Revision Made	Location
1. Irrelevant Assumption Checks (Methodology): The assumption checks (normality, linearity, homoscedasticity) and effect size calculations (Cohen's d, eta-squared) are irrelevant as the final analysis only reports reliability and correlations.	Yes, you are right. The methodology section was confusingly over-inclusive. We have removed all references to the irrelevant parametric assumption checks (normality, multicollinearity, linearity, homoscedasticity) and unnecessary effect size calculations (Cohen's d, eta-squared).	Methodology (Phase 5: Evaluation)
2. Justify Correlation Test (Table 6): The test used for the correlations (now Figure 8) is not stated, and the use of Pearson's r must be justified, as the ratings imply ordinal data, not interval data.	Agree. The statistical method and justification were not explicitly stated. I have added clarification to the text (before Figure 8) stating that I used Pearson’s correlation coefficient {r}. I further justified this by explaining that the data analyzed were composite mean scores, which are treated as continuous, approximating interval-level data for correlational analysis, consistent with psychometric practice.	Findings
3. Writing and Style: Some typos (e.g., Line 126, “AI-TRACK”) and repetitive phrasing (e.g., Line 499).	OK, I performed a final, dedicated stylistic review of the manuscript and corrected the typo "AI-TRACK" to "AI-TPACK" and revised some sentence to eliminate the redundant reference to "substantial agreement," thereby improving flow and conciseness.	Throughout the Manuscript

I believe these revisions fully address your concerns regarding methodological clarity and ensure that the paper is concise and accessible.

Sincerely,

Liat

Author Response File: Author Response.docx