From Access to Impact: A Three-Level Model of ICT Use, Digital Feedback, and Students’ Achievement in Lithuanian Schools

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thanks for this. There is clear potential for publication here, but there are a number of changes that are needed before then. Please see the attached file for all of them. I've said "Major Revisions" as there are quite a few, the key one being a lack of supporting literature - and a seeming misunderstanding of the word 'empirical'. I also believe you need to create a diagram of your model. I hope you agree that these - and all the other revisions identified - will significantly strengthen your submission, and I hope to see a revised version soon.

Comments for author File: Comments.pdf

Author Response

Dear Reviewer,

Thank you very much for your careful reading of the manuscript and for the constructive and detailed feedback. We appreciate the reviewer’s recognition of the paper’s potential and fully agree that the revisions suggested significantly strengthen the submission.

In response to the comments provided in the attached review file, we have undertaken a thorough revision of the manuscript. Specifically, we have substantially strengthened the theoretical and empirical grounding of the study by expanding and systematising the supporting literature across all key sections. We have also clarified the empirical positioning of the study by revising the use of the term empirical, explicitly framing the work as the conceptual development of a three-level ICT model and an examination of whether empirical patterns in PISA 2022 data are consistent with this conceptual structure, rather than as a confirmatory or causal test of the model.

In addition, we have created and included a visual diagram of the proposed three-level ICT model, clearly illustrating the relationships between ICT access, ICT use, digital feedback, and student achievement. All other comments and suggestions raised in the review have been addressed carefully and systematically, with corresponding revisions made throughout the manuscript.

We believe that these changes have significantly improved the clarity, coherence, and contribution of the paper, and we are grateful for the reviewer’s guidance in this process. We hope that the revised version now meets the expectations for publication and look forward to your further consideration.

 

We attach the draft with our corrections highlighted in yellow.

Kind regards,
Julija Melnikova

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The research addresses: Which ICT ecosystem conditions (infrastructure access, perceived quality, and ICT learning practices) predict students’ exposure to digital feedback, and how does digital feedback relate to student achievement in mathematics, reading, and science in Lithuania (PISA 2022)?

A secondary question explored is whether ICTFEED acts as an achievement-enhancing or compensatory mechanism.  

The topic is relevant and timely because it examines ICT-mediated feedback during a national post-pandemic digital transition.  

It is original because:  

It proposes a 3-level structured model (Access → Use → Impact) instead of simple ICT-achievement correlations.  

It evaluates determinants of receiving digital feedback, a dimension rarely analysed in PISA secondary studies.  

It tests ICTFEED as an independent impact variable, revealing a negative but systematic association with achievement, interpreted as compensatory.

The gap addressed is: digital feedback usage is assumed to depend on ICT availability, but this causal chain had not been empirically tested for Lithuanian schools.    

 

The paper contributes:  

Evidence that ICT availability at home (ICTHOME) and school (ICTSCH) does not predict digital feedback exposure, challenging the infrastructure-first assumption.  

Identification that perceived school ICT quality (ICTQUAL) is the only significant infrastructure predictor of ICTFEED.  

Finding that inquiry-based ICT use (ICTENQ) and out-of-class ICT learning activities (ICTOUT) are the strongest predictors of ICTFEED.  

Demonstration that ICTFEED is negatively associated with achievement across all domains even after controlling for ICT access, quality, and digital leisure, suggesting a compensatory rather than causal effect. This advances ICT-integration theory by showing that feedback emerges from pedagogical ICT engagement, not device availability.    

 

Although the regression and sampling corrections are appropriate, I recommend:  

Clarify model implementation – the manuscript refers to SEM logic but uses sequential regression structural proxies. This should be explicitly justified as a hybrid structural test rather than full latent SEM.   Address multicollinearity risk – correlations between some indices (e.g., ICTHOME with ICTSCH_mean, ICTOUT with ICTFEED) are moderate-to-high; a VIF or tolerance discussion should be added.  

Justify the use of official PISA indices without re-validating them via CFA/MGCFA in this specific model.  

Discuss endogeneity / reverse-directionality of ICTFEED → Achievement, even if interpreted as compensatory.  

Differentiate feedback sources if item-level data permit (teacher vs peer vs automated), since effects may not be equivalent.  

Include model-coherence validity evidence, e.g., mediation logic confirmation or model-fit approximation from the structural paths.  

These additions would strengthen the scientific critique, interpretability, and reproducibility of the 3-level framework.    

 

The conclusions are consistent and statistically supported, particularly:  

ICTQUAL and ICTOUT show stable positive effects on achievement.  

ICTFEED shows significant negative standardized betas (−0.14 to −0.16) across math, reading, and science even after controls.  

Digital leisure (ICTLAISVALAIKIS) also shows negative effects (−0.19 to −0.21), supporting the model distinction between productive and non-productive ICT engagement.  

The conclusions correctly answer the research question by showing that:  

Infrastructure alone does not create digital feedback.  

Feedback emerges from inquiry-oriented and autonomous ICT learning engagement.  

ICTFEED is best interpreted as compensatory, because it is more prevalent among students needing support rather than increasing achievement.  

However, the authors should briefly acknowledge alternative interpretations (e.g., feedback quality, misalignment, or over-remediation), even if not the main conclusion.    

 

The reference base is relevant and theoretically aligned (formative assessment, digital competence, IBL, ICT integration, PISA SDA literature).  

I suggest:  

Verifying consistency of naming (e.g., “OECDa 2023”) and ensuring no merged or incomplete references remain in the final list.  

Adding 2–3 additional sources on negative ICT effects or compensatory feedback use to balance contextualization.  

• Additional comments on tables and figures  

Table 1 is informative but would benefit from indicating significance levels and discussing possible multicollinearity.  

Table 2 and 3 correctly report standardized effects and t-values, but clarity could improve by using consistent decimal notation (replace commas with dots for international readability).  

Figure 1 (if retained in the final manuscript) should ensure labels map exactly onto Access / Use / Feedback / Impact constructs to visually validate model structure.

Author Response

Dear Reviewer,

Thank you for these constructive and detailed suggestions, which helped us further improve the clarity, methodological transparency, balance, and presentation of the manuscript.

First, we carefully reviewed the entire reference list and all in-text citations to ensure full consistency of naming conventions and formatting. In particular, all OECD references were standardised (e.g., OECD, 2023a; OECD, 2023b where applicable), and the reference list was checked to remove any merged, incomplete, or incorrectly formatted entries.

Second, to balance the contextualisation of the findings, we added additional scholarly sources addressing negative or conditional effects of ICT use and compensatory patterns of feedback provision. These references strengthen the theoretical grounding of the interpretation of ICTFEED as a compensatory mechanism and situate the findings more clearly within the broader international literature.

In response to the methodological comments, we clarified the implementation of the proposed three-level ICT framework by explicitly stating that it follows an SEM-inspired conceptual logic, but is empirically implemented through sequential regression models as structural proxies, rather than as a full latent-variable SEM. The use of official PISA indices is now explicitly justified by referring to their established OECD validation and by clarifying that the study focuses on analysing structural relationships among validated constructs rather than re-estimating measurement models via CFA/MGCFA. We also expanded the discussion of endogeneity and reverse-directionality, acknowledging that the negative association between ICTFEED and achievement is most plausibly interpreted as compensatory rather than causal, given the cross-sectional design of PISA data. In addition, we clarified that ICTFEED is analysed as a composite indicator and noted limitations related to differentiating feedback sources, identifying this as an important direction for future research. Finally, we strengthened the articulation of model coherence by explicitly describing the mediation logic embedded in the access–use–feedback–impact structure.

Regarding the tables, Table 1 was revised to indicate statistical significance levels for correlation coefficients, and an explicit discussion of correlation strength and potential multicollinearity was added to the text. This discussion clarifies that, although some ICT variables show moderate correlations, these do not reach levels associated with problematic multicollinearity and do not compromise the stability of the regression results.

Tables 2 and 3 were revised to ensure consistent decimal notation throughout, replacing commas with dots in accordance with international publication standards to improve readability.

Finally, Figure 1 was revised to ensure that all labels and groupings map explicitly onto the Access, Use, Feedback, and Impact constructs described in the manuscript. Level headings, variable groupings, and arrow logic were aligned to visually validate the conceptual structure of the three-level ICT model.

We believe that these revisions have substantially improved the clarity, coherence, methodological rigour, and presentation quality of the manuscript, and we thank the reviewer for these valuable suggestions.

 

We attach the draft of the article with our responses highlighted in green.

 

Kind regards,

Julija

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thanks for this. I'm glad you agree that it has been strengthened by the changes. I would still eradicate the word 'empirical' (change for data-supported?).

Two minor things that still need changing before publication: you have two Section 5s - check the numbering. Also, put the limitations before the conclusion, or - as I said before - these will be what your readers leave with thinking about. Have them at the start of the concluding section, and then show how, "Despite these, this study is significant and original because..."

Author Response

Dear Reveiwer,

Thank you so much for the valuable comments! 

We absolutely agree with your suggestions.

Regarding the word "Empirical", yes, it was our methodological inaccuracy, and we have replaced it with "data supported" (as You suggest), and we totally agree that it is a more precise term to be used in the context of our research. Many thank for this!

Our apologies for some mistakes with the sections' numbering - corrected! Thank you for your this thorough review!

Also we have changes the order of Limitations and Conclusions sections (according to your comment - and this really seems a better and more logical way). Also we have slightly changed the beginning of the Conclusions section in order to better respond to Your suggestion.

Thank you very much for your review once again - it has helped not only to revise and improve our article but to gain new methodological insights and research paper writing skills! We appreciate this very much!

We attach the manuscript with the changes highlighted in yellow.

Thank you for the kind collaboration!

 

Kind regards,

Authors

 

Author Response File: Author Response.pdf