The Influence of Teachers’ Professional Development Activities on the Factors Promoting ICT Integration in Primary Schools in Mongolia
Round 1
Reviewer 1 Report
The subject of the research is interesting and may have an impact on the scientific community. However, a series of recommendations must be addressed to improve the scientific quality of the paper:
Point 1: the bibliography used should be updated. Although the theoretical framework is consistent, the field of educational technology advances continuously and rapidly. Therefore, studies between 2015-2019 should be consulted to update the overview of the work, especially in the discussion. An outdated bibliography is used.
Point 2: the paper does not meet the standards of the publication in terms of format. The section of bibliographic references is not found nor appears correctly in the text, since the dates must be substituted by numbers between [].
Point 3: it is not clear if it is a longitudinal study, it is not specified, but a time range appears between 2012 and 2016. Would it be longitudinal ?.
In the methodology part there are serious deficiencies, a lot of bibliography is used to justify the research method: we speak of Cronbach's alpha, but the coefficient of the questionnaires used does not appear. What is the value of Cronbach's alpha? How has the content of the questionnaire been validated? Has a previous expert judgment been made?
Regarding the qualitative part: how has the information extracted from the interviews and the discussion forums been analyzed? By content analysis? No information appears in this regard.
Point 4: in the analysis of results there is a basic problem, the samples selected from year 212 and 2016 are not balanced, they are not equal. Therefore, the acceptance or non-acceptance of the hypotheses may have type I and / or II errors. There is no normal distribution of the sample, as a consequence there will be no equality of variances. Something that is clear when viewing the tables in the supplementary material, Levene's test indicates that the normal distribution of the sample does not exist. The sample of participants from both 2012 and 2016 should be equalized, since in this last year the number of participants is greater, for which the means and standard deviations will vary. Therefore, there is a greater probability that the participants of the year 2016 have better scores than those of 2012, not because it is so, but because they are more.
Author Response
Reviewer 1:
The subject of the research is interesting and may have an impact on the scientific community. However, a series of recommendations must be addressed to improve the scientific quality of the paper:
Point 1: the bibliography used should be updated. Although the theoretical framework is consistent, the field of educational technology advances continuously and rapidly. Therefore, studies between 2015-2019 should be consulted to update the overview of the work, especially in the discussion. An outdated bibliography is used.
Respone:
Thank you for your comment. I have updated the bibliography in literature review and discussion sections with more recent literature.
Point 2: the paper does not meet the standards of the publication in terms of format. The section of bibliographic references is not found nor appears correctly in the text, since the dates must be substituted by numbers between [].
Response:
Thank you for pointing out this fundamental mistake. The list of references is added. In terms of the reference style, I understand that you referred to IEEE style. However, this paper adopts the APA style which is formatted as (authors, year) or authors (year).
Point 3: it is not clear if it is a longitudinal study, it is not specified, but a time range appears between 2012 and 2016. Would it be longitudinal?.
In the methodology part there are serious deficiencies, a lot of bibliography is used to justify the research method: we speak of Cronbach's alpha, but the coefficient of the questionnaires used does not appear. What is the value of Cronbach's alpha? How has the content of the questionnaire been validated? Has a previous expert judgment been made?
Regarding the qualitative part: how has the information extracted from the interviews and the discussion forums been analyzed? By content analysis? No information appears in this regard.
Response:
This is a longitudinal study, I have included the information in section 1.4: Study objective.
In terms of the result of Cronbach’s Alpha, section 3.2 factor analysis is updated to show the result of Cronbach’s Alpha test for each latent variable. I have included more details on how the questionnaire was developed, how the factor analysis result was interpreted to name new factors.
Regarding the qualitative analysis, I have added explanations in section 3.1: Data source.
Point 4: in the analysis of results there is a basic problem, the samples selected from year 212 and 2016 are not balanced, they are not equal. Therefore, the acceptance or non-acceptance of the hypotheses may have type I and / or II errors. There is no normal distribution of the sample, as a consequence there will be no equality of variances. Something that is clear when viewing the tables in the supplementary material, Levene's test indicates that the normal distribution of the sample does not exist. The sample of participants from both 2012 and 2016 should be equalized, since in this last year the number of participants is greater, for which the means and standard deviations will vary. Therefore, there is a greater probability that the participants of the year 2016 have better scores than those of 2012, not because it is so, but because they are more.
Response:
Thank you for pointing out the problem of normal distribution and no equality of variances. The survey was distributed to all primary school teachers in the four counterpart provinces and one district of the capital city in Mongolia. The seeming unequal sampling was not a technical issue, since the survey samples include all the primary school teachers of these provinces and district. The inequality of responses could have arisen from the teachers’ unfamiliarity with the questionnaire in 2012, resulting largely invalid responses. In fact, Levene’s test results indicate that there is no homogeneity of variances for variables “Professional competency in educational use of ICT” and “Collaboration for ICT integration”. Therefore, for these two variables, unequal variance t-test (Welch t-test) result was utilized to interpret the findings. Specifically, for these two variables, the information in "Equal variances not assumed" row of Appendix 2 was used.
Reviewer 2 Report
The manuscript is a regional report about the progress of teacher professional development for ICT. As a report, it is a fairly well-written report. The theoretical significance is limited. The factor "School support on ICT related activities for student-centered education" should be changed to "Progress towards student-centered education" to reflect the items better. The finding for this factor would then be consistent with recent research such as those by reported internationally (For e.g.
Pringle, R. M., Dawson, K., & Ritzhaupt, A. D. (2015). Integrating Science and Technology: Using Technological Pedagogical Content Knowledge as a Framework to Study the Practices of Science Teachers. Journal of Science Education and Technology, 24(5), 648-662.)
Report about how the face validity of the items was established needs to be included.
Author Response
Reviewer 2:
The manuscript is a regional report about the progress of teacher professional development for ICT. As a report, it is a fairly well-written report. The theoretical significance is limited. The factor "School support on ICT related activities for student-centered education" should be changed to "Progress towards student-centered education" to reflect the items better. The finding for this factor would then be consistent with recent research such as those by reported internationally (For e.g.
Pringle, R. M., Dawson, K., & Ritzhaupt, A. D. (2015). Integrating Science and Technology: Using Technological Pedagogical Content Knowledge as a Framework to Study the Practices of Science Teachers. Journal of Science Education and Technology, 24(5), 648-662.)
Report about how the face validity of the items was established needs to be included.
Response:
Thank you for your comments. I have changed the factor "School support on ICT related activities for student-centered education" to "Progress towards student-centered education". I have added the above-mentioned recent research in discussion.
In terms of the face validity, I have added a brief description on how latent variables are validated in section 3.2: Factor analysis. I have also added the result of Cronbach’s Alpha test to show the reliability of constructing items of each latent variable.
Reviewer 3 Report
The article analyzes how teacher training programs in Mongolia focusing on ICT use in educational practices had influenced on teacher level factors that are important for ICT integration in educational activities, through the comparative study of data collected between 2012 and 2016. Results show that professional development activities for primary school teachers in Mongolia had positive influences on six teacher level factors: professional competency in educational use of ICT, collaboration for ICT integration, benefits on use of ICT, autonomy to innovate, recognition as a professional, and skills and practices in educational use of ICT.
Below are general and specific comments to improve the content and readability of the manuscript.
Broad comments:
The article does not follow the citation rules proposed by the journal. It is recommended to review and adapt to them.
Specific comments:
7/273: The Data Source section does not include relevant information about the surveys carried out. The sample of 826 teachers from four different geographical provinces, was it random or convenience? Did the authors had permission from ethics committees to carry out the research? How were the surveys conducted, online or in person? It would also be convenient to give some examples of the questions belonging to the three sections indicated.
It also refers to a qualitative analysis with 358 interviews, however not enough data is given on how these were carried out, for how long, if they were semi-structured interviews, etc.
16/571: Although the authors note that "Qualitative data collected from interviews and fieldwork 561 observations were used to interpret the results", a more detailed description of the results of these interviews is missing in the Results section.
Author Response
Reviewer 3:
The article analyzes how teacher training programs in Mongolia focusing on ICT use in educational practices had influenced on teacher level factors that are important for ICT integration in educational activities, through the comparative study of data collected between 2012 and 2016. Results show that professional development activities for primary school teachers in Mongolia had positive influences on six teacher level factors: professional competency in educational use of ICT, collaboration for ICT integration, benefits on use of ICT, autonomy to innovate, recognition as a professional, and skills and practices in educational use of ICT.
Below are general and specific comments to improve the content and readability of the manuscript.
Broad comments:
The article does not follow the citation rules proposed by the journal. It is recommended to review and adapt to them.
Response:
Thank you for pointing out this basic issue, I have added the bibliography to the manuscript.
Specific comments:
7/273: The Data Source section does not include relevant information about the surveys carried out. The sample of 826 teachers from four different geographical provinces, was it random or convenience? Did the authors had permission from ethics committees to carry out the research? How were the surveys conducted, online or in person? It would also be convenient to give some examples of the questions belonging to the three sections indicated.
It also refers to a qualitative analysis with 358 interviews, however not enough data is given on how these were carried out, for how long, if they were semi-structured interviews, etc.
Response:
I have added the corresponding information regarding the surveys carried out in section 3.1: Data source. In summary, population sampling was utilized in the selected four provinces and one district in the capital city. Permission was given by the local education administration bureau to conduct survey and research on teachers’ professional development. The surveys were conducted by post. I have added a couple of example questions for each section in the paragraph.
I have added the detailed descriptions regarding the semi-structured qualitative interviews with school administrators and focus group discussions with school teachers.
16/571: Although the authors note that "Qualitative data collected from interviews and fieldwork 561 observations were used to interpret the results", a more detailed description of the results of these interviews is missing in the Results section.
Response:
Thank you for your comment. Apologize for the lack of explanation on the purpose of the interviews, focus group discussions and fieldwork. The purpose of these efforts to collect qualitative data is to collect examples, practices, and teachers’ interpretations of the quantitative data analysis results. The raw results were therefore not a primary focus on this study, hence not reported in detail in section 4: Data analysis results. However, the qualitative data were utilized extensively in section 5: Discussion. Interview and focus group discussion results were utilized to interpret the changes in teachers’ perception on factors that are important for ICT integration in education.
Reviewer 4 Report
The paper showes an interesting topic, specially in the Mongolian context, as one of challenging for which the Government has supported the innovation and improvement of Primary School in rural zones through the use of ICT and the teacher training. However, there are many and significant lacks that do not let to be published in this journal, due to the absence of scientific requirements of quality in these type of papers. These are the following:
- Title needs to be more concrete, it is so general. So, it could be very necessary to specify clearly the relations between the factors promoting ICT (what they are) and in which type of activities, how is the influence of the factors to them. At the sane line, the keywords would need to be revised to concrete more the content of the paper.
- There is no abstract
- It is used a terminology in the framework of technologies inappropiated ("new technologies"), in line 41, pag. 1 and line 493, pag. 14.
- The structure of the paper should be reorganized all the sections of background and literature review following the same and main idea of the text (coherence, connection), from general to specific view, the main and secondary ideas, with more updating of citations about the last five years, more recent and significant contributions. On the other hand, it would be very suitable to add scheme, some charts to support the most relevant information written. It is a mixture of information, yuxtaposed but not linked very difficult to understand it (styles of leadership, perceptions, skills...). All of them are so different and it is not justified the results of specific studies have found about it and why it is established this relation between them; for instances, acording to the topic, could be more interesting to consider the digital competence, as the reference of European Model: http://formacion.intef.es/pluginfile.php/49681/mod_imscp/content/7/modelo_europeo_de_competencias_digitales.html
Not only, a description of the origins about the framework of Mongolian policy of ICT, but also, a synthesis of overcomes, aims got and limitations found would be more convenient.
So, in the section 1.2. it is about the context of the study and it would be convenient to add in empirical part, as the place where study has been developed.
- The study objective would need to be shown separately by specific aims and concrete deeply the association between different variables. Moreover, the study covers only from the period 2012 until 2016, and it should be updating data to 2018. Thus, it is an important handicap acording to the relevance of the results.
- The section of "methodology" (line 272) should be titled as "design of researching", as well as, in this part, there is no information about the method, methodology used, what the sampling is, how the instrumen- likert scale- has been designed "ad hoc" or adapted of others, what the validity has been obtained; how many interviews and focus group have used and how the choice has been.
- All the tables needs to be indicated the origin.
- If there is a general aim and the study has a descriptive nature in relation to the instruments used, probably, it is not necessary to add hypothesis.
- Analysis of data is only focus on quantitative results from the questionnaire, the data of interviews and focus group do not analysed and they are not clear about the way in which these factors influence in professional training and student outcomes. It would be very important to specify the software used, organizing relevant information to the general and specific aims.
- The discussion and conclusions are so general and not derivated to data. It showes lack of evidences to compare and discuss theory approaches (the sampling is the same in both period, is there a longitudinal study? In this case, it would be convenient to design other type of study based on the changes in this period and what are the possible reasons). For instance, "teacher perceived higher autonomy to innovate after a series of a teacher training activities" (how? where is the evidence? which type of activities? in relation to?....) (line 491, page 14). On the other hand, if we speak about teacher training, it could be very relevant to describe the training system/model for teachers in Mongolia, as well as, the new pedagogy (it is advised to review literature about active methdologies with the use of ICT through flipped classroom, blended, mobile learning, learning by projects, peer tutoring an so on).
- It has not a prospective or future directions of the study
- References are not included in the paper so it is not adapted to APA rules.
Author Response
Reviewer 4:
The paper showes an interesting topic, specially in the Mongolian context, as one of challenging for which the Government has supported the innovation and improvement of Primary School in rural zones through the use of ICT and the teacher training. However, there are many and significant lacks that do not let to be published in this journal, due to the absence of scientific requirements of quality in these type of papers. These are the following:
- Title needs to be more concrete, it is so general. So, it could be very necessary to specify clearly the relations between the factors promoting ICT (what they are) and in which type of activities, how is the influence of the factors to them. At the sane line, the keywords would need to be revised to concrete more the content of the paper.
Response:
Thank you very much for your comment. I understand that you are referring to correlational type of study. However, this paper aims to identify the factors that promotes teachers’ educational use of ICT from a set of questions, and observe how teacher training activities may have influenced on these factors. Therefore, it is not the scope of this study to identify what are the factors affecting teachers’ educational use of ICT, nor the relations among these factors. In short, the core interest is to observe how teacher training activities have influenced factors that are important for teachers’ educational use of ICT. I have altered the title accordingly to emphasize this point. I have updated the keyword list to reflect the contents better.
- There is no abstract
Response:
I have added the abstract in the main text.
- It is used a terminology in the framework of technologies inappropiated ("new technologies"), in line 41, pag. 1 and line 493, pag. 14.
Response:
I have changed “new technologies” to “ICT” to be specific and consistent.
- The structure of the paper should be reorganized all the sections of background and literature review following the same and main idea of the text (coherence, connection), from general to specific view, the main and secondary ideas, with more updating of citations about the last five years, more recent and significant contributions. On the other hand, it would be very suitable to add scheme, some charts to support the most relevant information written. It is a mixture of information, yuxtaposed but not linked very difficult to understand it (styles of leadership, perceptions, skills...). All of them are so different and it is not justified the results of specific studies have found about it and why it is established this relation between them; for instances, acording to the topic, could be more interesting to consider the digital competence, as the reference of European Model: http://formacion.intef.es/pluginfile.php/49681/mod_imscp/content/7/modelo_europeo_de_competencias_digitales.html
Response:
I have updated the literature in the literature review section reflecting updates in the field in recent years, specifically more studies from 2015. I understand that all of the factors I mentioned in literature are different, which is part of the nature of the literature review in this study. To be more specific, the literature review searches the previous studies about important factors for teachers’ educational use of ICT with regards to professional development activities. This study is not attempting to draw relations among these factors. Instead, the literature review of the study tries to identify factors important for teachers’ educational use of ICT as a basis to select relevant question items from the survey conducted as part of the professional development project. After that, conduct factor analysis to identify latent variables in this study context, and compare teachers’ perception on these factors in 2012 and 2016 respectively, so that the influence of teachers’ professional development can be observed.
- Not only, a description of the origins about the framework of Mongolian policy of ICT, but also, a synthesis of overcomes, aims got and limitations found would be more convenient.
So, in the section 1.2. it is about the context of the study and it would be convenient to add in empirical part, as the place where study has been developed.
Response:
In section 1.2: Mongolia country background and its ICT polices in education, the country background of Mongolia is given, followed by the review of policy development from 2000s.
- The study objective would need to be shown separately by specific aims and concrete deeply the association between different variables. Moreover, the study covers only from the period 2012 until 2016, and it should be updating data to 2018. Thus, it is an important handicap acording to the relevance of the results.
Response:
Thank you for your concern over the research objective and design. I understand your comment on the importance to explore the relationships among variables using correlational design, which however is not the specific focus of this paper. In fact, our research team has published a paper regarding the factors affecting teachers’ perception on use of ICT for student-centered education, which can be accessed using the link below: https://eric.ed.gov/?id=EJ1178363. Regarding the study period, the reason to study between 2012 and 2016 is that the surveys were carried out as part of the teachers’ professional development in Mongolia. This reflects the practical aspect of survey implementation. I have added the explanation in section 3.1: Data source.
- The section of "methodology" (line 272) should be titled as "design of researching", as well as, in this part, there is no information about the method, methodology used, what the sampling is, how the instrumen- likert scale- has been designed "ad hoc" or adapted of others, what the validity has been obtained; how many interviews and focus group have used and how the choice has been.
Response:
I have changed the section title to “Research design”. I have added more information regarding data source in terms of both quantitative and qualitative and samples. This study utilizes factor analysis and t-test to reach study objective.
- All the tables needs to be indicated the origin.
Response:
All the tables come with table number, which are used to be referred to in the main text.
- If there is a general aim and the study has a descriptive nature in relation to the instruments used, probably, it is not necessary to add hypothesis.
Response:
The hypotheses are written out as part of the t-test.
- Analysis of data is only focus on quantitative results from the questionnaire, the data of interviews and focus group do not analysed and they are not clear about the way in which these factors influence in professional training and student outcomes. It would be very important to specify the software used, organizing relevant information to the general and specific aims.
Response:
I have added the information on software used towards the end of section 3.1: Data source. Apologize for the lack of explanation on the purpose of the interviews, focus group discussions and fieldwork. The qualitative data were collected, analyzed and used for interpretation in discussion part. The purpose of these efforts to collect qualitative data is to collect examples, practices, and teachers’ interpretations of the quantitative data analysis results. The raw results were therefore not a primary focus on this study, hence not reported in detail in section 4: Data analysis results. However, the qualitative data were utilized extensively in section 5: Discussion. Interview and focus group discussion results were utilized to interpret the changes in teachers’ perception on factors that are important for ICT integration in education.
- The discussion and conclusions are so general and not derivated to data. It showes lack of evidences to compare and discuss theory approaches (the sampling is the same in both period, is there a longitudinal study? In this case, it would be convenient to design other type of study based on the changes in this period and what are the possible reasons). For instance, "teacher perceived higher autonomy to innovate after a series of a teacher training activities" (how? where is the evidence? which type of activities? in relation to?....) (line 491, page 14). On the other hand, if we speak about teacher training, it could be very relevant to describe the training system/model for teachers in Mongolia, as well as, the new pedagogy (it is advised to review literature about active methdologies with the use of ICT through flipped classroom, blended, mobile learning, learning by projects, peer tutoring an so on).
Response:
I have mentioned about the longitudinal nature of this study in section 1.4: Study objective and section 3.1: Data source.
Regarding your comment on how "teacher perceived higher autonomy to innovate after a series of a teacher training activities", I have reported in the paragraph that school leaders themselves were aware of the advantage and thus encouraged teachers to be innovative in its use (hence teachers have more autonomy to innovate). In terms of the specific activities in relation to higher autonomy to innovate, development of training and teaching materials using ICT has created the culture of sharing and collaborative learning further upgrading teachers’ knowledge and skills.
The study did describe the policy on teacher training and introduction of new pedagogy, pecially student-centered education in background, literature review, and discussion section.
- It has not a prospective or future directions of the study
Response:
I have added the future direction of the study.
- References are not included in the paper so it is not adapted to APA rules.
Response:
Thank you for pointing out this fundamental issue, I have added the reference list towards the end.
Round 2
Reviewer 1 Report
Great Job!. Congratulations
Reviewer 4 Report
The paper has been improved deeply and according to the proposals in the review.
