Instructional Design Models for Pervasive Learning Environment: Bridging Formal and Informal Learning in Collaborative Social Learning
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe article presents an innovative course design and its evaluation. The need for innovation arises from the finding that there is a gap between the expectations and the reality in the training of students. I agree with this observation and I find the presented learning environment (LE) both interesting and relevant.
I recommend a rework of the abstract. An abstract should be a balanced mix of summary and report of results, while also being as concise as possible. In this case I recommend to point to the context, the concrete educational model (PLE), the concrete design model, and the evaluation. Then it can be concluded that the developed course design was well received and some lessons were learned...
l. 12: awareness: readiness?
l. 14: which resources?
l. 23: or validity -> and validity.
I recommend to restructure the introduction. After the motivation part, I would appreciate a little more context on why this particular course was chosen for the experiment. The same holds for the selection of the educational model -- why this and not something different (there are plenty of alternatives on the market...). Also, I would find it easier to read and to understand, if you state more direct that you pick the educational model from [17] and try to adapt it the course mentioned above.
The paragraph in l. 90 states a number of goals. I feel it would be worthwhile to explicitly discuss (in a conclusion section) whether these goals were achieved or nor (or to which degree).
l. 63: which interaction?
l. 64: "wane to struggle"???
l. 102: "through technology"(?) -> "using technological solutions"?
The Materials and Methods section could be shortened. Your approach is pragmatic, I think you could be straight on this (there is no "theoretical method" (l. 113) involved here). I would strongly encourage to explain your methodological considerations and decisions instead of just refering to some literature.
the paragraph in l. 116 is very unclear: which publication (l. 117), sentence in l. 120... instead, you could state what was done and why it was done. Why ADDIE? What are "jar materials" (table 1)
Subsec 2.5: Why the same questionaire for all groups (or wasn`t it the same)?
Table 2: I feel, the learning aspect is most important and should be first...
Table 3: "Course opening" -> "introduction". What can I imagine "course closing" to be?
I feel sec 3.1 should also go into sec 2 (since it is a design principle).
l. 211 For the "five phases" please point to table 5! (Gave me a headache...)
Figure 2: Drop the "basic theory" corner or explain how it is relevant!
The description beginning in l. 233 is unclear to me. I think, the key problem is the term "mapping program". Actually, you are describing the course structure ("topics") and you explicitly specify how the five phases are implemented in each area. I recommend the term "program map" instead!
The program map would also be sec 2...
Table 6 "material link at p-learning" ???? No idea what this means... "Case examine" -> examination? "Community of practices sharing"??? "Practioner sharing"???
Sec 3.2 would also be sec 2... Are the 16 "segments" (??) actually lessons/dates/events? If so, please state. Why this structure?
The link in l. 282 would be VERY interesting but points to a login... I would appreciate at least a few screen shots!
Table 7 could be an appendix.
Is sec 3.3 really about product validity? For this you should have date about the learning outcomes! Why not just "Evaluation results"?
I recommend numbered lists or (even better) a table for the qualitative feedback. How were these items produced?
It is wonderful that the course was widely rated as very good. On the other hand, this suggests that the questionnaire may not have been specific enough. This leads to two conclusions: first, the high approval ratings should not be celebrated repeatedly, and second, there should be a discussion about which questions might have been missing from the questionnaire.
For a discussion (sec 4) I would be interested a differentiated analysis of the qualitative feedback. Please collect lessons and potential for improvement for future course designs -- your paper should inspire other educators, not just celebrate your achievements. Instead, most paragraph reflect on how certain elements of the course design were implemented -- this could be part of a reflective validation and then should be part of sec 3!
Comments on the Quality of English Language
Needs editing.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for Authors- The research question is clearly defined and structured by the study purpose.
- The research process seems to be methodologically sound and the validation process was supported by expert opinions. In addition, a pilot study was conducted with 95 students.
- The research shows that the developed non-formal learning model is feasible and suitable for educational settings. However, it is possible to say that the generalisability of the findings is limited because the application was only studied with specific student groups in Indonesia.
- The study emphasises the potential of the non-formal learning model in developing student creativity and collaboration skills. Further studies on the long-term effects of the model in practice are needed.
- The results of the study emphasise that the model has the potential for an extended application in education. In addition, new technologies and collaboration skills need to be integrated into learning environments. In this context, it is thought that it would be appropriate to test the model on a larger sample and to conduct long-term impact evaluations.
- As a conclusion, it can be said that this study can be considered as a study with a strong theoretical background and methodology that can make significant contributions to the field.
Comments on the Quality of English LanguageThe overall document is well prepared in terms of the use of English for an academic paper and only minor revisions are needed to improve readability and comprehensibility.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsI encouraged the authors to review the Design-Based Research (DBR) methodology. When validating a product/model/experience, describing the steps taken to improve the model is essential. A simple survey is a small part, but what else did you use to improve it?
The problem statement (lines 44-68) is far-fetched. How did you arrive at PLE as the solution? You may state that curriculum alignment to 21st-century standards is needed. What rubric was used to assess the learning materials (e.g., Quality Matters or other instrument)?
RQs are missing. Table 4 is unnecessary and does not inform the reader about the survey items. Use Table 4 to describe how many items were in each section.
Additional concepts regarding informal learning and the role of social media are missing. The introduction and methodology need to clarify key differences between formal and non-formal environments.
Comments on the Quality of English Language
None.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsI find the article significantly improved, thank you.
Minor issues:
l.401: for students... -- I recon that the needs/ appreciation of students do not depend too much on a certain group identity: first, the audience for this course is rather mixed (cf. intro), second the feedback from the students is not at all topic relalted...
l.421 correlation -- alignment?
l.423 what does stakeholder participation mean??
l.424 a little bit vaunting. Why not just "needs assessment in combination with... proves a solid foundation...
l.431 feasibility -- success?
l.433 language/grammar. Some redundancy & lengthiness in these paragraphs. I suggest to implement some substructure, e.g. along the main attributes of the learning models or/and key objectives.
l.550 ?? I suggest to extract some statements from the discussion into the conclusion. See https://www.nature.com/scitable/topicpage/scientific-papers-13815490/
Comments on the Quality of English Language
Still to be improved.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThank you for the detailed responses in the reviewer letter. I appreciate the lengthy revisions made to this manuscript. Since this is a mixed methods study using a survey, the qualitative aspect under section 2.6 needs clarification. What qualitative technique did you use to interpret suggestions and critiques by experts and students in lines 335 - 347?
Details on triangulation methods are missing based on what you derived from the qualitative and quantitative components. How did you triangulate results from the quantitative and qualitative factors (e.g., data, investigator, theory, and/or methodological triangulation methods)?
Figure 4 must provide a good sense of the overall product validation and trial averages. This could be a table or a series of bar charts that explain average scores by expert/student with alignment with the evaluated aspects. Lines 396-399 are simply a regurgitation of the numbers in Figure 4. I encourage authors to re-develop this figure with text interpretations and not merely re-articulate the same thing.
The abstract needs additional work. Add additional justification for the development of PLE. It stops at not keeping pace with technology in lines 14-15. Expand on the problem. Then, justify the method and provide the results with implications.
Author Response
Please see the attachment
Author Response File: Author Response.pdf
