Investigating the User Interface Design Frameworks of Current Mobile Learning Applications: A Systematic Review

Round 1

Reviewer 1 Report

The author(s) examine the user interface design guidelines/frameworks currently in use for mobile learning applications and discuss the respective criteria applied when using each of these guidelines/frameworks, with the aim to develop an understanding of the criteria that need to be considered when designing user interfaces for mobile learning applications. This study also investigates how these parameters or criteria affect the learner's cognitive load. The title however is not reflective of the main focus of the paper which is the former, that is examine the user interface design guidelines/frameworks currently in use for mobile learning applications and discuss the respective criteria. This needs to be re-considered. Furthermore, although there is emphasis on cognitive load, the connection between this construct and learning is only briefly discussed.

Although there is reference to how Covid-19 pandemic has shifted learning to online learning, and this is even stated in the abstract, there is not further analysis on this other than the fact that the studies were chosen between 2020-22.  

Overall, interesting results are extracted from this systematic review. Some effort is needed to improve clarity and syntax. Specific cases are extracted and provided below to ease the editing process. These suggested edits are separated per section as per the article's structure.

Abstract

The abstract portrays the main objective of the paper. In the text a side objective/aim is mentioned “….aimed at finding the possible research gaps and areas for future opportunities…..” (line 127). This could also be emphasised in the abstract/introduction.

The first 2 sentences talking about Covid-19 do not add to the discussion as the effects of mobile learning applications’ UID on cognitive load is a pervasive topic. The authors may consider moving these to the Introduction rather than the abstract. Alternatively, the reference to Covid-19 can be contextualised by mentioning that the study is based on articles selected based on the publication years after the COVID-19 outbreak i.e. between 2020-22, although since there is no aim to compare the criteria pre-, during, and post-pandemic this needs to be carefully considered.

Also, it would be useful and informative alike, to mention in the Abstract that the PRISMA approach is used. Such a reference would help in the current study being selected by other scholars when using filtering and searching criteria.

1. Introduction

While some of the references providing definitions of mobile learning or m-learning are provided, some newer ones could also be used and synthesised to show the currency of the definitions.

Consider rephrasing/check syntax or grammar/improve clarity:

-        Line 40-43: The paragraph starting with “After the pandemic…” somehow breaks the flow of the previous paragraph discussing the positive aspects of mobile learning while the following paragraph continues talking about the attributes of mobile learning in terms of learning participation and achievement. The line or argument can be improved to separate the positive aspects from the barriers.

-        Line 83-34: Please consider rephrasing or adding commas for clarity: “However, there are also some researchers who have found that when using mobile learning applications, the learners' cognitive load which relates to problematic UID increases”

-        Lines 85-86: Needs clarity (there is no prior reference to the factors): “Some of these factors…”

-        Lines 95-96: “Insufficient information on the criteria could cause [to] the developed…”

-        Line 100: “that need[S] to be considered…”

-        Lines 103-106: In the last paragraph of section 1 which describes how the remainder of the paper is structured, please consider tenses as simple-present and past tenses are used interchangeably.

2. Methods

PRISMA approach is used. A reference could be added to the version of the PRISMA used (e.g., 2020)

The Research Questions are focused and appropriate.

The search strategies/search string could also consider “heuristics” as this is a popular and relevant term in UID.

QAC Questions are appropriate. However, please check consistency (1^st question missing question mark, check tense to ensure consistency: “Does the article clearly mention[ed]…”

Consider rephrasing/check syntax or grammar/improve clarity:

-        Line 109: singular or plural? “…approaches…”

-        Line 117: The articles are again filtered based on its [their] title, keyword[s], …

-        Lines 118-122: Check consistency of tenses

-        Line 125 “guideline/framework that are currently”: check singular/plural

-        Lines 148-149: “. The same screening process were [was] implemented during the snowballing procedure.”

-        Lines 181-182: Needs clarity: “….with the relevant discussion based on subsection”

3. Results

Overall, the synthesis of the findings and discussion of emerging themes is appropriate and the discussion of the results reveals interesting findings. However, this statement is unclear: “The pedagogical component would not be further examined because the focus of this study is on the UID of mobile learning applications.” The focus of this present article makes the pedagogical component an important component to be further considered so it is unclear what this statements aims to suggest.

In Table 4, the middle column could be rephrased to Number of article[S] or simply denoted as #Articles to clearly indicate what the information in this column denotes (although easily inferred).

In Table 6, it is not clear what number four (4) refers to (in the last column): “No usability evaluation implemented (4)”

In Table 8 fix the typo (remove extra ‘s’ in the end): “circumstances when the learner is unsure where to go.s”

Consider rephrasing/check syntax or grammar/improve clarity:

-        Line 207-8: verb: “…They are based on the cognitive load of learners and emphasis[E]”

-        Line 212: “Nielsen's heuristic[S]…”

-        Line 325: Word ‘The’ is not needed

In the main discussion (lines 290-309) in-text citations are missing and should be provided along with the discussion.

5. Conclusions

The proposal to conduct a ‘detailed study’ is quite vague. This should be further substantiated or elaborated. What type of study? What should the focus of the study be? What research methods should be employed?

General points:

-        All Figures (1,2,3) appear to be in low resolution/poor quality, although this may be due to the preview format.

-        The title of Table A 1 needs rephrasing (or use singular: Criterion)

As stated above, the study reveals interesting insights and organises the literature around the key research questions in a meaningful way. Some refinements are needed to ensure the title and abstract portray the main objective which is to examine the user interface design guidelines/frameworks currently in use for mobile learning applications and discuss the respective criteria. Addressing the suggestions outlined and improving consistency can make this study a valuable contribution.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Online learning has shown significant growth over the last decade, as well as mobile learning, which provides a lot of opportunities for an efficient learning process but also has some limitations that should be taken into account when implementing tutoring systems for mobile devices. Cognitive load theory developed in the late 1980s provides the approaches and recommendations that can be applied to and implemented in e-learning and mobile learning systems. 

This systematic review is aimed at to list out the user interface design standards and criterias that are used in developing a mobile learning applications, and to investigate how these parameters affect the learner's cognitive load, to provide the outlines for designers, developers, tutors who are interested in creating an effective mobile learning systems.

The paper follows the PRISMA approaches for the reviewing and covers 586 publications on Phase 2 filtered to 20 articles on the Phase 4 that covers different frameworks, guidelines and case studies. The articles reviewed were published in the last 3 years, when the COVID-19 impacted the e-learning systems development. The objective of the paper is to identify the User Interface Design guideline that are currently being used for designing mobile learning interfaces and to find the possible research gaps for future opportunities in the development of UID guideline for mobile learning. Four research questions and three Quality Assesssment Checklist Questions were formulated. The effect of UID on cognitive load was analyzed as well for every approach. The results (the criteria of the UID guideline/framework for mobile learning applications, as well as the factors that affect the mobile learning application and the learner's cognitive load) were represetned as the Venn diagram as well. The categorized criteria is provided in detailed 6-pages table. So, the results are very important and usefull for the mobile learning systems engineering and implementation.

The only significant issue I can see is that Figures 1, 2, 3 have almost unreadable text, I recommend using the image in vector format instead of bitmap or using a higher resolution.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Under the methods section Figure 1 content is illegible. from phase 3 to phase 4 only 20 papers were included out of 258. The quality assessment checklist that eliminated 238 papers should be explicitly described. It says "Research contribution"  in the research question 4 but the presented answer only shows the research design not contribution. It is not clear why usability evaluation in Table 6 is presented and it's relation to research contribution part. Discussion part should be improved in terms of findings in relation to similar studies.

Author Response

Please see the attachment.

Author Response File: Author Response.docx