Review Reports

Round 1

Reviewer 1 Report

The proposed paper novelty is reflected by the authors extended research based both on interviews with game developers as well as the extended use and analyze of the case study games aimed to examines complex cognitive skills within game-based learning games and applications.

The significance of the proposed paper is represented by the findings and recommendation based on the thematic analysis of the case studies in complex game-based learning applications

The proposed paper follows a standard structure and is well documented with a wide variety of references that include game-based learning, gamification, game design, psychology, and human-computer interactions. The reference list includes the case study games (Good sudoku and Zachtronics) and applications (Duolingo) as well as other links regarding game development.

The introductions provide a well-documented summary of game-based learning applications. The main objective of the paper is to identify what makes a specific game to be educationally successful. The following section presents the materials and methods used by the authors, in form of semi-structured interviews that were analyzed using reflexive thematic analysis. A close reading technique was also conducted for the Duolingo application to better analyze the overall player experience.

The results section integrates 12 pages and 7 subsections that present various gamification systems, subsystems, and mechanics identified within the specific case studies. The results present original studies based on interviews and experiments. The current form of the results section is difficult to read and analyzed, mostly due to its length (relative to the total length of the paper). I consider that this section with multiple individual subsections is difficult to read and analyze, since the case studies games and language-learning application are so different. 

I don’t understand why the authors decided to list Duolingo (language-learning application) in Table 1 alongside general aspects regarding game-based complex learning. The findings and recommendations should not be linked to specific applications such as Duolingo, they should be linked with concepts identified by the authors within the results section.

The proposed article has elements related to a research paper in form of interviews and experiments designed by the authors to collect and analyze various aspects related to game-based complex learning, but at the same time the paper has the overall structure of a review article based on various aspects identified by the authors within the specific case study games.

The proposed work has a high interest for readers and scientists that are developing various game-based complex learning applications. The summary of findings and recommendations defined by the authors based on their thematic analysis are significant. The other aspects presented within the results and discussion sections represent a good overall view on various aspects that define successful commercial game-based complex learning applications.

The authors have managed to identify general findings and recommendation based on their extended research that made use of interviews with game designer as well as extended use and close analyze of a popular language learning game-based application.

Author Response

Thank you for your review. To your point regarding Table 1, we have renamed the section "Duolingo" to "Lessons from Duolingo" to make clear that those recommendations are not specific to the application, but rather takeaways from the close reading. Because our approach to analyzing Duolingo was a close reading and not a thematic analysis, there are no themes to categorize these takeaways into.

Unfortunately, without more specific suggestions on how to reorganize the results, we are unable to make revisions based on the comment that the section is too long. We have done our best to make it as concise as possible already, cutting out many minor details from our report for succinctness. If you have further suggestions for how to reorganize the material to make it easier to present, we would be happy to make revisions based on specific suggestions.

Reviewer 2 Report

This paper is a long and fruitful investigation of some educational games which contribute to GBL (Game-Based Learning), based on the interviews to the game designers Zach Barth, Zach Gage and Jack Schlesinger. The designers' answers cover a lot of contemporary issues in the field of games and education and a lot of hints and tips are provided about scores, game designs and formal and informal education.

The English language is excellent: I really do not have any concerns about that, moreover I congratulate the Authors for their remarkable skills in the exposition. The only 2 typos I found along the manuscript is 'Schlesigner' instead of 'Schlesinger' in line 480 and 'call' instead of 'calls' in line 582. The Authors should just amend such imprecisions.

Globally, I appreciated the paper and recommend to consider it for publication on MTI.

Author Response

Thank you for the review. I have corrected the typo on Schlesinger; however, I was unable to find the typo referenced on line 582.