An Analysis of Game Design Elements Used in Digital Game-Based Language Learning

Round 1

Reviewer 1 Report

I am not sure why this paper has been submitted to the "Sustainability” journal – it doesn’t belong here. Social Sciences would be a better fit. Having said this, I will review the paper itself anyway, assuming it will be published in the appropriate MDPI journal.

Within the context of education, and language learning specifically, this is an interesting and relevant review of studies related to digital game-based language learning, and the game design elements used in such games. Some clarification and further detail will help to enrich this study. I will begin with my main concern, and then I will share a few smaller questions and suggestions.  It is due to my main concern, explained below, that the review question: "Is the content succinctly described and contextualized with respect to previous and present theoretical background and empirical research (if applicable) on the topic? " is rated as "must be improved"

Main concern: Please provide more detail regarding Marczewski’s gamification mechanics and elements.  It is claimed on page 6 that “Marczewski’s approach to classifying elements and mechanics is well established….” yet no sources are cited to back this claim, and no explanation justifying this claim is provided.  The reader has to be convinced that this is a well-established approach, and requires a more detailed description of the approach.  Describe the framework and provide a full list of the mechanics and elements.   

In relation to Marczewski’s framework it is further stated that “these game design elements are reliable and beneficial in educational settings.” How do we know this? This is a sweeping claim that requires justification.

The further claim is made that “Marczewski’s framework enables a more meticulous analysis than other categorizing methods used in previous reviews.” Although the authors provide a sense that other frameworks such as ARCS are less nuanced, it is not clear why Marczewski’s framework enables a more meticulous analysis. Is it simply because there are more components?  Elaborate further on reasons why such a framework enables a more meticulous analysis?  

Prior to the discussion of Marczewski’s work reference to the MDA framework is mentioned, though it is not clear why. It is stated that “these three terms will be grouped into game elements or game features”, yet they are not mentioned again.  Remove these details, or clarify and elaborate their purpose.

Further questions and details required:

Pg. 2, line 47:   "gamification, or the use of game design elements to encourage engagement, rose in popularity around 2011 when it entered Gartner’s Hype Cycle" - what is Gartner's Hype Cycle? please explain

Pg. 2, line 50:  it is a simplification to say that serious games are “education focused" and "often simulations" – there are many types of serious games and they are typically differentiated from “educational games” – serious games broadly, are games made for purposes beyond entertainment (this is stated in the Susi paper) – it does get complicated because educational games  obviously have a purpose beyond entertainment, but "serious games: is the broad category that covers games for training; health; social impact/social change ....The purpose of this paper is not to get into the weeds on this issue, however these are slight misstatements.

Pg 3,  line 81: "RQ2: What are the differences in game elements between bespoke games, off-the-shelf games, games for entertainment, and entertainment?" I believe this should read Games for education.

Pg 3, line 98:  "For example, a digital game-based approach would use a game such as Monkey Island or Deponia to promote contextual learning, whereas gamification would introduce points and badges to a traditional classroom to create a game-like environment" Briefly describe contextual learning drawing from either game example

Pg 3, line 106: what do you mean by full-fledged games in contrast to serious games? Please clarify “serious or full-fledged games”

Pg 3, line 110: serious games don’t necessarily require previous knowledge

Pg 3, line 111: "The main differences are in the execution of content. Where serious games usually require previous knowledge, content can be executed in a vaguer and more contextually challenging way to simulate real life." What does this mean? Clarify

Pg 3, line 116: “Using games for learning has been linked to creating a state of immersion, also known as flow and encouraging engagement because of their characteristics [16].”  this is a vague statement – doesn’t contribute any vital information – suggest removing

Pg. 7,  line 242: “The categories other categories were communication, ….”

7 line, 254 – 256: Why is it surprising that identifying design features and the benefit of using specific elements were primarily found in bespoke game papers?  Expand briefly on the examples you provide by Ongoro and Yang – are you saying that the bespoke papers were more useful because they went into detail about the design features then? Clarify

Pg. 10, table 3: Are you able to provide a column with the dependent variables?  What outcomes were being measured in the studies?

Final suggestion: can you create an appendix with the list of games that are referred to in all of the papers? This would be a useful resource, and it would just be helpful to see them concretely, even though, as mentioned, many of the games, particularly of course the bespoke games are not accessible.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper discusses the interesting topic of DGBLL and provides a critical review of existing studies published in English from 2010 to 2021 as retrieved from the EBSCO Discovery Service.  The structure of the paper is clear and there is adequate literature and related work documented. Below I have some minor comments for specific lines in the paper:

• line 242: "categories" should be deleted
• line 337 you refer to an 80.53% percentage. I assumed you calculated 90/113 but this is not 80.53%. How did you come to this percentage?
• line 315: word "contained" should be deleted
• line 314-5: you say that "COTS educational games used fewer game elements on average than the bespoke games. This is not evident from the respective boxplots. 
• The phrasing of RQ3 is not clear (in the second part)

Another thing is that section 4.4 does not correspond to an RQ. Also, I would like to see some rationale on why you have selected these specific research questions to address in your work. The discussion could be strengthened with your opinion on how your findings are justified (you have done this in some cases but not for all findings). 

You focus your analysis on COTS versus bespoke games but you also distinguish between educational and entertaining games. I would like a more clear description of these 4 types and their overlappings if any.

It would be helpful for the reader to have a definition of elements like the virtual economy, exploration, consequences, signposting. Design elements are the main focus of your study.  

Your limitations are true I agree with all of them. To my opinion, the main limitation is that you searched with very few keywords if you included some of the alternative words you mention you would certainly have identified many more publications to include in your study. Still, 113 papers is a good number.  

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Arrange the abstract with more direct ideas to the focus of the research.
The scientific literature review (SLR), methodology applied in this research, presents selection criteria and analysis criteria.
The research presents significant findings for the thematic oriented to digital game-based language learning, specifically demonstrating the use of game elements in its development.
It is recommended to include the differentiation between game-based learning and gamification.
Table 2 is very important and its publication can be extrapolated to other researches.
It is noted that English is not the native language of the researchers, it is recommended to reread the article to denote some misprints.

Translated with www.DeepL.com/Translator (free version)

Author Response

Please see the attachment.

Author Response File: Author Response.pdf