Application of Online Transportation Mode Recognition in Games

Round 1

Reviewer 1 Report

The author had proposed an application of online transportation mode recognition in games using machine learning and persuasive multiplayer game. While the proposed approach had undergone a proper scientific methodology, the authors reported promising results. However, some other issues were found in the manuscript's current content and contribution, which require careful revision by the authors.

The followings are the issues found in the current version of the manuscript:

1. The author did not clarify whether the focus of the study is on persuasive games in a non-game (or serious) context? Or persuasive-based serious games? The former seems to be accurate but was not mentioned clearly in any part of the study.
2. The start of Section 2.1 (1^st paragraph) should be described holistically and systematically, where the presentation of the related works is not merely conveying information but a synthesis of ideas that leads to your work. Also, the associated works on transportation mode recognition were sufficiently covered, albeit the authors had mentioned no motivation or relevance of conducting such recognition.
3. In Section 4, do the components of your proposed architecture loop? For example, do the 1^st component pass data to the 2nd component, and the 2nd component passes them to the third, then returns to the 1^st or 2^nd component, and the process repeats kind of in a loop? Or is it working differently (like ML, first training, then testing)? Further clarification on the matter is not apparent in the study and should be clearly described.
4. The authors can improve figures in the manuscript in terms of quality and clarity. This situation also applies to the Tables. While both provided informative reports of the results, it does not self-contained and highlights each table/figure's main point. Basically, what those figures/tables trying to tell? If it is the same as what the text is describing, it wouldn't be relevant to put them in the first place since it just being redundant—also, the table caption typically at the top of the table, not the bottom. So, kindly check also formatting requirements.
5. Do the part on 'noise reduction by using historical set' your contribution or derived from someone else? If not, the authors should include a proper citation. If so, adequate justification needs to be mentioned.
6. It was not clear what is considered as the enemy in the Evergreen game design. Does it refer to other players only, or includes some NPCs or other form of an enemy? Also, how does the calculation (plus or minus values) of the "Emission" was determined? I may have missed it, but I don't think the author had given such calculation (or formula) since it can be considered the "point" system of the game? Also, how much can such "Emission" also change the background images' sequence (magnitude, values, multipliers, etc.)? Can the player influence it other than the transportation mode used?
7. In addition, the author considers the Evergreen game as a mixture of turn-based and role-playing games. However, what roles does the player have in the game? Is there multiple roles option supported? How such roles influence other game mechanics (i.e., "Emission")? It was not clearly described what the player would be role-playing as in the game. Such a role-playing system could also assist in catering to different players in the game (such as the player with mandatory driving required due to long commute to work, as mentioned in the discussion part).
8. Why do the four classification approaches (random forest, random tree, Bayesian network, and Naïve Bayes) were chosen? In addition, what are the general findings between offline and online transportation mode detections (similarity, differences, lesson learned)? The authors did not adequately discuss the results to determine the predicted values obtained in Table 5—Table 11 relative to the persuasive games and influences the effectiveness of the application in deployment or real-world impacts.
9. How was the game evaluation obtained or conducted? By interviews? Questionnaires? Observations? It was not apparent how the author receives such valuable insights in Section 6.1 from the test players. Also, Section 6.3 seems like "half-cooked" or "half-baked" afterthoughts, which should address more important questions, such as the limitation of the applications? What could be an appropriate measure of persuasion based on your own experience of developing your application? Any suggestions or insights accompanied by your results and findings? Also, where is "Section A"? How does it "appear" to promote awareness of green transportation while this study makes players aware of carbon emission from the frequent use of transportation?
10. Finally, the answers to research questions in the Conclusion don't make sense and should be rephrased or rethought. For example, how can iterative play become the impactful aspect of a persuasive game? Such aspects were rarely analyzed as the main component of your experiments to be viable, let alone impactful in the persuasive game. Instead, the impact of real-life transportation mode might be closer to the answer. Also, how does one answer a question on the battery life, but the experiment does not support such elements in the study, instead just presenting the ML approach's performance? Your investigation should include the observation of device battery life in the experiment to answer such a question.

Other minor issues include the unclosed braces, summary of sections of the study in roman numerals. At the same time, the format in numbers (check formatting requirement), some Tables were not correctly numbered (2^nd Table 1 should be Table 4). Extensive English editing is required (especially in Section 2 and Section 3). Nevertheless, the effort put into the presentation of the work is commendable and deserves to be published.

Comments for author File: Comments.pdf

Author Response

Comments from Reviewer 1 

Thanks to the reviewer for the elaborate and rigorous review. 

The author had proposed an application of online transportation mode recognition in games using machine learning and persuasive multiplayer game. While the proposed approach had undergone a proper scientific methodology, the authors reported promising results. However, some other issues were found in the manuscript's current content and contribution, which require careful revision by the authors. 

The followings are the issues found in the current version of the manuscript: 

1. The author did not clarify whether the focus of the study is on persuasive games in a non-game (or serious) context? Or persuasive-based serious games? The former seems to be accurate but was not mentioned clearly in any part of the study. 

Page 4, para2, line 6, ...Evergreen is a game with persuasive elements, have removed the term serious and replace them with persuasive. 

1. The start of Section 2.1 (1st paragraph) should be described holistically and systematically, where the presentation of the related works is not merely conveying information but a synthesis of ideas that leads to your work. Also, the associated works on transportation mode recognition were sufficiently covered, albeit the authors had mentioned no motivation or relevance of conducting such recognition. 

See page 3, para 1: However, the UbiGreen mobile application uses Global System for Mobile Communication System (GSM) and Global Positioning System (GPS) information for semi-automatic sensing of transit activity [18][50]. However, our proposed Evergreen Persuasive Game mobile application uses the mobile inbuilt accelerometer and gyroscope related data for automatic transit activity sensing. 

Section 2.1: Khaled et al. [16] discuss several challenges relating to effective impacts of persuasive games: managing player attention, balancing game contents with reality, and issues concerning identity and target audiences. 

Section 2.2. Strengthen the rationale for transportation mode recognition: 

In recent years, transportation mode recognition has been used for: identification of peoples’ physical activities [51]; user dynamic control of their optimal route [52]; and support intelligent transportation systems [53]. However, limited research has been conducted on transportation mode recognition to promote sustainability awareness.  

1. In Section 4, do the components of your proposed architecture loop? For example, do the 1st component pass data to the 2nd component, and the 2nd component passes them to the third, then returns to the 1st or 2nd component, and the process repeats kind of in a loop? Or is it working differently (like ML, first training, then testing)? Further clarification on the matter is not apparent in the study and should be clearly described. 

No. Data was gathered before-hand in order to train the Transport Recognition Service. It used machine learning (more specifically Random Forest which is mentioned later on). While iterating the classifier more data was gathered, but in effect the game had a “statically” trained machine learning classifier it used to deduce mode of transportation. I think the Figure 1 description covers this, and we also mention what data was gathered to trained the classifier in Table 1. 

 Section 4, See Subsection II 

Model Building and Deployment  

Historical sensor data have been gathered and trained using machine learning classifiers (i.e. Random Forest, Random Tree, Bayesian Network, and Naïve Bayes) for the Transport Recognition Service module (in Figure 1). Thus, 4 classification models have been built and deployed to detect transportation mode based on new accelerometer and gyroscope data input. During deployment, ...... 

1. The authors can improve figures in the manuscript in terms of quality and clarity. This situation also applies to the Tables. While both provided informative reports of the results, it does not self-contained and highlights each table/figure's main point. Basically, what those figures/tables trying to tell? If it is the same as what the text is describing, it wouldn't be relevant to put them in the first place since it just being redundant—also, the table caption typically at the top of the table, not the bottom. So, kindly check also formatting requirements. 

Have enhanced quality of Figures 11 and 12 

The quality of the figures will be evident when zoomed in. 

Final formatting will be done by MDPI team. 

1. Do the part on 'noise reduction by using historical set' your contribution or derived from someone else? If not, the authors should include a proper citation. If so, adequate justification needs to be mentioned. 

Embedded citations 

According to Gupta and Gupta (2019), noisy data could significantly impact on prediction accuracy. In our research, we adapt the use of time-based historical set [31] [32] as noise filter in classifier predictions. 

1. It was not clear what is considered as the enemy in the Evergreen game design. Does it refer to other players only, or includes some NPCs or other form of an enemy? Also, how does the calculation (plus or minus values) of the "Emission" was determined? I may have missed it, but I don't think the author had given such calculation (or formula) since it can be considered the "point" system of the game? Also, how much can such "Emission" also change the background images' sequence (magnitude, values, multipliers, etc.)? Can the player influence it other than the transportation mode used? 

It is clear if you read the citation 46 of the paper prototype – you fight monsters spawned from the polluted world (NPCs). You could also interact with other players though, which is mentioned. However, this is a game with impact of actions on the environment. 

I think the exact details of which transport gave how much emissions and how much emissions was required for certain events (like changing background, more difficult monsters spawning and attacking) rather outside the scope of the paper, but again reading/referring to the paper prototype here would give a good hint of what the mobile game was doing. 

1. In addition, the author considers the Evergreen game as a mixture of turn-based and role-playing games. However, what roles does the player have in the game? Is there multiple roles option supported? How such roles influence other game mechanics (i.e., "Emission")? It was not clearly described what the player would be role-playing as in the game. Such a role-playing system could also assist in catering to different players in the game (such as the player with mandatory driving required due to long commute to work, as mentioned in the discussion part). 

This is mentioned in the Game Design Details. A role-playing game doesn’t necessarily always require that you “choose” a starting role either, but you can design your own “role” yourself based on what actions you take. If the reviewer read just a bit more about role-playing games they would know about this. It is common knowledge for gamers.  

Perhaps a reference to Wikipedia would do good here for the non-gaming readers: https://en.wikipedia.org/wiki/Role-playing_game 

This is definitely the case, as some actions would get different bonuses and penalties based on what transport was chosen for that in-game “day”/turn. But catering to more players has more to do with what role you define yourself as based on your actions than the bonuses based on transport. Again referring to the Wikipedia article. 

1. Why do the four classification approaches (random forest, random tree, Bayesian network, and Naïve Bayes) were chosen? In addition, what are the general findings between offline and online transportation mode detections (similarity, differences, lesson learned)? The authors did not adequately discuss the results to determine the predicted values obtained in Table 5—Table 11 relative to the persuasive games and influences the effectiveness of the application in deployment or real-world impacts. 

See  Subsection 5.2 

To reiterate, four common classification approaches are employed for this research. They are: Random Forest, Random Tree, Bayesian Network, and Naïve Bayes. Criteria for selection are: versatility (Random Forest [58]); easy and fast processing (Random Tree [57], and Naïve Bayes [56]); extensive use (Bayesian Network) [55]. 

Tables 3-10 – have added additional interpretation of results. 

1. How was the game evaluation obtained or conducted? By interviews? Questionnaires? Observations? It was not apparent how the author receives such valuable insights in Section 6.1 from the test players. Also, Section 6.3 seems like "half-cooked" or "half-baked" afterthoughts, which should address more important questions, such as the limitation of the applications? What could be an appropriate measure of persuasion based on your own experience of developing your application? Any suggestions or insights accompanied by your results and findings? Also, where is "Section A"? How does it "appear" to promote awareness of green transportation while this study makes players aware of carbon emission from the frequent use of transportation? 

See subsection 5.1 - on questionnaires 

See subsection 6.1 - interview after playing the game 

Typically carbon emissions are closely linked to greenness of the transportation mode. 

1. Finally, the answers to research questions in the Conclusion don't make sense and should be rephrased or rethought. For example, how can iterative play become the impactful aspect of a persuasive game? Such aspects were rarely analyzed as the main component of your experiments to be viable, let alone impactful in the persuasive game. Instead, the impact of real-life transportation mode might be closer to the answer. Also, how does one answer a question on the battery life, but the experiment does not support such elements in the study, instead just presenting the ML approach's performance? Your investigation should include the observation of device battery life in the experiment to answer such a question. 

The Conclusion and Future Work section has critiqued the research processes and how they could be appropriately addressed. The main focus is on the transportation mode detection. In order to conduct an investigation on  behavioural change, a longitudinal study with a bigger sample size will be necessary. 

Other minor issues include the unclosed braces, summary of sections of the study in roman numerals. At the same time, the format in numbers (check formatting requirement), some Tables were not correctly numbered (2nd Table 1 should be Table 4). Extensive English editing is required (especially in Section 2 and Section 3). Nevertheless, the effort put into the presentation of the work is commendable and deserves to be published. 

Have been proof read. 
 

Reviewer 2 Report

Though I am not an expert of the design of persuasive games or the methodology of machine learning, I think this article is sufficiently written and I could understand the significance of this study.

Author Response

Comments from Reviewer 2 

Though I am not an expert of the design of persuasive games or the methodology of machine learning, I think this article is sufficiently written and I could understand the significance of this study. 

Thanks to the reviewer for the comment. 

Reviewer 3 Report

This research is interesting and the introduction does relatively well the job to illustrate the basic literature. Sometimes when dealing with some specific class topics (e.g serious game, gasification, etc.) a footnote explaining its basic meaning plus a fundamental citation on the topic could be of help for the reader. 

In the presentation of the methodology a sample of volunteers is mentioned. We need to know how the sample was created and how volunteers were recruited. 

How to circumvent the problem of privacy for game participants should perhaps be mentioned, since all people movements are detected by the App.

The conclusions could report about future research development along the same lines.

Author Response

Comments from Reviewer 3 

Thanks to the reviewer for the comment. 

This research is interesting and the introduction does relatively well the job to illustrate the basic literature. Sometimes when dealing with some specific class topics (e.g serious game, gasification, etc.) a footnote explaining its basic meaning plus a fundamental citation on the topic could be of help for the reader.  

In the presentation of the methodology a sample of volunteers is mentioned. We need to know how the sample was created and how volunteers were recruited.  

Have added in the results section since the methodology section focuses on the app.  

How to circumvent the problem of privacy for game participants should perhaps be mentioned, since all people movements are detected by the App. 

The conclusions could report about future research development along the same lines. 

Have addressed this in future work.