An Adaptive UI Based on User-Satisfaction Prediction in Mixed Reality
Round 1
Reviewer 1 Report
Manuscript ID: applsci-1694622
An adaptive UI based on user satisfaction prediction in mixed reality Special Issue: Virtual and Augmented Reality Systems
The authors present a study investigating a method based on interaction satisfaction prediction to provide adaptive UI in MR. The proposed method predicts interaction satisfaction based on interaction information (gaze, hand, head, object) and provides adaptive UI based on predicted interaction satisfaction. The method has been evaluated in a user study.
The authors could consider the following suggestions for improvement.
- More information should be provided about the study participants (age, gender, previous MR experience, handedness).
- The number of participants is quite low (7) and any statistical conclusions would have limits. The authors should briefly elaborate this issue under Study Limitations (sub)chapter.
- In Fig 10, user satisfaction is provided based on 6 defined position types. Front and right orientations are preferred. This could also be due to the participants’ right handedness factor. The authors should briefly elaborate on this issue.
- Some minor grammatical and misspell errors, e.g. line 231, “which had been extended om support vector machine (SVM) to solve multivariate regression problems,”
Author Response
Responses to Reviewer 1:
First of all, the authors would like to express sincere gratitude for reviewer’s invaluable time and thoughtful comments.
In order to save the reviewer's invaluable time, and expedite the processing of the revised manuscript, the authors tried to be as specific as possible in our response to the reviewer.
Overall, the authors agreed with the reviewer's opinion and comment. For better quality of the paper, the authors had read the manuscript carefully, and some sentences, figures were added and modified according to the reviewer's comment.
Points 1.
More information should be provided about the study participants (age, gender, previous MR experience, handedness).
(Answers)
Thanks for the invaluable comment. According to reviewer’s comment, we added the parts that explain the subject in chapter 4.1.2(Page 6 line 192-193), chapter 4.2(Page 7 line 216-219), and chapter 4.3(Page 12 line 344-347).
Please refer to the blue texts: Page 6 line 192-193, Page 7 line 216-219, Page 12 line 344-347
Points 2.
The number of participants is quite low (7) and any statistical conclusions would have limits. The authors should briefly elaborate this issue under Study Limitations (sub)chapter.
(Answers)
Thanks for the considerable comment. According to reviewer’s comment, we added the parts that explain the limitations of this study. In that part, we explain the limitation of the number of subjects (Page 13 line 393-395).
Please refer to the blue texts: Page 13 line 393-395
Points 3.
In Fig 10, user satisfaction is provided based on 6 defined position types. Front and right orientations are preferred. This could also be due to the participants’ right handedness factor. The authors should briefly elaborate on this issue.
(Answers)
Thanks for this invaluable comment. As the reviewer mentioned, we agreed that the explanation was not enough. According to reviewer’s suggestion, we added the part that explains that all subjects were right-handed(Page 6 line 192-193), and also added the part that explains the preference for the right side should be considered to change when the target user is left-handed (Page 11 line 321-326).
Please refer to the blue texts: Page 6 line 192-193, Page 11 line 321-326
Points 4.
Some minor grammatical and misspell errors, e.g. line 231, “which had been extended om support vector machine (SVM) to solve multivariate regression problems,”.
(Answers)
Thanks for the considerable comment. According to reviewer’s comment, we modified the typo error in our paper.
Please refer to the blue texts: Page 7 line 244
Author Response File: 
Author Response.docx
Reviewer 2 Report
This paper describes a method to adapt MR interfaces based on users’ satisfaction. The prediction model considers multiple features, including gaze, hand, head, and corrects the position of user interface elements depending on a predicted users’ satisfaction score. This approach is novel and interesting. Additionally, the paper well documents the design process of the model, as well as presents a preliminary usability evaluation, based on a A|B study.
My suggestion for improving this draft mostly focus on strengthening the contribution.
First, the Introduction should be edited to better clarify the problem space and the contribution of this work. For example, it would be helpful to add a figure in the introduction with one or two examples of user interfaces in MR environments. This would clarify the scenario/problem space that this work addresses. Additionally, the rationale for this work provided in lines 48 to 55 is weak: it makes it sounds that the only reason for this work is to measure users’ satisfaction in a more accurate manner. I would encourage the authors to highlight what are the additional benefits of the adaptive interface that they envision. Finally, there are limited references in the Introduction. For example, the claim “Meanwhile, it is also possible to analyze user satisfaction in real time by attaching a sensor such as EEG to the MR application user” should be supported by a citation.
Second, I would suggest adding a Discussion section, to discuss possible application scenarios, ideas on how to generalize the model beyond the specific object in figure 6, and highlight design recommendations for researchers and practitioners. In the discussion, I would invite the authors to elaborate on how practical some elements of this system could be (e.g., the glove with marker), depending on the application scenarios.
Overall, the paper may also benefit from some editing, as there are some cryptic sentences (e.g., abstract, line 12: “User interface (UI) is one of the representative factors that affect interaction satisfaction in MR”).
Author Response
Responses to Reviewer 2:
Points 1.
Extensive editing of English language and style required.
(Answers)
First of all, the authors would like to express sincere gratitude for reviewer’s invaluable time and thoughtful comments. In order to save the reviewer's invaluable time, and expedite the processing of the revised manuscript, the authors tried to be as specific as possible in our response to the reviewer. Overall, the authors agreed with the reviewer's opinion and comment. For better quality of the paper, the authors had read the manuscript carefully, and some sentences and figures were added and modified according to the reviewer's comment.
Points 2.
First, the Introduction should be edited to better clarify the problem space and the contribution of this work. For example, it would be helpful to add a figure in the introduction with one or two examples of user interfaces in MR environments. This would clarify the scenario/problem space that this work addresses. Additionally, the rationale for this work provided in lines 48 to 55 is weak: it makes it sounds that the only reason for this work is to measure users’ satisfaction in a more accurate manner. I would encourage the authors to highlight what are the additional benefits of the adaptive interface that they envision. Finally, there are limited references in the Introduction. For example, the claim “Meanwhile, it is also possible to analyze user satisfaction in real time by attaching a sensor such as EEG to the MR application user” should be supported by a citation.
(Answers)
Thanks for the considerable comment. According to the reviewer’s comment, we added Figure 1, which expresses various types of user interfaces in mixed reality environments (Page 1 line 42-43. Figure 1) to provide examples of user interfaces. Also, we modified some parts in the introduction to improve our rationale (Page 2 line 51-54, and line 59-64). Finally, to make the explanations clear, we added references to several sentences in the introduction (Page 2 line 55, and line 58).
Please refer to the blue texts: Page 1 line 42-43. Figure 1, Page 2 line 51-54, 55, 58 and 59-64,
Points 3.
Second, I would suggest adding a Discussion section, to discuss possible application scenarios, ideas on how to generalize the model beyond the specific object in figure 6, and highlight design recommendations for researchers and practitioners. In the discussion, I would invite the authors to elaborate on how practical some elements of this system could be (e.g., the glove with marker), depending on the application scenarios.
(Answers)
Thanks for the invaluable comment. As the reviewer mentioned, we agreed that the application scenario was not enough. We added the part that explains possible application scenarios (Page 13 line 387-391).
Please refer to the blue texts: Page 13 line 387-391
Points 4.
Overall, the paper may also benefit from some editing, as there are some cryptic sentences (e.g., abstract, line 12: “User interface (UI) is one of the representative factors that affect interaction satisfaction in MR”).
(Answers)
Thanks for the considerable comment. We wrote line 12 in the abstract to express line 46 on Page 2 ("Since UI is a channel that connects users with MR, it can affect user satisfaction [7, 8]."). However, most abstracts did not include references, so we excluded the reference in line 12. As the reviewer mentioned above, there were several parts required to explain in more detail. So, we modified and added several parts, not only in chapter 1 (Page 1 line 42-43, Page 2 line 51-54, line 55, line 58, and line 59-64) but also in chapter 4 (Page 6 line 192-193, Page 7 line 216-219, Page 12 line 344-347).
Please refer to the blue texts: Page 1 line 42-43, Page 2 line 51-54, line 55, line 58, and line 59-64, Page 6 line 192-193, Page 7 line 216-219, Page 12 line 344-347
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
The authors have somewhat addressed the issues raised. The paper is interesting and important for the field. However, some issues should still be clarified.
There are two studies mentioned in the paper (1) Experiment for implementation of proposed method and (2) Experiment for evaluation of proposed method. In the first experiment 26 participants were involved and in the second 7 participants.
It is not clear, which and how the participants were selected for the second experiment. Were they also all right handed?
The authors should clarifly this. How were both experiments balanced (the order of selection and moving tasks)?
Author Response
Responses to Reviewer 1:
First of all, the authors would like to express sincere gratitude for reviewer’s invaluable time and thoughtful comments.
In order to save the reviewer's invaluable time, and expedite the processing of the revised manuscript, the authors tried to be as specific as possible in our response to the reviewer.
Overall, the authors agreed with the reviewer's opinion and comment. For better quality of the paper, the authors had read the manuscript carefully, and some sentences, figures were added and modified according to the reviewer's comment.
Points 1.
It is not clear, which and how the participants were selected for the second experiment. Were they also all right handed?
(Answers)
Thanks for the invaluable comment. We explained the experimental environment and methodology, which are commonly used in our two experiments in chapter 4.1 (Page 5 line 161-162). And, according to reviewer’s comment, we added the parts that explain that subjects in our experiments were right-handed in chapter 4.1.2 (Page 6 line 192-193). In addition, we modified a sentence to clarify that the subjects of chapter 4.3 were also right-handed people (Page 11 line 324-325). Finally, we explained the information (gender and previous MR experience) about the subjects of the second experiment (Page 12 line 344-348).
Please refer the blue texts: Page 5 line 161-162, Page 6 line 192-193, Page 11 line 324-325, Page 12 line 344-348
Points 2.
The authors should clarifly this. How were both experiments balanced (the order of selection and moving tasks)?
(Answers)
Thanks for the considerable comment. According to reviewer’s comment, we added the parts that explain the order of the selecting tasks and the moving tasks (Page 6 line 205-207).
Please refer the blue texts: Page 6 line 205-207
