Brass Haptics: Comparing Virtual and Physical Trumpets in Extended Realities
Round 1
Reviewer 1 Report
Review of "Brass Haptics: Comparing Virtual and Physical Trumpets in Extended Realities."
This article outlines a systematic approach to creating virtual and physical trumpets by applying extended reality (XR) technology. Overall, the manuscript presents a good introduction and background (sections 1 & 2) to the subject. A clear hypothesis or research question would help situate this work in an "Arts" context, specifically concerned with haptics. Moreover, the lack of empirical evidence to support the central thesis contained in section 3 (Materials and Methods) drags the overall contribution of the work below an acceptable level to constitute publication at this level. With much of the discussion being conjecture and the conclusions drawn, I recommend that the authors undertake a significant rewrite of the work. These significant changes would include a minimum of a heuristic evaluation by subject matter expert practitioners and to rethink or restructure this approach's discussion and take-away qualities to instrument designs that apply haptic feedback.
I offer the following insight into how each section could be improved.
Introduction.
Line 21: Are there other factors at work here, too, such as the effort/reward of learning a new instrument?
Line 26: There are several slip-ups throughout the manuscript when referring to XR. I believe that an earlier explicit definition would help clarify for the reader what XR is and what it does. I would suggest that XR is suffixed with the term "technology" to explain that XR refers to the technology used to "mix realities" (as per the accepted definition of MR on the reality-virtuality continuum).
Line 33: The authors should be careful when referring to pedagogy. The manuscript eludes to teaching and training without offering insight into the many pedagogical approaches to learning via technology in general, XR, or, more specifically, music training systems.
Line 39: Is this true of all VR controllers available today? What about the Valve Index, which offers much more accurate tracking/control via the finger-tracking technology it implements in each controller?
Line 45: "there are no consumer-level controllers available that approximate the shape or feeling of a musical instrument" is generally only valid for VR. This statement should not be made in a general sense.
Line 53: Tactile feedback is more than "touch or texture" and should use a tighter definition at this point in the manuscript.
Line 55: Take the rubber hand illusion or virtual touch, for example.
Line 65: So why use VR? If there are some difficulties or constraints with creating an instrument, why stick specifically to existing physical instrument designs?
Line 67: Is this a pedagogy paper or a technology evaluation paper? The evaluation of the virtual learning environment would require further analysis. This area of concern needs to take a backseat in the current manuscript or take the wheel. While musical exercises can be used to evaluate technology, focusing on the presented technology is essential, not the learning process.
Line 71 to 100: This section could do with being tightened up. It would be helpful to state the authors' research questions explicitly, and motivations can follow more systematically.
There is a distinct lack of reference to the existing arsenal of digital and virtual musical instruments available today throughout the Introduction section.
Background.
Line 105: Are these systems explicitly designed to provide finger prompts or incentives to practice?
Line 103 to 116: It's crucial here to distinguish between the qualities of virtual environments and immersive virtual environments. While XR facilitates both, the lack of focus here is confusing.
Section 2.1: What does this mean within the context of the Special Issue? This topic would be an excellent section to tie the relevance of haptics to art practice in virtual environments. There should also be a smoother transition between 2.1 and 2.2. Why are hands-free gestures meaningful in virtual environments?
Line 167: According to who? There need to be more references to existing haptics in music research.
Line 174: Is this true for all devices?
Section 2.2: While this section starts strong, it becomes conjecture without supporting citations or background.
Line 195: is this a musical performance or task-based performance from HCI? Many examples of this are recorded in previous research, covering the many elements of haptics, specifically in musical practice.
Line 202: This is not MR. MR is measured on a continuum. Please address the definition of MR and how XR technology mediates its different forms.
Line 256: Why? This question is an important point and needs to be expanded.
Section 2 presents an extensive supporting narrative for virtual musical instruments and introduces the problems with wind instruments. However, it lacks a lot in acknowledging the advances made in other research on this topic. While digital and virtual musical instruments have been around for a long time, there is relatively little work on including haptics, which I feel should be the primary focus here. Furthermore, no explicit research hypotheses are being raised from the background materials. How are they relevant? What is being put forwards as a guiding research question in the presented work? This last section is a significant detractor to this manuscript and may not be publishable in a journal without a formal user study attached.
Materials and Methods
The materials in this section could be presented more clearly as a flow diagram to show how they are to be examined and the variations and relationships between them. The methods are not explained adequately and are required for repeatability. While I would expect reduced numbers of participation in applied human-computer interaction research, there is currently no reason why a simple user study cannot be conducted in person (or remotely in the case presented). Suppose users are hard to come by in person. In that case, a heuristic user study should have been undertaken with subject matter experts dialing in remotely and demonstrations given in place of direct experiences with the technology.
Results.
Line 456 to 462: These areas of interest should have been presented sooner, and their relevance to the topic should have been explained earlier. Better still, they should be informed by a thematic analysis from a formal user study. Still, they should have been a central topic of discussion throughout the previous sections and appear unrelated at this point in the manuscript.
Line 463: If this is a central topic of the results section, the Virtual Environment should come under the previous section as a "material" or stimulus, and feedback should be reported in the results section.
The authors speak throughout this section on topics they have raised themselves. While attractive and informative, they are not based on a scientific opinion from a targeted and unbiased user group.
Discussion.
The discussion section fails to draw meaningful conclusions between existing literature and the sketch conclusions drawn from the results section. More work needs to be done to ground the authors' suggested findings in the existing literature to show that it is informed and robust in its conclusions. The content of this section is interesting and provides much comment on the devices examined. Still, it is hard to look past the foundations of this commentary as polemic.
Conclusions.
Line 687: The device was not tested, so this is incorrect. These findings are conjecture, and the manuscript fails to prove that what the authors observed can be extrapolated any further.
Author Response
Please see attachment
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The authors assess the potential for commercial VR as a music learning tool by comparing four virtual trumpet interfaces: camera-tracking with tracked register selection (two options), camera-tracking with voice activation, and a controller plus a force-feedback haptic glove.
The work is interesting and well writen but some points must be carefully considered in order to improve the paper quality and readers' understanding.
I believe the authors should provide more detail regarding what is a music learning tool. If the goal is that the user train with the solution to later be capable of playing the real instrument, its representation in the virtual environment should be the closest possible to the real one. That being said, the authors should focus their efforts in adapting existing trumpets to that it is possible to acquire input from them. Camera-based tracking for musical instruments usually present a well known problem, which is associated to the delay for processing the image input and later generating audio feedback to the user. That is why musicians will more likely not prefer camera-based interaction instead of other sensors, because of the delay involved in the process. It is like to sing and hear your own voice with a few milliseconds of delay. This would be enough to make the student leave the classes.
The authors make clear that was not possible to perform user tests due to covid pandemic. Unfortunately, some tests are made in order to compare the proposed solutions. I believe the theoretical metrics are important but not enough to compare them. One option would be to use one of the authors to play all the instruments and provide his insights about the experience himself. This would improve the feedback over the solution, even if it is a feedback from the authors themselves.
Other point that would help to understand the different solutions being compared is to show (using videos from youtube, for instance) how was the experience when playing each of the instruments. This would certainly help readers understand the main advantages and restrictions pointed out by table 2.
Please be more assertive in the definition of future work. "In the future, a user study would help determine how trumpet player amateurs and professionals experience these scenarios, as well as further future directions for this work." -> it is not clear to say future user studies would determine future work. please be more specific.
Some general comments and minor errors found are listed as follows.
"Music Performance)" -> "Music Performance"
"interactions, (notably" -> "interactions (notably"
"Virtual reality environments also fosters" -> "Virtual reality environments also foster"
" inform a user" -> " inform users"
"the user moves their body," -> "the users move their body,"
"of of the" -> "of the"
"mouthpiece , generating" -> "mouthpiece, generating"
"preventing a user from resting their fingers," -> "preventing users from resting their fingers,"
the term "register" is used many times before being explained. I suggest moving its definition to its first occurence in the text
"as shown in fig. 5" -> "as shown in fig. 5."
"on an an" -> "on an"
"thereby hold’s" -> "thereby holds"
"the user feels like they" -> "the users feel like they"
"Haptics GitHub7" -> "Haptics GitHub7."
"Prototype Lucidglove" -> "Lucidglove Prototype"
"by its’ physical" -> "by its physical"
"voice-activataed" -> "voice-activated"
"can be reliable detected" -> "can be reliably detected"
"the user has sufficiently flexed their finger" -> "the users have sufficiently flexed their finger"
"in the user perceiving" -> "in the users perceiving"
"just miming the actions" -> "just mimicking the actions"
"in table 1" -> "in table 1."
"version 4 prototype Lucidglove," -> "version for the Lucidglove prototype,"
"A lip measurement systems" -> "A lip measurement system"
Author Response
Please see attachment
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The article has vastly improved with the additional changes that have been made. The authors have considered the feedback and either implemented or rebutted it satisfactorily.
However, I do have one sticking point with the use of mixed reality (MR). Line 32 states, “Mixed Reality (MR) where virtual objects overlaid on the real world can be interacted with by users in the real world.” Later, on line 128, it states, “Mixed Reality (MR), which refers to technologies that combine real and virtual interactions or displays.” While these terms are “contested in academic literature,” they are not being used on the MR continuum as described in the later text. As this manuscript is not the place to argue MR is interactive AR, I suggest sticking with Milgram’s definition and removing mention of MR on lines 32 and 128 (or somehow resolving this conflict of definitions).
Other than this minor revision, I am happy for the manuscript to be published.
Author Response
Dear reviewer,
We are delighted that you found our alterations satisfactory.
In terms of the definitions of Mixed Reality, we were trying to make our definitions work with the augmented virtuality keyboard, but have realized that this was the only instance of such an interface in our paper, and it could be explained with other terms. Following your advice, we have removed the definitions of MR from the paper.
We have also done general spelling and grammar checks.
Thank you again for your thoughtful insights, which helped us improve the quality of the work.
Reviewer 2 Report
I am satisfied with the solutions and modifications proposed by the authors. I think the paper quality improved a lot, congratulations!
Author Response
Dear reviewer,
We are happy to hear you found the responses satisfactory. Thank you again for your thoughtful insights, which helped us improve the quality of our work.
