Physical skills and abilities of athletes have been examined and improved for decades applying real-time [1
] or post-game [3
] analysis. Consequently, sophisticated methods are established to assess and train relevant physical properties of athletes depending on the type of sports [4
]. While physical and technical aspects are decisive to maximize performance, gaining superior skills encompasses performance aspects beyond solely individual and isolated movement execution, especially in team sports.
In fast-paced team sports, like soccer, athletes have to face an environment that is highly unpredictable and unstable. A large field including 22 players showing constant movement makes the game dynamic and complex, resulting in almost infinite possibilities to act and react. Athletes are forced to focus their attention on relevant information sources for efficient decision-making and anticipation processes. Consequently, with pro athletes showing superior skills in these areas, knowing more about them is beneficial for various areas like talent identification and performance improvement [7
]. Sports science has identified relevant cognitive and perceptual characteristics, like the ability to recognize task-related patterns [8
] and VEA [11
] that lead to superior on-pitch performance.
Visual exploration enables athletes to gain environmental information. In this sense, exploratory actions entail eye, head and body movements supporting the identification of opportunities for successful acting within a 360° environment in soccer. To investigate VEA, a broad variety of studies focuses on gaze behavior of athletes using eye tracking technology [14
]. These studies derive recommendations about context-specific and efficient visual exploratory behavior based on multiple gaze measures, including the number and duration of fixations, saccades and viewing time [15
]. However, the application of eye tracking is still cumbersome to implement and necessitate expert knowledge to interpret results correctly [14
Taking an easily applicable approach, the potential of head movement metrics within VEA is not holistically explored. First research in this area was able to identify parameters like the frequency or excursion of head turns to be relevant [16
]. Nevertheless, these studies often lack to provide realistic scenarios, displaying simple video-based stimuli, or show a low reproducibility (e.g., on-pitch evaluations). To further explore the potential of head turn movements and hence evaluate the importance for performance analysis, more sophisticated approaches are needed, to provide a realistic and controlled assessment environment.
In this regard, technology had and continues to have a major impact on sports. For example by allowing a deeper understanding of athletes’ performance in diverse sport situations and by providing decision support for coaches and athletes. Therefore, sensor- and display-based methods were developed and employed to establish meaningful objective performance measures [17
Compared to traditional media formats, VR can allow for a greater degree of presence and hence enable training experiences closer to a realistic game context [21
]. It can further function as an amplifier for interest, motivation, and engagement of users regarding their task [22
]. The application of VR allows for the creation of an interactive sport assessment system in a sense that user actions can influence the behavior of the digital content using controller modalities and user behavior. Further, while the sensors embedded in an HMD are primarily used for determining the camera pose, they can simultaneously function as valuable data source to derive movement-based assessment measures like HTF or HTE [24
]. Since every object and action within a VE can be predefined, precise monitoring at any point in time is possible which in turn allows comparable evaluations of performance metrics [26
Even though it could be shown that inertial sensors can be used for assessing athletes exploratory behavior in different sport situations [24
], little research has been conducted using HMD integrated sensors to derive performance measures [27
]. Specifically, to the best of our knowledge, no work investigates the assessment of VEA based on head movement characteristics under the condition of realistic dynamic VR-based soccer scenarios.
In this paper, we contribute the design and implementation of a dynamic virtual soccer environment, consisting of multiple virtual plays that are created using real-world tracking data. We demonstrate the use of the simulation to assess the head motion behavior of soccer players while performing a recognition and recall task. We propose an assessment strategy and pipeline consisting of six different head turn measures based on head velocity data gathered from inertial sensors integrated into an HMD. Additionally, we utilized the gained data to investigate the focus area of athletes by determining the cFOV time on relevant objects (i.e., ball, opponents and teammates). In a user study including 28 soccer players of two different skill levels, we investigated the plausibility of the developed analysis pipeline. We further assessed the quality of the VE by measuring the experienced presence and the user experience of participants.
Results show significant differences between skill levels for the head turn measures HTC, HTE, HTF, as well as HTVel and its variability. Significant differences could also be observed for the duration of centered objects. Further, we found a significantly high correlation between the HTE and the experienced presence. The findings in this work demonstrate the validity of the investigated measures and VR as a medium for assessing the exploratory performance of team athletes. Resulting implications are important for manifold team sports assessment simulations and guide further research.
2. Background and Related Work
The human process of perception comprises interpretation and understanding of geometrical shapes or objects, registered by the visual system [29
]. Regarding the information process theory of Haber and Hershenson, interpretation is a collaboration of the central nervous system and working long-term memory [30
]. For interpretation and organizing, not only the efficient transfer of environmental stimuli is decisive for sense-making, but also previous experiences stored in the long-term memory. Memory is based on human experiences and further enables to build domain-specific structures that result in superior performance [31
]. Therefore, the ability of efficient visual exploration is an essential component for human decision-making and action execution.
VEA in sports encompasses all movements of the eye, head and body in regards to perceiving relevant environmental stimuli prior to dedicated events and actions [33
]. In team sports, it is decisive for athletes to register all situation relevant information which requires active looking behavior and hence full task engagement. This implies a constant alignment of orientation and attention regarding rapidly changing sources of information such as the position of opponents and/or teammates. Superior decision-making further requires efficient selection, processing and filtering of relevant information [36
]. To understand the effect of certain perception and search strategies on decision-making and action execution, previous research exists using laboratory and on-pitch evaluations.
To investigate search behavior and perception of athletes most prevalent eye tracking is used in previous studies for the quantification of performance. Therefore, different dependent variables like the mean, location, duration and total number of fixations as well as amplitude, frequency, latency or velocity of saccades are used. In a review conducted by McGuckian et al., the number of fixations and fixation duration could be identified as the most dominant measures to be applied for analysis [14
]. Even though yielding great potential, results of previous research are heterogeneous towards their meaning and usability for performance analysis in soccer. When comparing experienced and inexperienced players some literature concludes [12
], that the frequency of fixations is higher for more experienced players. In a study conducted by Roca et al., skilled and less skilled players experienced life-size 11 vs. 11 soccer situations in which the ball was located in the defensive or offensive half of the pitch. Overall, higher-skilled players showed more accurate anticipation and decision-making skills accompanied by a significantly higher mean number of fixation and shorter fixation duration [38
In contrast, there exists research that found fewer fixations for experienced or more skilled soccer players [9
]. Cañal-Bruland et al. conducted a study, presenting overview images of offensive and defensive game situations to a total number of 56 participants. Their study could show that skilled soccer players performed significantly fewer fixations of longer duration for decision-making compared to their counterparts. Based on the results they concluded that skilled players extract more information from a single glance [39
In a recent study conducted by Aksum et al., the search behavior of five players in the Norwegian premier league in an 11 vs. 11 game was investigated [41
]. They could observe higher fixation rates of players in case more information sources became available. Further, a shorter fixation duration could be observed in real-world game situations compared to laboratory results. The authors claim that simple laboratory studies using media like images or videos on screens may not be sufficient to capture soccer players experience under match-play conditions.
Further studies investigated the head activity of athletes to draw conclusions on exploratory activity. Relevant measures in these studies are the duration, excursion and frequency of a head turn. The majority of these studies uses a manual evaluation process annotating relevant actions supported by video footage to analyze the behavior of participants [42
]. Eldridge et al. examined the effect of VEA of midfield soccer players in relation to their actions using post-event video analysis. Within their study, VEA refers to players’ body and/or head movement away from the ball with the goal of searching for teammates, opponents or events in the environment prior to receiving the ball. Study results could show that players experienced significantly less pressure when performing more VEA before receiving the ball and made more passes into the attacking half. Overall, they claim to present evidence that more visual activity can influence tactical and technical aspects of performance. However, even though video-based manual annotation approaches can be applied for determining the visual activity of athletes, they show disadvantages due to labor intensity and are prone to human errors [42
Other research suggests the application of IMUs or MEMS to determine the VEA of athletes. Using a head-worn sensor-based method, Chalkley et al. defined a head turn as a distinct movement along the longitudinal axis that exceeds an angular velocity of 125° s−1
]. To validate their approach one participant wearing an IMU instrumented headband was placed in the center of five pre-defined labeled targets. During the experiment, the participant was asked to perform three sequences of 24 random selected trials looking at different targets. For gold standard validation, a total of 72 head turns were counted manually using video recordings. The result could be confirmed by the applied algorithm, detecting the same amount of head turns. Several follow up studies adapted this approach to investigate the effect of VEA in relation to affordance [25
], on-ball performance [46
], playing role [47
] and passing performance [16
]. To explore the latter, McGuckian et al. conducted a study in which fourteen high-skilled U13 and U23 soccer players conducted a 32-trial sequence within the Footbonaut. The Footbonaut (CGoal GmbH, Berlin, Germany) is a system that includes eight ball-dispensers and 64 square target gates. Balls can be dispensed at varying speeds, angles and spin to the athlete centered in the middle. To display stimuli (i.e., target the ball should be kicked to), gates are visually highlighted. During the study the head turn count, frequency and excursion were measured during and before ball possession. Results could show that a faster performance can be associated with a higher number of head turns before ball possession and a lower number of head turns during ball possession [16
In summary, previous research investigating the implications of VEA mainly employed methods that are based on simple displaying procedures like 2D video clips, images or require labor and/or cost-intensive systems like the Footbonaut. However, due to high immersion and presence, VR has great potential for the assessment of behavior. Especially when it comes to identifying and developing new objective metrics to quantify athletes’ performance, only few research is conducted. Addressing this research gap is crucial since it provides the possibility of getting deeper insights into the behavior of athletes while being able to provide a realistic 360° reproducible environment based on stereoscopic vision.
For different team sports there exists previous work investigating the feasibility of VR for assessing the decision-making process of team sports athletes [19
]. In a study conducted by Wirth et al. they used 360° video footage displayed via an HMD to investigate PCS of soccer players [19
]. Therefore, a variety of pre-recorded 5 vs. 5 in vivo game scenes were experienced by amateur and expert players. Results revealed a shorter reaction time for expert players. Further, the habituation-time of amateur players was found to be significantly longer compared to their counterparts.
Previous research work also investigated the applicability of VR for improving the decision-making process of quarterbacks [23
]. In a study conducted by Huang et al. an HMD was used to examine the effect of a VR-based training protocol on situational awareness of American Football quarterbacks. Different match-game situations were simulated in the virtual environment and trained over a period of three days. Results could show an increase of 30% in pre-snap reads and in-game decision-making [23
Other research work addressed anticipation skills of athletes [50
]. McLeod et al. examined different ball heading strategies of soccer players using VR [50
]. Therefore, they altered the ball’s trajectory in mid-flight from a normal quasi-parabolic path to a linear one. Results suggest that players use a combination of expectation and memory to determine the rate of change that enables them to follow a ball during flight. Comparable to this study, Hoinville et al. employed the Magnus effect to create a spinning ball simulation in a VE [51
]. Experienced and novice soccer players were asked to head during a free kick with and without sidespin. The qualitative results of the study showed that the interception performance is dependent on the ball spinning direction and arrival position for both groups.
Additionally, the anticipation of rugby players was evaluated by Bideau et al. using VR [53
]. In their experiment, they tested the ability of novice and expert rugby players to detect deceptive movements in attacking player’s run. Therefore, participants had to define the final running direction of the virtual agents under different occlusion conditions. Results of the study could reveal a significantly higher positive decision rate for experts compared to novices. Experts also showed quicker anticipation and decision-making processes. Most recent research conducted by Ferrer et al. examined the read-the-game skills of soccer players in VR [27
]. According to the authors this ability encompasses VEA, necessary for accurate in-game decision-making [55
]. For measuring VEA the long exploratory activity and 180° exploratory activity was determined dependent on participants’ head orientation while experiencing a VR-based soccer scene with a duration of seven seconds. Results show that there is a significant difference in VEA between beginners and amateur soccer players.
In summary, whereas there is research conducted investigating the performance of athletes using VR, these evaluations are mainly based on simple measures (e.g., successful actions or reaction-time) and hence do not characterize athletes’ performance in more detail. Precisely, to the best of our knowledge, there is no work that examined head motion characteristics for defining VEA of athletes in dynamic game situations using VR. To address this research gap, we developed and evaluated a new VR-based method, including 18 dynamic soccer plays, that uses different head turn-based measures, namely the head turn frequency, the duration and the excursion as well as average head turn velocity and its variability to characterize VEA of athletes. Further, athletes’ FOA is evaluated by determining the duration of relevant objects within their cFOV. The developed method gives deeper insights into athletes’ performance and consequently can be used to optimize individualized training as well as talent development and scouting processes.
5.1. Head Turn Performance
As depicted in Figure 4
, we analyzed the head turn measures in regard to the two defined groups.
For the number of head turns (HTC), we found a significant difference () between the groups with a mean of () for inexperienced compared to experienced players (). Further, the mean variability of velocity (HTCoV) was significantly higher () for the more experienced athletes () % than for their counterparts () %. For HTD only a small non-significant difference between the two groups could be observed with () ms and () ms for inexperienced and experienced players, respectively. Nevertheless, the less experienced group showed a significantly higher () HTE with a mean of () deg, whereas their counterparts had an excursion of () degrees. Further, we could observe differences in HTF, with a significance of . Inexperienced and experienced players on average performed () and () head turns per second, respectively. In terms of the overall mean yaw-velocity (HTVel), results indicate a significantly () higher value of () ° s−1 for the inexperienced group and () ° s−1 for experienced athletes.
5.2. Focus of Attention
For the FOA, we calculated the duration of how long the three different object types, ball, opponent and teammate, were closest to athletes’ cFOV during each play. A significant difference (
) for the ball between the two groups could be observed. Experienced players had a prolonged focus on the ball with a mean of
) s, whereas inexperienced athletes only spent
) s looking at the ball (see Figure 5
For opponents and teammates we also could observe a significant difference between the two groups with and , respectively. Inexperienced players observed the opponents on average for () s and the teammates for () s. Contrary, experienced players had, on average, the opponents for () s and teammates for () s within their cFOV.
5.3. Recognition and Recall
The RRS reflects the performance of recognized marked agents by the participants. The calculated Welch’s two-sample t-test resulted in a significant difference between the two groups of . Results show that the inexperienced players with correct identification of () % outperformed their counterparts, which showed a performance of () %.
5.4. Presence and User Experience
Results of the UEQ showed that inexperienced athletes perceived the system to be significantly more stimulating than experienced players. Further, a significant difference between the two groups was found for novelty (NOV). Less experienced players rated this factor higher compared to their counterparts. Table 1
summarises the results of the UEQ.
No significant difference could be found for the results of the presence questionnaire between the two groups. Inexperienced athletes (, ) rated the sensory factor higher than their counterparts (, ). The more experienced group (, ) perceived a higher level of realism than their counterparts (, ). Comparable results could be shown for distraction with a mean of () for inexperienced and () for experienced athletes. Also, for the factor control experienced athletes showed a higher rating (, ) than the less experienced (, ).
We further investigated the correlations of the individual measures, including the subjectively experienced presence and the user experience of the athletes. Figure 6
a depicts the correlation matrix of all athletes regarding the different measures by highlighting the significant results of the pairwise calculated Pearson’s correlation coefficient in color.
Results showed a strong linear positive correlation for the RRS with HTC, HTE, HTF, as well as HTVel, respectively.
Additionally, the presence scores of the participants indicate a significant linear correlation with HTE and HTVel. As depicted in Figure 6
b,c, we further investigated the correlations for the individual experience levels. For the less experienced group, we could observe a significantly high correlation for presence with HTE and HTVel, whereas for the experienced group, presence correlated significantly with HTE and HTD.
The results of the UEQ showed a high positive correlation with the PQ as well as a moderate correlation when compared to the RRS. In line with the PQ, significant positive correlations could further be observed with HTE and HTVel. For the inexperienced group the UEQ scores correlated with the RRS and the PQ. In terms of the HTVel a correlation of could be observed. Comparable to these results, the user experience results of the experienced group showed a significant high correlation with PQ and HTE as well as HTD.
When looking at the individual subcategories of the UEQ and the relevant head turn measures for both groups, HTE significantly correlated with attractiveness and dependability. Comparable HTVel showed a significant correlation with attractiveness and dependability. Further, a moderate significant correlation with perspicuity could be observed. The RRS correlated significantly with the dimensions attractiveness and perspicuity.
The correlation matrices in Figure 6
b,c depict the significant results of the individual measures for the inexperienced and experienced group, respectively.
In summary, our results show that the proposed method can identify differences in VEA assessed by head motion characteristics between experienced and less experienced soccer players. We found that the head turn performance, the FOA, and the RRS were explicitly impacted by athletes’ experience level and may therefore also act as discriminator or judgment of a specific athlete’s performance.
6.1. Head Turn Performance
Concerning the head turn performance, our study could show that the results of the developed VR-based assessment pipeline are comparable to previously conducted in vivo studies. McGuckian et al. compared the head activity of differently skilled youth players using the Footbonaut, which allows the investigation of the visual search behavior of soccer players, before and during an action [16
]. They report that superior soccer players seem to make less head turns and further have a lower excursion of these turns. In our study, a slightly lower frequency of head turns per second could be observed (
) compared to the in vivo measurements of McGuckian et al. (
]. One reason for this could be that the additional weight of the deployed HMD influenced the behavior of participants. Meaning that a lower amplitude of angular velocity might be observed when wearing the headset, which consequently results in a reduced number of head turns. Future research could determine the actual influence of weight on users’ behavior. Further, the participants may have slightly changed their behavior due to the relatively naive use of novel technology, despite our acclimatization. Based on the findings of future studies investigating the matter, the underlying head turn detection algorithm could be optimized.
Nevertheless, results in this study are comparable to in vivo experiments and suggest that our method can objectively assess VEA of athletes, and therefore optimize the examination of perceptual-cognitive skills in sports.
In our study, experienced player had a significantly lower head turn frequency. This may be caused by a beneficial attitude of orienting their body for action and observation concerning relevant events on the pitch. Consequently, they do not need of adapt their position and orientation that frequently compared to less experienced players [16
]. According to literature, professional team athletes also show superior skills and knowledge in handling and filtering relevant information during a game. Since it is more demanding for amateurs to decide on decisive visual cues, a more intense exploration of the environment can be assumed [62
]. Our findings support this theory by revealing a higher frequency and excursion of head turns for the less experienced group during the study conducted in this work.
Contrary to the absolute amount and frequency of head turns, the duration of head turns did not seem to significantly differ between the two groups. This indicates that even though athletes with less experience show a higher exploratory behavior, this is not influenced by the individual time a movement takes. We could observe an increase in HTD over trials which could be related to the need of habituating to motion in an immersive VR [63
]. Highly dynamic soccer plays were used to evaluate the behavior of athletes and to provide a comparable assessment to in vivo situations. Almost all participants had only minor experiences with an HMD and the task to perform in an immersive VE. Although a habituation phase was included in the study design, the need of getting comfortable with the VE might have caused the differences in head turn duration and consequently in excursion and frequency over trials. Even though the experience with and habituation to VR is to be considered, our results could also show that this influence seems to apply in the same way to both groups.
Another interesting observation is the difference in the variability of velocity between the two groups (HTCoV). More experienced players showed a significantly higher volatility. Meaning, the velocity characteristic of less experienced athletes can be described as performing more frequent spontaneous movements in comparison to their counterparts, who tend to have a higher standard deviation, but a lower average angular velocity. To the best of our knowledge, this effect has not been previously reported. Future research should further investigate this finding using in vivo soccer scenarios.
6.2. Focus of Attention
Our results regarding the FOA show that low experienced players tend to focus significantly longer on opponents compared to more experienced ones. Further, the latter had a significantly higher interest in observing the ball and spent more time focusing on teammates. These findings go in line with previous in vivo experiments [12
]. For example, Vaeyens et al. examined the decision-making and search behavior of soccer players. They used realistic film simulations of offensive plays in combination with eye tracking assessments. Results suggest that higher-skilled players tend to fixate more extensively on the player in ball possession compared to less skilled players [12
]. An explanation could be the advanced skill of gathering and selecting peripheral information of professional players. It gives them the possibility to have a central focus of the action, but receive relevant context information at the same time. Unlike, amateur players tend to shift their focus more towards the information sources to be explored [32
]. During our experiment, the more experienced players spent more time focusing on the ball and the teammates, which indicates that they showed higher interest in the ball position and the player in ball possession. Our method did not explicitly investigate the player in ball possession, but rather the focus on the ball, opponents, and teammates individually. Future research could include a more detailed differentiation of objects of interest (e.g., including the player in ball possession).
Recently published work by Aksum et al. indicates that the search behavior of professional athletes differs between laboratory results [37
] and actual on-pitch performance. They conducted a study where Norwegian premier league midfielders’ gaze behavior was analyzed during an 11 vs. 11 play. Precisely, they observed that in competitive situations, professional players’ fixation locations were mainly on the ball, opponent, and teammate category. Since their observations are partially contradictive with previous literature, they conclude that traditional laboratory settings may not be sufficient to provoke the behavior of soccer players performed in dynamic soccer competitions [41
]. The results found in this VR study seem to agree with the observations of Aksum et al. [41
Overall, we provided a new approach to determine athletes’ FOA in dynamic VR-based soccer scenarios. Its capability to distinguish different experience levels contributes by creating a more holistic view of athletes’ behavioral characteristics in dynamic game situations and hence informs future research and development. To enable a more detailed analysis for future research in VR, this approach could be optimized to allow the differentiation of more object types like the player in possession of the ball. Another alternative would be the investigation of gaze behavior using eye tracking to be able to not only provide information of the center focus area of an athlete, but additionally determine the area of interest. However, results of exploratory behavior of athletes seem not to be clear when looking at previous literature [12
Nevertheless, the applied method could provide comparable results to previous in vivo experiments. This indicates that the employed VR-based approach is an effective alternative to in vivo measurements for assessing soccer players’ FOA. Further, having a holistic view of the athlete by providing more head-based exploratory measures can be beneficial to understand the behavior of athletes in dynamic competitive situations.
6.3. Recognition and Recall
In our study, the findings for the RRS showed that, unexpectedly, less experienced players reached a significantly higher score of approximately
correct answers, whereas their more experienced counterparts reached an average score of
. However, previous literature indicates that experienced players only possess superior cognitive knowledge and processing when it comes to structured plays [66
]. The reason for this is that experienced soccer players have developed knowledge that facilitates meaningful associations between players’ positions and their relevance in game-specific situations [66
]. The marking of the player agents in this study was rather unstructured in a way that the number of marked players varied over plays. The non-task specific (i.e., unstructured) nature of these visual hints might be one reason for the higher recall error of experienced players observed in this study.
Another reason for this finding could be the restriction in peripheral vision. According to previous in vivo studies, superior soccer players make more use of peripheral information than their counterparts [11
]. Since the utilized HMD provides a FOV of 110°, the peripheral vision of the athletes was restricted during the experiment and could therefore have impacted their recall performance.
In addition, we found that the RRS strongly correlated with the HTC, HTE, HTF, and HTVel. These results revealed that a more intensified overall head activity led to a higher recognition and recall ability for unstructured visual information during the experiment. While for less experienced players this could be observed for the frequency of head turns, experienced players’ RRS also showed a strong positive correlation for HTE. Previous literature states that extensive exploratory activity within the period before receiving the ball of soccer players results in more successful actions with the ball [43
]. In this regard, our results provide supporting evidence.
In summary, the assessed RRS could show significant differences between the two examined experience levels. The positive correlations found between the head activity measures applied in this work and the RRS underpin the assumption that higher exploratory activity in team sports leads to more information retrieval decisive for successful decision-making [42
]. However, in our experiment, less experienced players outperformed their counterparts in recalling the number of displayed stimuli, which could result from the unstructured nature of the applied task. To get deeper insights into the applicability of realistic and dynamic VR environments for the evaluation of athletes’ recall and recognition behavior further studies using a structured task design should be conducted.
6.4. Presence and User Experience
The employed VR simulation achieved a high score when comparing user experience results to the UEQ benchmark dataset, except for the factor dependability. The reason for the low rating might be that the task of the participants within this study was to only explore the environment and hence no direct interaction possibilities to manipulate the VE were given. For future research, this could be improved by providing intuitive interaction in such a sports scenario as foot interaction.
For the UEQ we could observe significant differences for the factors novelty and stimulation between the two investigated experience levels. Reasons for such a difference may lie in the fact that higher experienced athletes are more used to technique supported assessment and training methods. Prior use of HMDs should not have influenced the results, since the experience with those devices was comparable between the two groups. Previous literature could show that athletes’ personality traits affect the perceived user experience in VEs [10
]. Hence, the player types and also experience levels seem to be relevant when it comes to decisive motivation factors like novelty and stimulation [68
]. We therefore follow the suggestion of Wirth et al. to not only consider task-relevant aspects (e.g., realistic dynamic sports scenes), but also consider hedonic quality (i.e., novelty and stimulation) as criteria when designing VR-based assessment and training simulations in sports.
We could not observe significant differences in presence between the two groups. Concerning the UEQ, this is an interesting finding, since even though the experience level seems to significantly influence the perception of a simulation’s hedonic quality, the same does not apply for the perceived presence. From our perspective, this underpins the suggestion of having additional quality measures besides the factor of presence to get a holistic view of the user experience when evaluating VR simulations.
The subjectively perceived presence of athletes showed a significant positive correlation with HTE and HTVel. While this could be observed for less experienced players, presence correlated significantly with HTE and HTD for more experienced athletes. From our perspective, HTE could be a relevant parameter, to objectively define presence. As stated by Usoh et al., an increased presence results in a greater similarity of comparable users’ real-world behavior in VEs. Therefore, such measures yield great potential to relate to presence [69
]. This potential could, among other research [70
], be shown by Freeman et al. [72
]. Their study of presence revealed effects on postural response when users perceived a variety of stimuli in an immersive VE [72
]. Moreover, behavior measures like posture seem to be efficient since users are not conscious of their observation and hence subjective biases can be excluded [71
However, to the best of our knowledge, there is no research work yet which could show that users’ excursion of head turns (HTE) strongly correlates with their subjectively perceived presence for an exploration task in VR.
In summary, this work contributes in various ways. We created a new analysis pipeline consisting of multiple head motion characteristics that have not been investigated in a VE before. These measures were applied within a dynamic recognition and recall task to examine the VEA of inexperienced and experienced soccer players. Further, the experienced presence and overall user experience of participants regarding the VE was evaluated. Our results show that our approach could be successfully used to distinguish between different experience levels of players and can therefore be used for VEA assessment. In team sports, this is highly relevant since it enables to design more efficient assessment processes and hence is supportive in creating a more holistic view of athletes to optimize individual performance improvement processes.
We further found that the subjective measure of presence strongly correlated with the HTE of participants. Since to the best of our knowledge, the correlation between those two variables has not been examined in any previous research, we see this as an additional major contribution of this work, that needs to be further investigated.
Results of the head motion characteristics showed a significant difference between the two investigated groups in terms of HTC, HTCoV, HTE, HTF and HTVel. The findings go in line with comparable in vivo studies conducted in team sports. Consequently, this makes these measures a promising tool for the assessment and training of athletes in VR. The results further strengthen the applicability of VR as an efficient immersive medium for performance analysis in sports.
Another contribution of this work is the exploration of athletes’ focus of attention in VR. Therefore, an approach was chosen and evaluated that measured the focus time of soccer players on the visual ball, opponents and teammates. Significant differences were found for the two examined groups varying in experience level. This could be further improved in the future by using eye tracking.
Less experienced athletes outperformed their counterparts in the recognition and recall task. One reason for this might be the lack of full natural peripheral vision caused by the used HMD. Professional soccer players tend to derive decisive information from this visual area more extensively than amateurs [11
]. This is currently a limiting factor of the hardware devices, therefore, it can be expected to vanish with newer generations of HMDs.
For the subjective experience of athletes, we found significant differences between the two groups for hedonic quality (i.e., stimulation and novelty), whereas no significant variation could be observed for presence. However, in sports, motivation and the experienced stimulation of the athlete play a decisive role. Therefore, to get a holistic view on the user experience for the VE to be developed, derived from the observations made in the conducted study we recommend to not only investigate the presence of an athlete, but also consider motivational linked factors like novelty and/or stimulation.