Exploring the Role of Distraction in Weak Flow–Performance Link Based on VR Searching Tasks

Abstract

The weak association between flow experience and task performance (also known as weak flow-performance link) can reduce the positive effect of virtual reality (VR) applications. Distraction caused by incongruence between the primary task and interactive artifacts may be a direct factor leading to the weak link, but it has still not been tested. To empirically test this assumption and explore approaches to alleviate it, we developed the ‘VR searching paradigm’ and a prototype VR system, based on which three comparative experiments were conducted. Study 1 tested the effect of distraction and proved that high levels of distraction caused by incongruence can lead to the weak link (β = 0.198, p = 0.391). Next, two common design guidelines were proposed to deal with distraction. Study 2 tested the effect of reducing conspicuous but task-irrelevant distractors (guideline 1) on flow-performance link. Study 3 tested the effect of providing visual cues (utilizing distractors to achieve task-oriented selective attention), which is guideline 2, on flow-performance link. The results of studies 2 and 3 revealed that both guidelines helped enhance the task performance without damaging flow experience, alleviating the weak link problem (β = 0.351, p = 0.031; β = 0.255, p = 0.041). Our results provide the first piece of evidence that directly proves the effect of distraction on weak flow-performance link, which helps improve the explanation of the mechanism. Moreover, this paper is the first that proves the effectiveness of two easy approaches to alleviating the weak link by way of guiding the user’s task-relevant attention.

1. Introduction

Flow experience (shortened to ‘flow’) represents a highly enjoyable mental state where an individual is fully immersed and engaged in the process of an activity [1,2,3,4]. It is seen as a key component of high-quality user experience in virtual reality (VR) activities due to its contribution to the enhancement of task performance. However, the link between flow and task performance was found to be weak in studies conducted in many virtual environments, showing a weak flow-performance link [2,5,6]. Bian defined it as weak association at first [2] and renamed it weak flow-performance link in a follow-up paper [7]. If the enhancement of user experience cannot directly promote performance effectively, pursuing a good experience would be less meaningful and may even be counter-productive, especially for learning, training, and education [2].

In a previous study, the weak association model (WAM) was proposed to explain the mechanism of weak flow-performance link in virtual environment (VE) [2]. It reveals that the reason leading to the weak link was the disjunction/incongruence between the primary task and interactive artifacts. It further points out the disjunction/incongruence caused by disjointed features distracts the user’s attention and interest away from the primary task, which might be a direct cause of the weak link. However, it has not been tested directly.

Games of different genres were used to test the WAM in previous studies [2]. Although diversity of genres can help prove the generalization of the theoretical model in different types of VEs, it might lead to extra variables that are difficult to control and may play a role in the results. Therefore, to clarify the mechanism of weak flow-performance link, the VE genre should be controlled. To solve this problem, a VR task paradigm needs to be developed to enhance the comparability of different conclusions on a broad scale.

In this paper, we further validate the WAM, and clarify the reason for weak flow-performance link hidden in poor VR design. Moreover, it develops effective approaches to alleviating this problem by guiding attention/distraction in VR activities. Our work provides methodological improvements to exploring flow-performance link. Moreover, it helps to clarify some UE problems and has practical significance for promoting optimal VR design. Overall, this paper makes the following contributions:

• This paper improves the ‘VR searching paradigm’ and develops an experimental VR system, which is the first paradigm and experimental prototype system to specially explore flow-performance link in VR.
• This paper provides the first empirical evidence directly proving the effect of distraction in weak flow-performance link, i.e., distraction caused by disjunction is an antecedent, which helps improve the explanation of underlying mechanism of weak link.
• This paper is the first to provide design guidelines with a view to guiding task-relevant attention and prove its effectiveness at alleviating the weak flow-performance link and optimizing the VR design.

2. Related Work

2.1. What Is Weak Flow-Performance Link?

Although many authors argue for a positive association or even a causal relation between flow and performance, the empirical evidence documenting such an association is meager. For instance, in some previous studies, the influence of flow on performance in playing computer games was not found after controlling different playing modes [2,8,9,10]. Engeser and Rheinberg [5] conducted a study in which participants played a computer game at different degrees of difficulty. When controlling for the baseline performance, they found that even at an appropriate level of difficulty, flow experience only explained a small amount of the variance in performance, and this effect was only marginally significant. To our knowledge, there is not a single methodologically adequate study available in the literature documenting the causal effect of flow experience on (subsequent) performance.

In two previous studies, Bian et al. [2,7] paid attention to the weak flow-performance link in their recent work and verified the universality of it. In their study, the weak flow-performance link was found in different activities in a VE, such as virtual Tai Chi learning, VR shooting game, and VR tennis game. Moreover, they discussed the possible causes of the weak flow-performance link in these virtual activities and examined them. Finally, they established the WAM.

2.2. Explanation of the WAM

The WAM attempted to explain the reasons for the weak flow-performance link in VEs [2]. Based on the model, flow can be produced in two ways: performing the primary task, and manipulating artifacts. Only the flow produced from performing the primary task can directly improve the task performance, flow from the latter only influences experience but not performance. Many VEs have disjointed features (i.e., the use of interactive artifacts is incongruent with the primary task). In these VEs, flow might not be produced from performing the primary task (black arrows in Figure 1) but from using interactive artifacts (white arrows in Figure 1). In this case, there may be a strong link between flow and experience, but the flow-performance link will be weak [2].

Figure 1. The weak association model (WAM) developed from [2].

It reveals that the incongruence between interactive artifacts and the primary task may be the reason for weak flow-performance link. Attention may play the role of an important mediator. Irrelevant artifacts may distract users’ attention and interest from the primary task [2]. Therefore, the distraction caused by disjointed features may be a direct cause of weak flow-performance link in VEs.

2.3. Distraction and Selective Attention

To pursue the mechanism of weak flow-performance link, we argue that it is essential to draw upon the study of distraction and selective attention. Selective attention can be broadly defined as the ability to facilitate the processing of one source of environmental information while attenuating the processing of others [11].

According to limited resource theory, people have a limited capacity system to process visual information. Therefore, an act of selection must occur at some point, after which only some of the available information can be processed further [12].

There is load in cognitive processing. The LT indicates that the perceptual load influences selective attention. The LT framework is well rooted in traditional cognitive models of dual-task information processing, demonstrating behavioral costs when attention is divided among relevant stimuli [12,13,14]. LT is an extension of these ideas into the realm of task-irrelevant information processing.

Usually, there is some physical distinctiveness between the task-relevant and task-irrelevant information (e.g., spatial location, color). In the design of ideal tasks, the allocation of perceptual processing capacity should first prioritize task-relevant information and then, if capacity remains, task-irrelevant information. In other words, the allocation of perceptual resources should occur first to relevant information and then to irrelevant information.

However, in practice, task-irrelevant information is often allocated much attention in the experimental environment. Distractions in activities are common and often unavoidable [15,16]. For example, in some VR activities, rich virtual scenes often have some task-irrelevant visual information. Interactive artifacts provide distracting notifications while users are performing the primary tasks, competing for the user’s attention. Irrelevant parts of them (distractors) can be distracting because users have limited cognitive resources to allocate to the primary tasks. It results in a reduction in task performance [17]. Therefore, features of disjunction in VEs may lead to distraction, and then weaken the flow-performance link [2]. However, this viewpoint has not been directly verified.

To test this viewpoint, we first put forward a VR task paradigm to avoid potential interference of different task genres in studying weak flow-performance link.

3. A VR Task Paradigm for Flow Study

In flow studies, experimental manipulations for inducing flow are essentially based on a variation of the difficulty levels of the respective task. The difficulty should be either experienced as too low, too high, or as largely compatible with the individual’s skill level [18]. According to this general logic, several previous paradigms were proposed by using different types of tasks (i.e., in the math, chess, knowledge quiz, and mental arithmetic paradigms). Flow studies in VEs also induce flow through manipulating difficulty levels of various tasks [19], but there is no relatively fixed task paradigm. There may be two existing problems:

Level of task difficulty is hard to design properly. It should be noted that the specific operationalization used to vary the difficulty levels of the task affects the quality of flow experience during task engagement. Therefore, it is crucial to take into consideration that participants may frequently experience “interruption from flow”, “give up”, or not engage in the task any more after some experiences of failure. The ideal overload experience should reflect a continued struggle instead of a sudden change. Although some flow task paradigms have attempted to solve this problem, it still exists (e.g., in the mental arithmetic algorithm, although the algorithm is designed in a way that the difficulty level decreases—to a still relatively appropriate level—after a certain number of failures, the individual has experienced these failures and interrupted the flow state).

Existing task paradigms are not applicable to study the problem of weak flow-performance link, especially the role of distraction. One reason is that the existing flow task paradigms often avoid the emergence of distraction and therefore do not include the manipulation of distractors. Another reason is that some existing classic paradigms (e.g., in the math, chess, knowledge quizzes, and mental arithmetic paradigms) are not suitable for flow studies in VR activities, and there is still no paradigm sophisticated enough for a VR task.

To address these problems, a VR task paradigm for flow study is proposed in this paper. This paradigm should: (1) ensure the overload experience to reflect a continued struggle (only in this way can a continuous and fluent flow experience be induced) and (2) be able to manipulate distraction. Inspired by past studies [20,21], we proposed a novel task paradigm that is called the VR searching paradigm. The task paradigm is applicable to VR activities, and especially suitable for flow studies in the VEs. The principle of this paradigm is given as follows:

This paradigm is based on the LT and limited resource theory. In the VEs, a primary task with stable challenge level should run throughout the whole process consistently. The primary task here is a visual search and collection task when navigating in VR. Using this task, the difficulty is steady and easily matches the individual’s skill level, i.e., if the skill level is high, they will find the targets faster, if the skill level is low, they will find the targets slower. Therefore, it guarantees the task challenge’s good adaptivity to individuals’ skill during the entire task, and it is universal for people with different levels, unlike the abovementioned paradigms that require individuals to have a certain background of knowledge.

When performing the primary task, there are potential distractors that may distract the user’s attention and interest (see Figure 2). According to the manipulation of distractors (such as manipulating the type, fun, intensity, or number of the distractors), different levels of attention allocation will be induced, and this may lead to different levels of attention when performing the primary task. The more distractors there are, or the more attractive/disruptive they are, the more attention may be transferred, thus the flow from performing the primary task will decrease. By comparison, the less distractors there are, or the less attractive/disruptive they are, the less attention will be transferred, and then the flow from performing the primary task will increase. It should be noted that the distractors here are different from interruptions. Distractors do not suddenly intrude or appear in a VR scene to interrupt the process of engaging in the primary task, but constantly exist in the scene due to poor design. The task paradigms can help the flow study, especially in exploring the role of distraction in weak flow-performance link.

Figure 2. A task paradigm based on a VR searching task.

Next, a VR task system is developed according to this task paradigm.

4. Construction of Testing System

We developed a VR system according to one of Bian’s previous studies [7], and designed the primary task and distractors according to the proposed “VR searching paradigm”.

4.1. Storyline Design

A VR underwater-treasure-hunting system is constructed for our experiments. To make it easy for the user to be involved in the VE and the task, we designed a storyline [4].

At the beginning of the task, the user falls into a mysterious area under the sea (see Figure 3). The user is “cursed”, and then turned into a king crab (using avatars helps to enhance immersion, and the degree of immersion increases the body ownership, agency, as well as the feeling of presence [22]). There are eight treasure chests scattered on an underwater mountain with many curves and slopes. After finding all the treasure chests successfully within 4 min, the user can escape from the sea and become human again. Otherwise, they will “be a crab forever” and it is “game over”.

Figure 3. The user plays as a king crab in the VR treasure hunting system (from the third person view).

4.2. Natural Interaction

Our study mainly refers to the role of visual distraction. To avoid unnecessary distractions due to poor interactions, we adopted a first-person perspective and embodied interaction design.

To improve the sense of presence, the user navigates in the VE using the first-person view. A near to real underwater exploration experience is simulated with an HTC vive: (1) The controllers were embodied into a pair of virtual crab claws in the VE. (2) user’s angle of view was controlled by adjusting the head movement and orientation. By simply using a swivel chair (just for rotation, not actual movement), users can move like a real crab to control the orientation in the VE (Figure 4). (3) In addition, the user can control the movements and open the treasure chests by using the handle. (4) The player can interact with other marine creatures. By blowing out a virtual water column, the user can drive marine creatures away (Figure 5). (5) The feeling of water resistance under the water is simulated by using a visual motion delay effect [23].

Figure 4. The equipment of our VR testing system.

Figure 5. If there is a marine creature in the way (left), the player can drive the marine creature away by blowing out a jet of water (right).

These interaction designs are integrated with an immersive VE, and they will eventually make the user feel like exploring the virtual world with their own body [24], which can provide a good, embodied interaction experience.

4.3. Practice Scenarios

A training scenario was developed for the user to practice the necessary operations in advance. The system gives instruction to the user as “Your new body has some new abilities, so get used to it”. Then, the user follows visual and auditory instructions to practice. After practice, the system says, “Now that you’re familiar with your new body, there’s going to be a formal task”.

4.4. Formal Scenario: Primary Task and Distractors

The formal scenario is a mysterious sea area with various kinds of marine creatures. The primary task and distractors were designed according to the VR searching paradigm to investigate the player’s response.

The primary task is to find eight targets (i.e., treasure chests) as soon as possible within the limited time (Figure 6a). Distractors in this VE are various sea creatures randomly navigating in the virtual scene (Figure 6b). Although the player can interact with them, they do not play a role in the primary task (incongruence), thus may attract the user, and then interfere with his/her attention on the task. Based on this VR system, we conducted three empirical studies in the following sections.

Figure 6. The primary task is to find treasure chests (a); some examples of distractors are shown in (b).

5. Empirical Study 1: Testing the Role of Distraction in Weak Flow-Performance Link

Based on the analysis above on WAM and distraction in Section 2, the first question we need to verify is the assertion of WAM. The core assertion is that distraction caused by disjointed features maybe a reason of weak flow-performance link, which is the Hypothesis (H1) in this study.

Hypothesis 1 (H1).

In VEs where the artifacts are disjointed with the primary task, distraction caused by disjointed features is a direct antecedent of weak flow-performance link.

5.1. Design

A within-subjects quasi-experimental design was adopted to test the hypothesis. It is commonly used in situations occurring in natural settings where full experimental control is lacking, allowing the researcher to introduce something like true experiment design. With this design, both a control group and an experimental group is compared, however, the groups are chosen and assigned out of convenience rather than through randomization [25]. Participants were asked to perform a specific task, and the frequency of distracted interaction during the task was recorded as an independent variable. Two levels of distracted interaction would be distinguished: high level and low level. The operational definition of high level (of distracted interaction) in this study is “having more interactions (about the top thirty percent) with irrelevant marine creatures while performing the primary task”. It reflects a strong tendency to shift attention away from the primary task. The low level (of distracted interaction) is operationally defined as “having little interaction (about bottom thirty percent) with irrelevant marine creatures during the primary task”. It reflects a weak tendency to shift attention away from the primary task. This method of dividing high-level and low-level groups is commonly used in psychological experiments. The dependent variable is the flow-performance link. Considering that the participants’ existing familiarity with VR and visual discomfort during the task may affect flow experience and concentration, they were controlled as extra variables.

5.2. Participants

In this study, we recruited 32 volunteers (15 males and 17 females) to participate in the study. The age ranged from 14 to 34 years old (M = 24.75 years, SD = 6.49 years).

5.3. Environment and Procedure

After practice, all participants performing the task two times (we designed two parallel tasks by changing the location of treasure chests) according to their own intention. When finished, the participant immediately completed an online questionnaire.

5.4. Measures

Flow Experience. The flow of participants during the task was measured with Flow Short-Scale. The scale has been proven to be an effective instrument to measure flow in VR activities [19,26]. The participants answered these items on a seven-point Likert scale from 1 (I don’t agree) to 7 (I agree). The reliability of the scale was good in this study.

Task Performance. The task performance of participants was measured by the number of treasure chests found during the task. Most participants could not find all eight chests in 4 min, thus there was no ceiling effect.

Distraction. Distraction was evaluated with the frequency of distracted interaction, which is the time spent shooting irrelevant marine creatures during the primary task. This indicator is not extremely sensitive and so does not reflect the state of distraction, but it can be used as an external indicator of distraction. The data were recorded in the system.

Visual Comfort

The Visual Comfort Questionnaire was used to assess the participants’ visual comfort during the task. The questionnaire was developed by Zhou et al. [27] from referencing Lambooij et al.’s questionnaire [28]. The questionnaire evaluates the overall visual experience in VR activities from four aspects: 3D experience, naturalness, viewing experience in interaction (including three indicators: comfort in stability, fluency and viewpoint), image quality, and avoidance of discomfort (including two indicators: avoidance of dizziness and avoidance of fatigue). The participants answered these items on a scale with the adjectives [bad]-[poor]-[fair]-[good]-[excellent].

5.5. Results

A total of 64 valid data were collected. We firstly performed a pre-test to ensure that the weak flow-performance link occurred in our constructed VR environment. According to Bian et al. [2,7], the weak link was tested with the regression analysis. A significant prediction from flow to performance means strong flow-performance link, while an insignificant prediction from flow to performance means weak flow-performance link. Then, correlation analysis and regression analysis were performed. Results showed that the correlation between flow (M = 61.13, SD = 7.94) and task performance (M = 4.45, SD = 1.59) was low and the prediction from flow to performance was marginally significant (β = 0.215, p = 0.086). These results showed that the designed distractors could be used as disjointed features to induce weak association to some extent.

Next, to examine the role of distraction in weak link, 44 data (half in high distraction condition, half in low distraction condition) were finally selected for analysis according to the operational definition of high distraction and low distraction. The visual comfort and familiarity with VR during the task were measured, and there was no difference between the two groups (ps > 0.05).

After performing correlation analysis and regression analysis, results showed that there was no significant correlation between flow and task performance in high distraction condition, and flow did not significantly predict task performance (β = 0.198, p = 0.391). In comparison, data from low distraction condition showed that there was significant correlation between flow and performance, and flow significantly predicted task performance (β = 0.437, p = 0.042). These results support H1 and demonstrate that in VEs where the artifacts are disjointed with the primary task, distraction level caused by disjointed features is a direct antecedent of weak flow-performance link. When obvious distractions occur, this problem will arise.

If the results of this study support that distraction is an antecedent of weak flow-performance link, how can we alleviate this problem by dealing with distractions? WAM suggested a basic design guideline [2], which is to improve the congruence between interactive artifacts and primary tasks (or reduce the disjointed features on the other side). Based on the basic guideline, two more specific design guidelines can be proposed to deal with distractions in VR activities: (1) Reducing conspicuous but irrelevant distractors directly; (2) Increasing the congruence between distractors and primary task by guiding attention. As to the latter, we got inspiration to utilize visual cues to guiding attention from some previous work. Grogorick et al. proposed an adaptation of different existing gaze guidance stimuli to immersive environments and investigated the efficiency of five different gaze guidance techniques [29]. In augmented reality-based assistance system, guiding attention towards the relevant targets will reduce the time needed for visual search and reduce errors, based on the several attention-guiding techniques developed in [30]. We think it is a constructive way to guide attention and transfer distracted behavioral consequences to task-relevant cues through appropriate visual or interaction design. The effectiveness of the two guidelines was tested in the following studies 2 and 3.

6. Empirical Study 2: Effect of Reducing Distractors on the Weak Flow-Performance Link

The purpose of this study is to test the effectiveness of guideline 1. According to this guideline, we proposed Hypothesis 2 (H2).

Hypothesis 2 (H2).

Reducing task-irrelevant distractors can help alleviate the problem of weak flow-performance link.

6.1. Experimental Design

We adopted a single factor between-subject design with distractor conditions (reducing distractors/control condition) as the independent variable. Using between-subject design can control the potential learning effect/ordering effect. Dependent variables are flow, task performance and the flow-performance link. The visual discomfort and familiarity with VR game were still controlled as extra variables.

6.2. Participants

A total of 38 volunteers (17 males and 21 females) were recruited to participate in the study. The age of the sample ranged from 14 to 31 years old (M = 22.93 years, SD = 3.02 years). Participants were randomly divided into two equal groups, one group was assigned to the condition of reducing distractors, and the other group was assigned to the control condition.

6.3. Environment

In contrast to study 1, two versions (version 0 and 1) of the VR system were developed in this study to test H2.

Version 0 is the control condition. It is the same VR system (with distractors) as that in study 1 (Figure 7a).

Figure 7. Differences between version 0 (a) and version 1 (b) on distractors. (a): Many kinds of marine creatures navigating in the scene as distractors. (b): Very few distractors appear.

Version 1 is developed based on design guideline 1. Specifically, we remove the conspicuous marine creatures that wander around in the virtual scene (see Figure 7b). Since there is no change in the primary task, it will not affect the difficulty of the task.

6.4. Procedure

Before the study, we first explained the VR tasks to the participants in advance. After that, each participant played one of the two versions. When they finished (or the time is out), task performance was recorded by the system. Then, the participant immediately completed a questionnaire.

6.5. Measures

The measures of flow, performance, and visual comfort were the same as study 1.

6.6. Results

We firstly conducted two independent-sample t tests to investigate the difference in visual comfort and familiarity with VR between the two versions. Results showed that there was no significant difference (ps > 0.05). Thus, these extra variables were controlled without affecting the subsequent analysis.

Additional independent-sample t tests were performed to investigate the difference in flow and performance between the two conditions. Results showed that there was a significant difference in flow [t (37) = 2.906, p = 0.005], and participants in version 0 had higher flow than those in version 1 (Figure 8a). The difference in task performance was also significant [t (37) = 5.302, p < 0.001]. Contrary to the results of flow, participants in version 1 achieved better task performance than those in version 0 (Figure 8b).

Figure 8. Differences between the effects of these two versions on flow (a), and task performance (b). In (a), flow in version 1 is significantly lower than version 0. In (b), performance in version 1 is significantly better than version 0 (p < 0.01). ** means p < 0.01; *** means p < 0.001.

Correlation analysis and regression analysis were conducted un-der each condition. Results showed that the correlation between flow and performance was not significant in high distraction conditions, and flow did not significantly predict performance (β = 0.110, p = 0.509). In comparison, data from low distraction condition showed that the correlation between flow and performance was significant, and flow significantly and positively predicted task performance (β = 0.351, p = 0.031).

The results showed that reducing conspicuous distractors not only strengthened the flow-performance link but also improved the task performance. However, this approach impaired the quality of flow experience to some extent. It meant that although distractions reduced task performance, they helped achieve better experience during the task. Anyway, these results supported H2, suggesting that guideline 1 was an effective way to alleviate the weak link. Next, experiment 3 tested the effectiveness of guideline 2.

7. Empirical Study 3: Effect of a Congruence Design Approach on the Weak Flow-Performance Link

The purpose of this study is to test the effectiveness of guideline 2. According to this guideline, we proposed Hypothesis 3 (H3).

Hypothesis 3 (H3).

Promoting the congruence of distractors and primary task can help alleviate the problem of weak flow-performance link.

7.1. Design

We conducted another comparative experiment. In contrast to study 2, we adopted a within-subject design with the congruence conditions (congruent/control condition) as the independent variable. Dependent variables are the same as study 2.

The reason for changing the study design is as follows: although we try to control individual differences (such as familiarity with VR game and visual comfort), there might be others that may affect the results. Within-subject design can avoid the problem. However, it brings the risk of rising learning effect or order effect. To avoid this effect, the positions of the treasure chests are different in the two conditions without changing the task difficulty (the perceived difficulty was controlled as an extra variable). Moreover, we counterbalanced the order of experiencing the two conditions.

7.2. Participants

A total of 65 volunteers (35 males and 30 females) were recruited to participate in the study. The age of the sample ranged from 14 to 45 years old (M = 23.05, SD = 5.11).

7.3. Environment

We set two versions (version 0 and 2) in this study to test H3.

Version 0 is the control condition. It is the same VR system (with distractors) as that in study 1 (Figure 7a).

Version 2 is a VR system developed according to a design guideline.

In this version, the number of distractors were not reduced. To improve the congruence between distracted interactions and the primary task, visual cues were designed to utilize distractors to provide attention guidance [31], and then achieve more task-oriented selective attention. When the participants actively attack task-irrelevant creatures due to distraction, different from version 0, the attacked creatures will move towards the nearby treasure chest. In this way, the participants’ attention is probably unconsciously shifted back from the distractors to the primary task. The rest of the conditions (including the perceived difficulty of the task) in the two versions remain the same. The differences between the two versions are shown in Figure 9.

Figure 9. After frequently interacting with the marine creatures, version 0 (a) and version 2 (b) show differences in outcomes. (a): The creatures still move randomly after being attacked. (b): The creatures move towards the location of the nearby treasure chest more frequently after being attacked.

7.4. Procedure

Before the study, we first explained the VR tasks to the participants in advance. After that, each participant experienced the two versions of the tasks, and the order of experiencing the two versions was counterbalanced. When they finished (or the time is out) each time, task performance was recorded by the system. Then, the participant immediately completed a questionnaire.

7.5. Measures

The measurements of flow, performance, and visual comfort were the same as those in studies 1 and 2. The perceived difficulty was assessed on a scale with the adjectives [very easy]-[easy]-[moderate]-[difficult]-[very difficult].

7.6. Results

We firstly performed a paired-sample t test to investigate the difference in perceived difficulty between the two conditions. Results showed that the perceived difficulty of participants was moderate (M = 3.46, SD = 0.47) and that there was no significant difference [t (64) = 0.659, p = 0.512]. Thus, this extra variable was controlled well.

7.6.1. Effects on Flow

A paired-sample t test was conducted to test the difference of flow experience between the two conditions. It was found that the difference in flow between version 0 (61.13 ± 7.94) and version 2 was not significant (see Figure 10a).

Figure 10. Differences between the effects of these two versions on flow (a), and performance in the primary task (b). In (a), there are no significant differences between version 0 and version 2. In (b), performance in version 2 is significantly better than version 0 (p < 0.01). ** means p < 0.01.

7.6.2. Effects on Performance

We conducted another paired-sample t test to test the difference in performance (see Figure 10b), and a significant difference was found [t (64) = 3.313, p = 0.002]. Specifically, the participants in version 2 (5.18 ± 1.74) had better task performance than those in version 0 (4.45 ± 1.59). It indicated that the approach of providing attention guidance did help participants improve their task-relevant behavior and performance.

7.6.3. Effects on the Flow-Performance Link

Correlation analysis and regression analysis were conducted under each of the two conditions. Results showed that the correlation between flow and performance was not significant in version 0, and flow did not significantly predict task performance (β = 0.215, p = 0.086). In comparison, data from version 2 showed that the correlation between flow and task performance was significant, and flow significantly predicted task performance (β = 0.255, p = 0.041). It could be seen that the correlation between flow and performance did increase to some extent after promoting congruence design.

These results basically support H3 and demonstrate that distraction behavior can be transformed into task-relevant behavior by adding visual cues to provide attention guidance. When attention shifts from distractors to the task, the weak link will be alleviated.

8. Discussion

Our studies further verified the WAM. Some meaningful results were found through three empirical studies.

8.1. The VR Task Paradigm

In this paper, we developed a VR task paradigm. Based on this paradigm, we constructed a VR system and conducted a series of studies. From the research results, it is concluded that this paradigm can be well used to investigate flow experience and flow-performance link in VR activities.

This paradigm was proposed based on the LT and limited resource theory. From the perspective of cognitive resources allocation, it can be used to investigate how VR users’ attention resources are allocated in the process of completing tasks with distractors.

Different from the interruption diagram [32], in which the primary task was interrupted in special phases, the primary task is not interrupted in our paradigm, and the distractors are constantly existing in the primary task. Moreover, it is different from the visual search paradigm in traditional cognitive experiments, which denotes the task of finding a target amongst a set of distractors. This is typically done by moving the eye gaze to potential target locations through an active scan of static graphics or images presented by fixed-position screens [20]. By contrast, the visual task in our paradigm is set in a more natural and interesting VR navigation scene, and it is interactive.

According to the paradigm, various experimental VR systems can be designed. Since this paper is a basic study, the task designed in this paper is relatively simple. Both the task design and indicators of performance can be further developed and expanded in more extensive research. We expect that this paradigm can be increasingly used in VR learning and training systems to further examine the relation between distraction and flow-performance link.

8.2. Role of Distraction in Weak Flow-Performance Link

According to WAM, the main reason leading to weak flow-performance link in VE is the incongruence between interactive artifacts and the primary task. Weak flow might be caused by irrelevant VE contents or using interaction artifacts (they can induce flow experience that is independent of the primary task) instead of performing the primary task [2].

In fact, distraction is a key mediator, mediating the effect from the disjunction features to flow-performance link. The problem of weak flow-performance link arises when characteristics of disjunction cause a certain level of distraction. Although the role of distractions was briefly mentioned in WAM, previous studies did not directly examine it. The results of study 1 verified this point: By designing task-irrelevant distractors to accompany the primary task, we constructed a VR activity with disjointed features that cause a wide range and degree of distraction. Moreover, distraction levels directly predicted the weak link problem. Higher levels of distraction were followed by weaker flow-performance link, while lower levels of distraction were followed by stronger link. Therefore, these results revealed the direct effect of distractions on flow-performance link. Therefore, further paying attention to distraction and clarifying the effect of distractors on the user could be a breakthrough to avoid the problem of flow-performance link and optimize VR designs.

8.3. Design Guideline and Its Practical Implication

Based on the principle proposed in WAM [2], we proposed two more specific design guidelines to deal with distractions in VR: (1) Reducing conspicuous but irrelevant distractors directly; and (2) increasing the congruence between distractors and primary task.

When conspicuous but irrelevant distractors induce distractions during the primary task, directly reducing the number of them is an easy and practical approach. This approach can effectively reduce distraction, improve task-relevant behaviors, and the performance of primary task. Not only that, but it can also strengthen the flow-performance link to a certain extent. However, this approach impairs the quality of flow experience to some extent. Some distractors can improve the vividness and interactivity of the virtual environment. Researchers have agreed that interactivity and vividness are two key variables affecting presence [33,34,35,36], and more vivid and interactive virtual environments are associated with higher levels of telepresence [37,38,39,40]. Therefore, distractors are not totally useless and they may have the potential to enhance the user’s feeling of presence and motivation of automatic exploration (autotelic experience) in VR, which helps achieve better flow experience. It may be better to take advantage of these distractors than to remove them entirely. Guideline 2 was proposed in this direction.

Guideline 2 gives another approach to dealing with distractions during the primary task. It is a constructive way to transfer the consequences of distraction into task-relevant ones through providing appropriate visual cues. In addition to the previous several studies that gave us inspiration, there are also some existing studies that are in line with this direction of design. Beck and Hollingworth [31] used a gaze-contingent search paradigm to manipulate selection history directly and examine the competition between multiple cue-matching saccade target objects. Quiros’ et al. conducted A meta-analysis to explore a concurrent working memory load task that does not impair visual selective attention [41]. These works can also provide inspiration to adjust visual information. Finally, we optimized the disjointed VR system to a more congruent one and proved the second guideline is effective and practical too. This approach can help transfer the participants’ attention back to the primary task without reducing the number of distractors, and it does help participants improve task-related behavior and performance without damaging the quality of flow. In this way, much flow may be produced from performing the primary task and task-relevant behavior, and a stronger flow-performance link can be built.

These findings can provide reference for designers to find ways to optimize VR systems. As to how to make the best use of these guidelines, the specific features of each VR system and the type of the primary task need to be fully considered.

8.4. Discussion about the Applicability of the Findings

In fact, the findings discussed in this paper are more applicable to the VR systems for the purpose of learning, training, and education rather than pure entertainment. For the latter systems, the key points of the system design are vividness, interest, and richness, so they merely need to provide users with a good experience, not necessarily to transfer these experiences into better task performance. In this case, any spontaneous exploratory behavior is acceptable, even if it is irrelevant to the primary task. Therefore, the weak flow-performance link is not a real problem for these systems. For example, the VR system in this paper is more like an example of entertainment-oriented systems. A variety of distractors (marine creatures) can enrich the scene, bring interest and fun to the users, and then produce a good experience. For entertainment, there is no need to transfer the good experiences into task relevant performance, the enhancement of attention on distractors or task-irrelevant interactive behaviors are also acceptable. Although this may not be a real problem for entertainment-oriented systems, it could be a serious and usually imperceptible problem for learning- oriented systems (or education-oriented systems) [2]. As stated in the introduction, if the enhancement of experience quality cannot promote performance effectively, pursuing good experience would be meaningless or may even be counterproductive. Still, if take the system in this paper as an example, if the primary task is not simply finding things, but to find and learn some knowledge hidden in the chests (such as biological knowledge of marine creatures), then even if distractors make the VR environment lively and interesting, they distract users’ attention resource away from the primary task. Thus, it will break the further transferring from good experience into task-relevant performance and outcome.

8.5. Limitations and Future Work

There are still some shortcomings in this paper. First, the VR systems designed in this paper are more game-like. Although we believe that the fundamental conclusions are applicable to other learning systems, they need to be further tested in various virtual learning environments. Moreover, we will further perfect the VR task paradigm and enrich the statistical test methods of weak flow-performance link in the future. Second, this paper provides two design guidelines, but there must be others. Since distraction is a key predictor, all factors affecting distraction may affect flow-performance link. For instance, task demand is a key factor affecting the selective processing of target-related information [42,43]; increasing the demands on lessons would reduce distractions (i.e., increase selectivity) in the classroom [4]; the perceptual load of a task influenced the spatial selectivity of attention [44]. Lavie and Tsal have developed a framework that explains the early and late selection conditions based on difficulty of tasks [43]. Design guidelines can be further explored and extended from these aspects. Third, tracking users’ eye movements is an effective method for revealing distractions and visual information processing [45]. However, due to the limitation of equipment, this paper did not evaluate distraction directly by measuring eye movements. Hence, it is necessary to adopt effective ways to measure distracted eye movements in VR activities to further verify these findings in the future. Last, people with some different characteristics (such as cognition style, executive control resources, etc.) handle distractions differently [7,46,47], Thus, understanding their role in the weak flow-performance link could also be a research direction.

9. Conclusions

Through a series of empirical studies, we draw the following conclusions:

(1) the VR task paradigm and prototype experimental system introduced in this paper can be used to investigate flow experience and the problem of weak flow-performance link in VR activities.

(2) Based on the views of WAM, this paper directly proves that in VEs where the artifacts are disjointed with the primary task, distraction caused by disjointed features is the antecedent of weak flow-performance link.

(3) This paper proposes two design guidelines of guiding selective attention to deal with distraction and prove their effectiveness in alleviating the weak flow-performance link in VR activities. One is directly reducing conspicuous but task-irrelevant distractors, and the other is increasing the congruence between distractors and primary tasks, i.e., adding visual cues to transfer distracted behavioral consequences to task-relevant behavior, and then contribute to task-oriented selective attention.

In future work, the specific features of VR systems and the types of primary task need to be fully explored to make the best use of these guidelines.