A study was set up to investigate the impact of the proposed approach that consisted of three conditions that differed in the way players were guided (i.e., two feedback variants guided players by either their gaze or through a crosshair, and one variant without gaze support). Only small aspects differentiate one condition from each other to grant the comparability of the conditions (see Figure 3
). In all conditions, movement and picking up objects are carried out via mouse (look around, pointing) and keyboard (WASD-movement) actions. In the following, conditions are described in detail.
In the first condition, called Gaze guidance (GazeG)
), players were guided to the coins via gaze. To grant them a challenging experience and to address the aspect of exploration, players received visual feedback driven by gaze through a vignette effect (i.e., a black and blurred circle mask darkened the corners and edges of the screen). When players looked at a sensitive gaze area (where one of the coins was located), the vignette effect appeared (see Figure 4
The design rationale behind the feedback type (i.e., vignette) and the feedback intensity was as follows: Based on the categorization by Fagerholt & Lorentzon [47
], a meta-interface solution was chosen, where representations can exist on a meta-layer between the player and the game world. The most obvious example is the effects rendered on the screen, such as blood spatter on the camera to indicate damage. The research team explored various visual user interface designs (i.e., visual cues in different shapes with different sizes). In our case, a vignette effect was deemed to be suitable, because it draws not only from SGD design approaches (i.e., subtly showing information on the peripheral field of vision) but also from OGD by directly guiding the player’s gaze in the foveal region (i.e., gaze intensity is driven by the distance between the player’s gaze and the coin).
By using a meta-interface design in the form of a vignette effect, the gaze-based approach can be integrated into different genres (in contrast to diegetic interfaces, which are closely tied to the narrative and the game world). Additionally, the approach is not dependent on the visual channel (spatial game interface such as outlined objects) and can be integrated subtly (in contrast to “traditional” game interfaces with an overt virtual layer). Furthermore, we aimed at creating feedback that is as simple as possible. Animations of the visual gaze condition are relatively subtle (continuous animation that increases or decreases the vignette’s size and opacity to indicate proximity) and simple (i.e., no pulsating movement) to avoid distracting players.
The vignette effect (size and opacity) was realized through Unity post-processing effect “vignette” [48
] and is driven through two dependent parameters:
the distance between the player’s avatar and the gaze-sensitive area (i.e., the effect only kicks in, when the player is in close distance to the object—in our case: 2 m in Unity)
the distance between the gaze position and the coin in screen space within a gaze-sensitive area (i.e., the closer the gaze position is in relation to a coin, the stronger the effect—in the game prototype, a gradual intensity transition was implemented: 1/2 screen width distance between gaze and coin: 0% effect strength; 0 distance: 100% effect strength)
The second condition, coined Crosshair Guidance (CrossG)
is similar to condition 1 with one exception: instead of using the player’s gaze to guide the player, the crosshair, located and locked in the center of the screen, manipulates the intensity of vignetting effect (i.e., GazeG
builds on CrossG
by using the gaze position as a crosshair—see Figure 5
). Via this condition, we wanted to investigate if the inclusion of gaze in the guidance process leads to a different game experience. If not, one could assume that a well-established input form, such as a mouse, would be sufficient to achieve a similar or even better result. By doing so, we want to contribute to the research body of comparative studies in the field of gaze vs. mouse (e.g., [49
Condition 3: The last condition, No Guidance (NoG) served as a control condition: It was aimed to find out if, in general, the inclusion of a guidance system is perceived useful and arouses an engaging game experience. Here, no cues are given that reveal the position of the gold coins.
With our comparative study, we investigated three hypotheses that relate to the previously described conditions (GazeG, CrossG, and NoG). In general, we deem that the gaze-based player guidance approach contributes to an immersive experience by offering guidance and a certain degree of challenge. Furthermore, the gaze-based variant could provide a more engaging experience in comparison to other solutions, by offering a natural form to get in contact with the game (i.e., eyes are employed to perceive visual information). Thus, we assume that the approach arouses a better experience than a solution that does not include gaze-based feedback (in our case: when the crosshair is used for guidance). We also wanted to investigate if the inclusion of a guidance tool is reasonable and grants an engaging experience in the context of an exploration game in general, or if it bores players by offering no challenge to them in achieving the game’s goals. It could be the case that players would enjoy the challenge by getting no clues at all, with the consequence that a guidance approach is not applicable and even counter-productive. The following hypotheses are formulated:
Hypothesis 1 (Game Experience).
In an exploration game, due to the integration of gaze as a natural interaction form, players will have a better game experience when being guided through a gaze-based guidance approach in comparison to a crosshair guidance-based or a no guidance approach.
Hypothesis 2 (Game Performance).
In an exploration game, players, supported via guidance features, will perform better concerning the game goals (here: find specific objects) in comparison to a no guidance approach.
Hypothesis 3 (Game Challenge).
In an exploration game, players with no guidance support, will perceive the game (goals) as more challenging (i.e., perceived game difficulty) in comparison to gaze- and crosshair guidance-based approaches.
4.4. Participants and Procedure
This section comprises information regarding the experiment, which was conducted at the Playful Interactive Environments (PIE)
research laboratory, University of the University of Applied Sciences Upper Austria. Recruitment of subjects was carried out by utilizing mailing lists provided by the University of Applied Sciences Upper Austria. The experimenters invited subjects by providing information on the type of experiment (i.e., experimental study in the field of games), the study location, and the duration of the experiment, and the targeted age range (we were interested in subjects between 18 and 35 as they are a relevant group when it comes to games [60
]). No incentives were offered. In total, 24 people between the age of 21–34 participated (M = 23, SD = 3), ten were female, 14 were males. To get information on the subject’s playing habits the following questions were asked: “How often do you play (computer-)games?”
(62% several times a week, 21% several times a month, 17% daily) and “How often do you play exploration games? (Stanley Parable, Gone Home, etc.)”
(17% several times a week, 83% several times a month). None of them had any previous experience with eye-tracking devices and gaze interaction. Furthermore, the test subjects signed a consent form. During the experiment, one member of the research staff was present who was responsible for guiding the participants through the experimental procedure, offering support if necessary, and carrying out interviews (responsibilities: introduction, interview).
The experimental procedure had the following structure: As a first step, the experimenter welcomed the subjects and provided a brief introduction to the scope of the study. After this, the game goals, the mechanics, and the means of interaction were introduced (i.e., ways to interact with the game world, control scheme, the functionality of the eye tracker, playing a short demo [61
]). When this step was finished, the eye tracking device was calibrated. The subjects filled in a questionnaire that contained demographic information (i.e., age, gender, education, and experience with games). After that, they play the first out of three conditions resulting in three play sessions per participant. The locations of the coins were randomized (8 coins in each condition with 128 possible locations) to avoid biases. Furthermore, conditions were presented in a randomized order in each playtest (e.g., subject 1 played condition 3, condition 2, condition 1; subject 2 played condition 2, etc.).
All subjects were asked to play all three conditions (within-subject design). In all conditions, the game was stopped after 3 min to make sure that there was enough time for playing, filling out the questionnaires, and carrying out the interviews.
By having relatively short play sessions, it was avoided that subjects might get tired or even bored, which could have an impact on the information obtained in the interviews. Furthermore, it was one of our goals that each subject was exposed to the game within a predefined time limit to make conditions more comparable. It was also crucial that the exposure time was not being influenced by the player’s performance (i.e., gold). We also did not inform the players how many coins they already collected and how many coins are still left. When the goal of the condition was reached (time limit exceeded/all coins were found), players were asked to fill in a questionnaire that asked them about their impression of the game prototype focusing on the perceived game experience.
The game design also reflects the goal of having a brief exposition to the game. The game is made up of simple mechanics (move, look around, and pick up gold), is easy to understand concerning its controls (mouse plus WASD-keys on the keyboard), has a clear goal (collect all the gold coins), and the game design is consistent in all three conditions.
Additionally, participants were interviewed (for more information on the employed measures see Section 4.5
). When players were finished with filling in the questionnaire and giving answers, the experimenter presented the second out of three conditions. As in the first part, players were asked to fill in the same questionnaire after completing each condition. When all conditions were completed, an informal interview was carried out by the experimenter. Players were asked to compare each condition with each other. The procedure took about 60 to 80 min per participant.
To measure the perceived immersive game experience, the Immersive Experience Questionnaire (IEQ)
by Jennett et al. [62
] was employed. It was used in various studies (e.g., [63
]), and measures the experience via five factors and a single question to indicate the perceived immersive experience. For the comparative study the following factors were employed: cognitive involvement (effort and attention—Coin
), emotional involvement (affect and suspense—EmIn
), control (use of the interface—Cont
), challenge (game difficulty—Chal
), and total immersion (Toim
In order to get a better impression of the factors the following example items shall be given
Emotional involvement (EmIn): “To what extent did you feel that the game was something fun you were experiencing, rather than something you were just doing?” (rated on a seven-point Likert scale ranging from “not at all” to “very much so”).
Challenge (Chal): “To what extent did you find the game challenging?” (rated on a seven-point Likert scale ranging from “not at all” to “very difficult”).
Apart from the IEQ, the number of collected coins (player performance regarding the game goal) was measured. Furthermore, after each condition, players were interviewed via a semi-structured interview dealing with the topics overall game experience, perceived difficulty, perceived the visual quality of the vignette effect (only GazeG and CrossG condition), usefulness and clarity of the player guidance feedback system (only GazeG and CrossG condition). At the end of the evaluation, an interview was carried out focusing on the comparison of the conditions (overall game experience, perceived challenge, the usefulness of the GazeG condition concerning the other conditions).
4.6. Data Analysis
In order to examine the underlying hypotheses, all analyses were conducted using repeated-measures analysis of variance (rANOVA). A benefit of the repeated-measures rANOVA is the limited number of subjects required. All parametric tests were performed after validating the data for assumptions of rANOVA use. Following the argumentation of Iacovides et al. [65
], normality is established as a theoretical assumption that derives from the employment of a questionnaire to measure a unidimensional latent concept. Condition of sphericity was satisfied by carrying out Mauchly’s sphericity tests (emotional involvement: Mauchly-W(2) = 0.77,
, control: Mauchly-W(2) = 0.71,
, challenge: Mauchly-W(2) = 0.98,
, total immersion: Mauchly-W(2) = 0.71,
, collected coins: Mauchly-W(2) = 0.97,
). Pairwise comparisons used the Bonferroni method of adjusting the degrees of freedom for multiple comparisons (post-hoc-tests). All statistic tests were carried out with SPSS 24. Significance was set at