Next Article in Journal
Impact of the Sub-Grid Scale Turbulence Model in Aeroacoustic Simulation of Human Voice
Next Article in Special Issue
The Application of Virtual Reality in Engineering Education
Previous Article in Journal
Plume Divergence and Discharge Oscillations of an Accessible Low-Power Hall Effect Thruster
Previous Article in Special Issue
Spatial Skills and Perceptions of Space: Representing 2D Drawings as 3D Drawings inside Immersive Virtual Reality
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Immersive Serious Game for the Behavioral Assessment of Psychological Needs

by
Irene Alice Chicchi Giglioli
*,
Lucia A. Carrasco-Ribelles
,
Elena Parra
,
Javier Marín-Morales
and
Mariano Alcañiz Raya
Institute for Research and Innovation in Bioengineering (i3B), Polytechnic University of Valencia, 46022 Valencia, Spain
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(4), 1971; https://doi.org/10.3390/app11041971
Submission received: 20 January 2021 / Revised: 17 February 2021 / Accepted: 18 February 2021 / Published: 23 February 2021
(This article belongs to the Special Issue Applications of Virtual, Augmented, and Mixed Reality)

Abstract

:

Featured Application

The use of virtual serious game and machine learning techniques in the assessment of psychological needs can enhance the traditional assessment adding behavioral-based information.

Abstract

Motivation is an essential component in mental health and well-being. In this area, researchers have identified four psychological needs that drive human behavior: attachment, self-esteem, orientation and control, and maximization of pleasure and minimization of distress. Various self-reported scales and interviews tools have been developed to assess these dimensions. Despite the validity of these, they are showing limitations in terms of abstractation and decontextualization and biases, such as social desirability bias, that can affect responses veracity. Conversely, virtual serious games (VSGs), that are games with specific purposes, can potentially provide more ecologically valid and objective assessments than traditional approaches. Starting from these premises, the aim of this study was to investigate the feasibility of a VSG to assess the four personality needs. Sixty subjects participated in five VSG sessions. Results showed that the VSG was able to recognize attachment, self-esteem, and orientation and control needs with a high accuracy, and to a lesser extent maximization of pleasure and minimization of distress need. In conclusion, this study showed the feasibility to use a VSG to enhance the assessment of psychological behavioral-based need, overcoming biases presented by traditional assessment.

1. Introduction

Motivational psychology is a broad branch of psychology that studies human needs and their fulfillment for mental health and well-being [1,2,3]. Motivation can be defined as an unobservable and internal psychological process that drives observable human behaviors in order to satisfy needs [1,2]. The study of motivation involves many theoretical approaches, including dispositional, learning, humanistic, biological, cultural, and psychodynamic perspectives [1,2,3,4]. According to these, 20 years ago, the psychotherapist Klaus Grawe proposed a new theoretical model, named the consistency model, which conceives mental functioning as the interaction between motivational schemas and the satisfaction of four psychological needs: attachment, self-esteem, orientation and control, and maximization of pleasure and minimization of stress (Figure 1) [3]. Environmental and interpersonal factors perceived to satisfy these needs moderate the maintainment and enhancement of the self, whereas those that impede need satisfaction foster distress and mental illness [3].

1.1. The Four Basic Psychological Needs

Attachment represents the reflection in adulthood of the relational bonding developed within the child-caregiver dyad over a child’s first years of life [5,6,7,8]. In accordance with attachment theory [7,8], the quality of the relationship between a child and the main caregiver generates the development of internal working self-representations and others that can be recognized in three interrelated dimensions of attachment styles [9,10,11]: beliefs and attitudes, goals and needs, and plans and strategies [12]. Three attachment styles have been conceptualized [5]: secure, ambivalent-insecure, and avoidant-insecure. Secure attachment is the most common (60% of the worldwide population) [11] and is defined as a strong relationship between child and caregiver, who is perceived by the infant as a secure base that allows the exploration of real environments and gives comfort in stressful situations. Children with ambivalent-insecure attachment, representing 15% of the worldwide population [10], tend to keep a distance from caregivers and struggle to explore the environment. Lastly, avoidant-insecure attachment leads children to independently explore the environment and to avoid closeness with the caregiver; 25% of the worldwide population is characterized by this attachment style [11]. Regarding the three dimensions, the beliefs and attitudes of secure people refer to high self-esteem, usually liking others, and they think that others are trustworthy and have good intentions. With respect to goals and needs, they desire intimate relationships and search for balance between closeness and autonomy in relationships, and regarding plans and strategies, they recognize stress and constructively modulate its negative effects [12,13,14]. On the contrary, ambivalent-insecure individuals believe that people are complicated to understand and have little control over their lives, and they need extreme intimacy, showing lower levels of autonomy and fear of rejection. Regarding plans and strategies, they demonstrate intensified stress and anger to elicit responses from others, and at the same time they collaborate with others to obtain acceptance. Similarly, avoidant-insecure individuals show beliefs and attitudes of doubt regarding others’ motives, believing that others are not trustworthy and doubting their honesty and integrity, and show a lack of confidence in social situations. Regarding goals and needs, they need to keep distance, limit intimate relationships, and maintain autonomy and independence, and they control stress, not showing anger and emotions in general.
Self-esteem is defined as perception of oneself and a personal evaluation one makes about oneself [15,16]. It is characterized by two dimensions, competence and worth; the former regards how people perceive themselves as capable and efficacious, whereas the latter is related to the value that everybody feels to have and reflect. Social interactions can affect the self-worth dimension of the construct, and individuals try constantly to increase their self-esteem [3].
Orientation and control are psychological concepts related to self-efficacy theory [17] and refer to the self-belief of having an influence on the environment, successfully accomplishing specific goals, and producing favorable outcomes [3]. Self-efficacy is a personal judgment about the subjective perception of being able to execute a task or cope with a situation positively. According to Grawe [3] and Bandura [17], goal-setting is relevant to ensure that an individual’s efficacy beliefs are in line with aims and competencies and not working against them. A positive interaction between efficacy beliefs, aims, and competencies results in the successful management of many situations and tasks. On the contrary, people with low self-efficacy believe they are not able to cope with different situations and tend to fail. Furthermore, people with high self-efficacy perceive themselves as being in control of their own lives and decisions, and people with low self-efficacy tend to think that they cannot manage situations and tasks by themselves. Finally, efficacy beliefs depend on four sources of information and processes: (a) mastery experiences: the direct experience of mastery increases self-efficacy, and failures can undermine self-efficacy beliefs; (b) vicarious experiences: self-efficacy comes from the observation of other people; (c) verbal persuasion: positive or negative verbal responses can strengthen or weaken subjective self-efficacy beliefs toward succeed or failure; and (d) emotional and physiological states: positive or negative emotional and physiological states can influence the self-belief of efficacy to succeed or fail.
Finally, the need to maximize pleasure and minimize stress/pain is a basic pleasure principle that has been theorized and studied by many authors, first and foremost by Freud [18]. The idea is that individuals seek pleasant experiences and states rather than unpleasant or painful circumstances [19,20,21]. Sought-after pleasure states can be physical, psychological, emotional, or social [22]. Avoiding or approaching pleasant or unpleasant situations relies on subjective motivation and various psychological processes, such as decision making, learning, and self-regulation [23,24]. In accordance with this assumption, two subjective motivational systems emerge: the behavioral activation system (BAS), which addresses self-enhancement, desire for pleasure, and rewards, and the behavioral inhibition system (BIS), which is oriented toward self-protection, caution against pain, and avoidance of threats [24,25,26,27,28].

1.2. Traditional Assessment and Serious Games Approaches

Traditional psychological assessment approaches refer to qualitative measures, such as the Adult Attachment Interview, and/or quantitative measures, such as the Adult Attachment Questionnaire or the Rosenberg Self-Esteem Scale in which respondents provide explicit and conscious information about themselves [29,30,31]. Although these measures are well validated and present a high internal consistency, they are showing some limitations in that they are too abstract and decontextualized from real situations and they are not able to activate specific phenomena that need to be activated in order to be manifested, such as attachment situations [32]. Furthermore, methodologically, the assessment is conducted in neutral laboratory or clinical settings to maintain control on confounding variables that can influence the veracity of the final outcomes. Finally, they are also presenting biases, such as the social desirability bias that is the tendency of self-reported respondents to answer questions in accordance with socially favorable norms to maintain a socially favorable self-presentation, affecting responses veracity [33,34].
Recently, advances in technological systems, such as videogames and computer games, as well as the amount of time children, adolescents, and adults play computer games has significantly gone up, providing the opportunity to use them for enhancing traditional psychological assessment. Indeed, their playability and appeal led researchers to become interested in the development of applications with “serious” and specific goals for the evaluation and training of abilities, going beyond common entertainment [35,36]. Serious games (SGs) are proving to be able to generate motivational engagement, due to main game features such as challenges, rewards, narrative, virtual agents, and fantasy worlds, as well as immersion and interaction, enhancing user experience. Indeed, the first computer games were supported by non-immersive systems (2D), as desktop, and the interaction was afford by keypad and mouse; more recently, SG can be experienced also through immersive systems, as head mounted display (HMD) able to completely isolate the user from reality, allowing them to explore, navigate, and interact with objects (through control sticks and gloves) in real-time as if the user was in the physical reality. Furthermore, SGs are proving to be effective to assess and train abilities adding behavioral-based information to traditional methods [37,38,39].
To create valid assessment in games, “stealth” game design is providing significant evidence mainly related to learning [40,41]. The stealth assessment (SA) game design approach aims to implicitly measure user performance during the game without asking users to self-report information about their personality, behavior, or state. SA refers to evidence-centered design (ECD) as a conceptual framework based on three models to develop valid assessments: competency, structural, and task models [42,43]. The competency model refers to the abilities or constructs and traits, also called unobservable indicators, that researchers want to measure using the game; the structural model identifies the multifaceted behaviors, also called observable indicators, that can reveal the theoretical abilities or constructs and traits; and the task model designs tasks or situations that can produce those behaviors related to the abilities, constructs, or traits that researchers want to measure. The SA approach has been shown to have greater predictive validity and less bias than traditional assessment [37,43]. Indeed, SA allows the creation of several SG contexts in which participants perform a sequence of actions, reducing test anxiety and bias with respect to traditional assessment and invisibly providing a large amount of data on various ranges of individual attributes and skills [37,43,44]. Until now, serious games (SGs) supported by SA approach, on one hand, have been mainly tested in educational field in order to enhance learning abilities [39,40]. On the other hand, SGs have been also investigated in the assessment of abilities in educational and learning [45], health [44], and military fields [46]. In motivational psychology, SGs have been less addressed and have been proposed as valid assessment methods to measure also those psychological needs’ behavior-based dimensions along with traditional self-reported questionnaires [47].
Based on these premises, the aim of this study was to investigate the content feasibility of a virtual serious games (VSG) to assess the four basic psychological needs of attachment, self-esteem, self-efficacy, and stress avoidance according to the consistency model. The main hypothesis concerned the assumption that VSG contents were able to elicit the psychological needs phenomena and assess them using user performance.

2. Materials and Methods

2.1. Participants

A total of 61 subjects (30 women and 31 men; mean age 35.95 years; SD = 11.17) participated in this study. Table 1 shows the distribution of the sample according to demographics. The inclusion criteria were age between 18 and 55 years and a cut score higher or equal to than 24 on the Mini-Mental State Examination (MMSE) [48]. Before participating, each subject received written information about the study and was required to give written informed consent for inclusion. The study obtained ethical approval by the Ethical Committee of the Polytechnic University of Valencia (Protocol code: P17_23_01_2019).

2.2. Psychological Assessment

The following questionnaires were administered to participants before the virtual sessions:
Demographics: Participants completed a general sociodemographic questionnaire including age, gender, education, family and marital status, labor status, and economic entry.
The main questionnaires were related to Grawe’s model of consistency:
Attachment: The Adult Attachment Questionnaire [30] is a 40-item self-reported scale that assesses four factors: (1) low self-esteem, need for approval, fear of rejection (13 items); (2) hostile conflict resolution, rancor, and possessiveness (11 items); (3) secure affect or expression of feelings and comfort with relationship (9 items); and (4) emotional self-sufficiency and discomfort with intimacy (7 items). Participants responded to the statements on a 6-point Likert scale from 1 (strongly disagree) to 6 (strongly agree). Internal consistency of the four factors was α = 0.86, 0.80, 0.77, and 0.68.
The relationship questionnaire (RQ) [11] consists of four short paragraphs, designed to measure the four attachment styles (secure, insecure-preoccupied, insecure-fearful, and insecure-dismissing). Participants are asked to indicate which paragraph describes themselves and others.
Self-esteem: The Rosenberg Self-Esteem Scale (RSE) [31] is a 10-item measure that assesses general self-esteem. Participants rate the extent to which each item applies to their self-evaluation on a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree). Five negatively worded items are reverse scored. Total scores are summed, and higher scores indicate elevated levels of self-esteem. Internal consistency is high (α = 0.93).
Self-efficacy: The General Self-Efficacy (GSE) scale [49] is a 10-item questionnaire that assesses positive self-perception in coping with various difficult situations. Participants are asked to rate each item on a 10-point Likert scale. Total score is summed, and higher scores indicate more self-efficacy. Internal reliability for GSE is between 0.76 and 0.90.
Behavioral approach and behavioral inhibition system: The Behavioral Approach/Inhibition Scale (BIS/BAS) [25] is a 24-item questionnaire rated on a 4-point Likert scale ranging from 1 (very true for me) to 4 (very false for me). The scale presents four subscales, 1 for BIS and 3 for BAS (drive, reward, and fun seeking). Cronbach’s α for the BIS, BAS drive, BAS reward, and BAS fun seeking scales are 0.74, 0.76, 0.73, and 0.66, respectively.
Coping: Brief Cope [50,51,52] consists of 28 items that measure strategies used to cope or regulate cognition in response to stress. Participants are asked to rate each item on a 4-point Likert scale from 1 (I have not been doing this at all) to 4 (I have been doing this a lot). It includes 14 two-item subscales that can be divided into the two primary coping styles of approach and avoidant coping: (1) self-distraction (avoidant); (2) active coping (approach); (3) denial (avoidant); (4) substance use (avoidant); (5) emotional support (approach); (6) use of instrumental support (approach); (7) behavioral disengagement (avoidant); (8) venting (avoidant); (9) positive reframing (approach); (10) planning (approach); (11) humor; (12) acceptance (approach); (13) religion, and (14) self-blame (avoidant). Humor and religion cannot be classified either as approach or avoidance coping. The reliability and validity of the scale indicated high Cronbach’s alpha values for some domains, such as religion (α = 0.82) and substance use (α = 0.90). Other domains indicated acceptable values: active coping (α = 0.68), planning (α = 0.73), positive reframing (α = 0.64), acceptance (α = 0.57), humor (α = 0.73), using emotional support (α = 0.71), using instrumental support (α = 0.64), self-distraction (α = 0.71), denial (α = 0.54), venting (α = 0.50), behavioral disengagement (α = 0.65), and self-blame (α = 0.69).

2.3. Task Modeling of Serious Game: ATHENEA

The VSG system was developed using Unity 5.5.1f1 software, applying c# programing language using the Visual Studio tool. Participants performed the VSG wearing an HMD device (HTC VIVE) and the interaction was ensured through two hand controllers.
In the game, a story narrative was created, taking place in a spaceship (Athenea), whose aim was to discover and settle a new land because Earth is no longer habitable. The VSG included seven virtual agents: six of them were adults, designed with specific personality traits and personal competencies according to the attachment working models [53]; more specifically, pairs of virtual agents (one man and one woman) reflected the secure, ambivalent-insecure, and avoidant-insecure attachment styles (Figure 2). The last virtual agent was a child without a specific attachment style. Each virtual agent and the study participant had an explicit role inside the spaceship according to their personality traits and personal competencies. The specific role of the study participant was to maintain technical control of the Athenea and ensure the correct direction to possible destinations. In addition, the participant was responsible for caring for the child. Furthermore, a spaceship artificial intelligence controlled and supervised the correct functioning of the spaceship and, if something was wrong, alerted the crew to solve the problem. The spaceship was composed of four areas: a red area formed by the engine room and the hangar; a green area, which was the zone for life support, orchard, and water and air purification; the control area, formed by the weapons and system control rooms; and a blue area, including relaxation and well-being rooms, such as bedrooms, cafeteria, infirmary, and EMO room. The EMO room was designed for crew well-being, in which participants self-assessed their mood states as well as those of the rest of the crew, and their performance, and according to the crews’ mood states, they could decide to improve and enhance them (Figure 3).
The VSG consisted of 10 situations, and in each situation, several episodes and tasks were designed according to the theoretical psychological framework and the SA method. The following sections introduce the four competency models, one for each basic psychological need, and their relative indicators, including the specific tasks for each psychological need. For each of the four competency models, we presented a graphic model of the indicators (unobservables—theoretical psychological constructs—and observables—tasks and data gathered from user performance).
Attachment assessment: This included three episodes of loss, four of loneliness, four of threat, and two of suspicion. Two of these episodes (one of threat and one of loss) were related to childcare. The attachment episodes were distributed in the SG and in order for participants to go ahead, they had to make a decision based on (a) solving the problem alone; (b) seeking the help of other crew members (in this case, deciding on a crew member); (c) letting another crew member solve the problem; or (d) doing nothing. Each decision-making response was designed according to the referenced attachment theoretical framework and the main traditional questionnaires used to assess attachment style (Table 2).
Self-esteem assessment: At the end of each situation, participants entered the EMO room to self-evaluate how they felt, to see how the rest of the crew felt according to the two dimensions of valence (positive/negative/neutral) and arousal state (low activation/high activation) and according to the feelings of the rest of the crew, participants could improve their well-being (Table 3).
Self-efficacy assessment: This assessment involved three situations through 12 games each. Games were developed and modulated according to the four sources of information proposed by Bandura (1997) [17]: mastery experience, vicarious experiences, verbal persuasion, and emotional and physiological states. Each game lasted three minutes and thirty seconds and at the minute and a half one of four sources of information appeared. At the beginning of each game, the virtual system presented to participant the specific explanation of the game. At the end of each self-efficacy situation, participants were asked to self-evaluate their perception of efficacy during the game performance. The main performance data gathered during the gameplay concerned the reaction times-starting from the end of game explanation to participant beginning and for the entire duration of the game every single interaction with the virtual stimuli were measured. Furthermore, all the correct and incorrect performance responses, as well as the number of completed trails were gathered (Table 4).
Stress avoidance: For this assessment, 12 episodes on avoiding or approaching pleasant or unpleasant situations were designed. The episodes were distributed throughout the course of the SG and in order to go ahead, participants had to decide based on cognitive, emotional, or behavioral approach or avoidance responses. Each decision-making response was designed according to the main theories and questionnaires related to the BIS/BAS (Table 5).

2.4. Experimental Procedure

The study consisted of five sessions of almost 1 h each, presented to participants every other day. Before the VSG experience, participants were administered the above-mentioned questionnaires, and completed an introductory tutorial to familiarize themselves with the virtual reality system. The tutorial was conducted in a neutral room, including a video with the narrative introduction to the history, and subsequently participants were asked to move in the room using the two controllers and then back to a table where an activity was proposed. The activity consisted of picking up three geometric figures, rotating and inserting them in the right location. To verify the correct activity execution, participants had to press a button, and if the activity was correct, the button changed color from red to green and the VSG started. Subsequently, participants were introduced in the first VSG experience session, including the first three situations mainly related to attachment, self-esteem, and stress avoidance phenomena. In the second, third, and fourth sessions participants experienced the fourth, fifth, and sixth situations, respectively, and mainly related to self-efficacy need. In the last session, the seventh, eighth, and ninth, mainly related to attachment, self-esteem, and stress avoidance phenomena were performed by participants. Finally, during the last session, the tenth situation involved a conclusion episode of the narrative storytelling not related to any need was presented to participants. The narrative storytelling situations followed a sequence order, from 1 to 10 (conclusion episode) and all participants were presented with all situations and episodes following the narrative storytelling.

2.5. Data Analysis

Multivariate outlier detection was performed to find the participants whose scores on the questionnaires were extreme. Outliers within each of the four groups of variables (attachment, self-esteem, self-efficacy, stress avoidance) were identified independently of the variables of the rest of the groups. The Mahalanobis distance between subjects according to the variables in each group with more than one score (e.g., attachment, with four subscales) was calculated, along with the probability that it belonged to a Chi-square distribution. For groups with only one variable (e.g., self-esteem), the z-score was calculated. In both cases, subjects falling into to the most extreme 1% of the data distribution were defined as outliers. We found one outlier subject regarding the attachment variables and two in avoidance stress. The level of statistical significance was set as p < 0.05.
We had five datasets, one for each main variable and one with demographic data from the subjects and their respective outputs in all the scales. These demographic data (sex, age, education, marital status, employment status, and income) were added to the four main datasets. These datasets consisted of the following:
Attachment: 41 variables describing the decisions taken by the subject (which actions they did and with which virtual agent) and nine variables relating to how many times each virtual agent was chosen and which one was chosen the most. Data from 59 subjects were available after outlier removal.
Self-esteem: 42 variables relating to the decisions taken by the subject (which actions they did and with which virtual agent), 26 relating to how the subjects described themselves during the game (12 were removed as their value was constant), and 224 relating to how the subjects perceived the emotional status of the virtual agents. Data from 60 subjects were available after outlier removal.
Self-efficacy: 291 variables relating to the subject’s performance during the different games; 50 of these were removed as their value was constant. They were grouped into eight groups according to the cognitive function evaluated in each game: (1) attention (201), (2) memory (114), (3) velocity (106), (4) planning (88), (5) perception (53), (6) inhibition (44), (7) flexibility (35), and (8) control (6). Some of the variables belong to several groups since some games measured different cognitive functions. Data from 60 subjects were available after outlier removal.
Stress avoidance: 12 variables relating to the decisions taken by the subject (which actions they did and with which virtual agent). Data from 58 subjects were available after outlier removal.
Due to the high dimensionality of both self-esteem and self-efficacy datasets, we worked separately with the group of variables described.
Outputs: There are five main scales, some with subscales, as shown in Figure 4 (attachment has four subscales, together with RQ; stress avoidance has six subscales; self-esteem and self-efficacy have only one dimension). The subject’s score on each of these scales, drawn from the self-assessment questionnaire, was predicted, and simplified as high or low. This categorization of scores was done using population means as a threshold, in order to have a similar number of subjects with low and high values, and to improve discrimination between categories, with the exception of the variable RQ, which was originally qualitative, and its distribution remained the same.

2.6. Machine Learning Analysis

Machine learning was applied to find the best selection of variables from the VSG that predict whether the subjects would have high or low scores in the study variables. The best predictive model was obtained for each scale. If the scale had more than one set of variables (self-esteem and self-efficacy), we also investigated which set achieved better results. In addition, whether or not to include demographic variables to predict each scale was also studied.
The process of feature selection and modelling was common to all scales and followed the scheme described in Figure 5. Different machine learning algorithms (Table 6) were tested to predict the study variable subscales. The following steps were validated using 15-fold cross-validation (CV):
Feature selection: The wrapper method of backward sequential feature selection was chosen. Starting from a model with all the features, in each step the feature that decreased the performance measure the most was removed. The maximum possible number of features that could be selected was 15 to avoid overfitting.
Hyperparameter tuning: Once a set of the best features was obtained, hyperparameter tuning was performed. Different hyperparameters, as shown in Table 6, were optimized in each machine learning algorithm. Ten equal-sized values in the range defined for each hyperparameter were evaluated. In the case of the support vector machine (SVM) hyperparameters, an exponential transformation (2x) was applied to the values. The search for the best hyperparameters for the machine learning models was limited and not continuous, so the chosen hyperparameters might not be optimal. This also might help to avoid overfitting.
Modeling: Once the best features and hyperparameters for each machine learning algorithm were obtained, the model was trained and validated with 15-fold CV to obtain the average of metrics (accuracy, Cohen’s kappa, sensibility (True Positive Ratio; TPR), and specificity (True Negative Ratio; TNR)). The average of the metrics achieved in each fold is reported.
All analyses (outlier detection, Chi-square test, and machine learning) were performed on R software (version 3.6.1).

3. Results

3.1. Descriptive Results

Table 7 shows the descriptive analysis of quantitative variables and the related high/low categorization using the mean of each subscale, except for RQ, which is a qualitative variable.

3.2. Machine Learning

Recognition of Attachment, Self-Esteem, Self-Efficacy, and Stress Avoidance Using the VSG.
Table 8 shows the results of machine learning models for each self-assessed psychological construct. All four variables (attachment, self-esteem, self-efficacy, and stress avoidance) have at least one subscale that was modelled with accuracy ≥0.8 and kappa ≥0.5 using data from the VSG.

4. Discussion

Motivational psychology is a broad branch of psychology that studies human needs and their fulfillment for mental health and well-being [1,2,3,4]. Motivation can be defined as an unobservable and internal psychological process that drives observable human behaviors in order to satisfy needs. Traditional assessments of basic psychological needs involve qualitative and quantitative measures, such as interviews and self-reported scales that are presenting limitations and biases, affecting the responses veracity [32,33,34]. To overcome these limitations, VSGs are probing to be effective in assessment and training in different research fields alongside traditional approaches [53,54]. According to this, the purpose of the present study was to explore the feasibility of a VSG to assess the four basic psychological needs of attachment, self-esteem, self-efficacy, and stress avoidance according to the consistency model [3]. In order to provide VSG contents’ validity in the recognition of each basic psychological need, the data analysis included a broad set of supervised machine learning (SML) models combining the VSG performance for each variable and the scores in traditional questionnaires.
SML algorithms achieved high mean attachment recognition between the VSG and the four attachment questionnaire subscales (accuracy >85%, using a maximum of 15 selected features). More specifically, the main recognition was between the individual’s VSG performance in the specific episodes of loss, loneliness, threat, and suspicion and the first two subscales, related to the ambivalent-insecure attachment style (1: low self-esteem, need for approval, and fear of rejection; 2: hostile conflict resolution, rancor, and possessiveness), achieving 90% accuracy (kappa: 0.76 and 0.74, respectively), including 88% of true positives for the first subscale and 68% for the second. Regarding the third subscale, related to the secure attachment style (3: secure affect or expression of feelings and comfort with relationship), the model recognition achieved 85% accuracy (kappa: 0.63) with 88% of true positives. The fourth subscale, related to the avoidant-insecure attachment style (4: emotional self-sufficiency and discomfort with intimacy) achieved 88% accuracy (kappa: 0.67) including 93% of true positives. These results seem to suggest, on one hand, that the VSG situations are able to activate attachment phenomena, and on the other, the ability to recognize attachment style using VSG.
Regarding self-esteem, a Generalized Linear Models (GLMNet) using 12 features related to how the subject reported the status of virtual agents achieved 98% accuracy (kappa: 0.96) including 97% of true positives. The set of variables related to participants’ decisions and the combination of all variables reached 93% accuracy (kappa: 0.68) including 81% of true positives. On the other hand, the subjects’ self-report status was clearly not useful in predicting self-esteem (kappa: 0). These results suggest, on the one hand, consistency regarding the sample frequency distribution, in which we found a larger imbalance between subjects with low (n = 37) and high (n = 23) self-esteem, and on the other hand, a positive model of others and a negative model of the self, typical of individuals with an ambivalent-insecure attachment style [12].
Self-efficacy beliefs depended mainly on attentional, planning, and overall ability to manage a task or situation according to the four sources of information, reaching accuracy of 95% and 92% (kappa = 0.89 and 0.88, respectively). This result, consistent with previous studies, suggests that if mastery and vicarious experiences, as well as verbal persuasion and emotions and physiology, are positive at the beginning and the feedback during the experience is positive, this will enhance attentional abilities and planning to achieve a goal, with the perception of high self-efficacy [17].
On the contrary, stress avoidance and coping subscales clearly had worse predictions than those of attachment, which could be due to the lack of variety in either the features (only 12 available) and/or the SG situations related to the underlying psychological need.
Despite the promising results, some limitations of the present study should be considered. The main limitation is related to the limited number of subjects for model validation. None of the models were tested with new or external data. Future studies should include a more extended sample in order to test the models and improve the validity of the study. The SML models were validated using as ground truth the psychological questionnaires presented in the introduction. Even though they are commonly used questionnaires, an assessment performed by a psychologist could have been preferred. Another limitation of this study is represented by the categorization of predicting classes of psychological needs (low and high) rather than continuous data scores. The categorization was generated around the fixed central tendency estimate (mean) and previous research on personality has shown that the distribution of trait scores tends to be Gaussian, so that most individuals will fall into the central tendency estimate of the scale. The binary classification imposes a constriction on the mean, distancing trait scores from reality. Future studies should take into consideration the use of psychological construct data for continuous scores.
Regarding the SG contents, our results support our initial hypothesis on attachment, self-esteem, self-efficacy, and not on stress avoidance. Indeed, the virtual content definition of stress avoidance could not predict the related psychological construct, which highlights, on the one hand, the importance of validating the virtual construct-related content and SML as an effective technique to check consistency. Future works, according to this result, should modify the actual virtual situations related to this construct, test it, and validate as a final step.
Despite these limitations, the results of this study show that by using SML techniques on psychological dimensions with a VSG, it may be possible to enhance, first, the assessment of behavioral-based psychological needs, overcoming biases presented by traditional assessment, and second, to validate the appropriateness of virtual contents based on psychological needs.

5. Conclusions

Motivation, as mentioned in the introduction, is an unobservable and internal psychological process that drives observable human behaviors in order to satisfy needs for maintaining and improving mental heath and well-being [1,2]. According to this, virtual serious games and a stealth assessment method can be powerful tools for improving the assessment of psychological needs, overcoming limitations and biases presented by traditional measures, providing simulated situations similar to real ones, and gathering data from observable human behaviors during the gameplay.
Furthermore, SML techniques can handle vast datasets, with effectiveness in the recognition of psychological constructs by advanced technologies, allowing the control of individual variability dimensions and factors. Another advantage of applying SML techniques involves validation of the related virtual content, in this case specific personality constructs. According to these factors, two conclusions can be inferred: (1) the VSG content was able to recognize attachment styles, self-esteem, and self-efficacy beliefs and less stress avoidance need; and (2) all of the results were consistent with studies on previous traditional measures, suggesting that VSG can also be an effective measurement tool alongside traditional ones to assess personality constructs.

Author Contributions

Conceptualization, I.A.C.G., E.P. and M.A.R.; methodology, I.A.C.G., E.P. and M.A.R.; validation, I.A.C.G. and E.P.; formal analysis, L.A.C.-R. and J.M.-M.; investigation, I.A.C.G. and E.P.; resources, I.A.C.G. and M.A.R.; data curation, I.A.C.G. and L.A.C.-R. Writing—original draft preparation, I.A.C.G. and L.A.C.-R.; writing—review and editing, I.A.C.G., L.A.C.-R. and J.M.-M.; supervision, M.A.R. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Spanish Ministry of Economy, Industry and Competitiveness funded project “Advanced Therapeutically Tools for Mental Health” (DPI2016-77396-R) and by the European Union through the Operational Program of the European Regional Development Fund (ERDF) on the Valencian Community 2010-2020 (IDIFEDER/2018/029).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Polytechnic University of Valencia (Protocol code: P17_23_01_2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to restrictions on privacy policy on sensitive data categories.

Acknowledgments

We thank Daniel Beneito Brotons and Salvador Cerda Egea for developing the virtual serious game and virtual agents.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Deci, E.L.; Ryan, R.M. The general causality orientations scale: Self-determination in personality. J. Res. Pers. 1985, 19, 109–134. [Google Scholar] [CrossRef]
  2. Deci, E.L.; Ryan, R.M. A motivational approach to self: Integration in personality. Neb. Symp. Motiv. 1990, 38, 237–288. [Google Scholar]
  3. Grawe, K. Neuropsychotherapy; Routledge: Boca Raton, FL, USA, 2017. [Google Scholar]
  4. Lester, D.; Hvezda, J.; Sullivan, S.; Plourde, R. Maslow’s hierarchy of needs and psychological health. J. Gen. Psychol. 1983, 109, 83–85. [Google Scholar] [CrossRef] [PubMed]
  5. Ainsworth, M.D. Patterns of attachment behavior shown by the infant in interaction with his mother. Merrill Palmer Q. Behav. Dev. 1964, 10, 51–58. [Google Scholar]
  6. Ainsworth, M.S. Infant-mother attachment. Am. Psychol. 1979, 34, 932. [Google Scholar] [CrossRef] [PubMed]
  7. Bowlby, J. Attachment and Loss: Volume 1. Attachment; Basic Books: New York, NY, USA, 1969. [Google Scholar]
  8. Bowlby, J. Attachment and loss: Retrospect and prospect. Am. J. Orthopsychiatr. 1982, 52, 664–678. [Google Scholar] [CrossRef]
  9. Hazan, C.; Shaver, P. Romantic love conceptualized as an attachment process. J. Personal. Soc. Psychol. 1987, 52, 511. [Google Scholar] [CrossRef]
  10. Mickelson, K.D.; Kessler, R.C.; Shaver, P.R. Adult attachment in a nationally representative sample. J. Personal. Soc. Psychol. 1997, 73, 1092–1106. [Google Scholar] [CrossRef]
  11. Bartholomew, K.; Horowitz, L.M. Attachment styles among young adults: A test of a four-category model. J. Personal. Soc. Psychol. 1991, 61, 226. [Google Scholar] [CrossRef]
  12. Collins, N.L.; Read, S. Cognitive Representations of Attachment: The Structure and Function of Attachment Models. In Attachment Processes in Adulthood: Advances in Personal Relationships; Kingsley: London, UK, 1994. [Google Scholar]
  13. Mikulincer, M.; Shaver, P. Mental Representations of Attachment Security: Theoretical Foundation for a Positive Social Psychology. In Interpersonal Cognition; Guilford Press: New York, NY, USA, 2005; pp. 233–266. [Google Scholar]
  14. Mikulincer, M.; Shaver, P.R. Attachment in Adulthood: Structure, Dynamics, and Change; Guilford Press: New York, NY, USA, 2007. [Google Scholar]
  15. Rosenberg, M. Beyond self-esteem: Some Neglected Aspects of the Self-Concept. In Proceedings of the Annual Meeting of the American Sociological Association, New York, NY, USA, 30 August–3 September 1976. [Google Scholar]
  16. Avison, W.R.; Rosenberg, M. Conceiving the Self. Can. J. Sociol. 1981, 6, 212. [Google Scholar] [CrossRef]
  17. Bandura, A. Self-Efficacy: The Exercise of Control; Freeman: New York, NY, USA, 1997. [Google Scholar]
  18. Freud, S. Beyond the Pleasure Principle; W. W. Norton & Co.: New York, NY, USA, 1920. [Google Scholar]
  19. Armor, D.A.; Taylor, S.E. Situated Optimism: Specific Outcome Expectancies and Self-Regulation. In Advances in Experimental Social Psychology; Academic Press: Cambridge, MA, USA, 1998; Volume 30, pp. 309–379. [Google Scholar]
  20. Krizan, Z.; Windschitl, P.D. The influence of outcome desirability on optimism. Psychol. Bull. 2007, 133, 95–121. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Weinstein, N.D. Unrealistic optimism about susceptibility to health problems: Conclusions from a community-wide sample. J. Behav. Med. 1987, 10, 481–500. [Google Scholar] [CrossRef]
  22. Dahlitz, M. Neuropsychotherapy: Defining the emerging paradigm of neurobiologically informed psychotherapy. Int. J. Neuropsychother. 2015, 3, 47–69. [Google Scholar] [CrossRef]
  23. Higgins, E.T. Beyond pleasure and pain. Am. Psychol. 1997, 52, 1280. [Google Scholar] [CrossRef] [PubMed]
  24. Burtt, H.E.; Thorndike, E.L. The Psychology of Wants, Interests and Attitudes. Am. J. Psychol. 1936, 48, 179. [Google Scholar] [CrossRef]
  25. Carver, C.S.; White, T.L. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales. J. Personal. Soc. Psychol. 1994, 67, 319. [Google Scholar] [CrossRef]
  26. Fowles, D.C. Application of a behavioral theory of motivation to the concepts of anxiety and impulsivity. J. Res. Pers. 1987, 21, 417–435. [Google Scholar] [CrossRef]
  27. Gray, J.A. The Psychology of Fear and Stress; CUP Archive: Cambridge, UK, 1987; Volume 5. [Google Scholar]
  28. Gray, J.A. Brain Systems that Mediate both Emotion and Cognition. Cogn. Emot. 1990, 4, 269–288. [Google Scholar] [CrossRef]
  29. George, C.; Main, M.; Kaplan, N. Adult Attachment Interview; University of California: Berkeley, CA, USA, 1996. [Google Scholar]
  30. Melero, R.; Cantero, M.J. Affective styles in a Spanish sample: A questionnaire for the assessment of adult attachment. Clin. Health 2008, 19, 83–100. [Google Scholar]
  31. Rosenberg, M. Rosenberg, M. Rosenberg Self-Esteem Scale (SES). In Society and the Adolescent Self-Image; Princeton University Press: Princeton, NJ, USA, 1965. [Google Scholar]
  32. Ravitz, P.; Maunder, R.; Hunter, J.; Sthankiya, B.; Lancee, W. Adult attachment measures: A 25-year review. J. Psychosom. Res. 2010, 69, 419–432. [Google Scholar] [CrossRef]
  33. Grimm, P. Social Desirability Bias. In Wiley International Encyclopedia of Marketing; Wiley: Hoboken, NJ, USA, 2010. [Google Scholar]
  34. Nederhof, A.J. Methods of coping with social desirability bias: A review. Eur. J. Soc. Psychol. 1985, 15, 263–280. [Google Scholar] [CrossRef]
  35. Connolly, T.M.; Boyle, E.A.; MacArthur, E.; Hainey, T.; Boyle, J.M. A systematic literature review of empirical evidence on computer games and serious games. Comput. Educ. 2012, 59, 661–686. [Google Scholar] [CrossRef]
  36. Ritterfeld, U.; Weber, R. Video games for entertainment and education. In Playing Video Games: Motives, Responses, and Consequences; University of Southern California: Los Angeles, CA, USA, 2006; pp. 399–413. [Google Scholar]
  37. Shute, V.J.; Ventura, M.; Bauer, M.; Zapata-Rivera, D. Melding the power of serious games and embedded assessment to monitor and foster learning. Serious Games Mech. Eff. 2009, 2, 295–321. [Google Scholar]
  38. Rossiter, M.; Garcia, P.A. Digital storytelling: A new player on the narrative field. New Dir. Adult Contin. Educ. 2010, 2010, 37–48. [Google Scholar] [CrossRef]
  39. Ferguson, C.; Broek, E.L.V.D.; van Oostendorp, H. On the role of interaction mode and story structure in virtual reality serious games. Comput. Educ. 2020, 143, 103671. [Google Scholar] [CrossRef]
  40. Shute, V.J.; Wang, L.; Greiff, S.; Zhao, W.; Moore, G.R. Measuring problem solving skills via stealth assessment in an engaging video game. Comput. Hum. Behav. 2016, 63, 106–117. [Google Scholar] [CrossRef]
  41. Lumsden, J.; Edwards, E.A.; Lawrence, N.S.; Coyle, D.; Munafò, M.R. Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. JMIR Serious Games 2016, 4, e11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Mislevy, R.J.; Almond, R.G.; Lukas, J.F. A Brief Introduction to Evidence-Centered Design; American Psychological Association: Washington, DC, USA, 2004; Volume 1, pp. 2–9. [Google Scholar]
  43. Giglioli, I.A.C.; Parra, E.; Cárdenas-López, G.; Riva, G.; Raya, M.A. Virtual Stealth Assessment: A New Methodological Approach for Assessing Psychological Needs. In Serious Games, Proceedings of the Constructive Side-Channel Analysis and Secure Design, Valencia, Spain, 23–24 November 2017; Springer: Cham, Switzerland, 2017; Volume 10622, pp. 1–11. [Google Scholar]
  44. Bellotti, F.; Kapralos, B.; Lee, K.; Moreno-Ger, P.; Berta, R. Assessment in and of Serious Games: An Overview. Adv. Hum. Comput. Interact. 2013, 2013, 1–11. [Google Scholar] [CrossRef] [Green Version]
  45. Michaud, L.; Alvarez, J. Serious Games. In Advergaming, Edugaming, Training; IDATE Consulting & Research: Montpellier, France, 2008. [Google Scholar]
  46. Pasquier, P.; Mérat, S.; Malgras, B.; Petit, L.; Queran, X.; Bay, C.; Perez, J.P. A serious game for massive training and as-sessment of French soldiers involved in forward combat casualty care (3D-SC1): Development and deployment. JMIR Serious Games 2016, 4, e5. [Google Scholar] [CrossRef]
  47. Giglioli, I.A.C.; Pravettoni, G.; Martín, D.L.S.; Parra, E.; Raya, M.A. A Novel Integrating Virtual Reality Approach for the Assessment of the Attachment Behavioral System. Front. Psychol. 2017, 8, 959. [Google Scholar] [CrossRef]
  48. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef]
  49. Schwarzer, R.; Jerusalem, M. The general self-efficacy scale (GSE). Anxiety Stress Coping 2010, 12, 329–345. [Google Scholar]
  50. Carver, C.S. You want to measure coping but your protocol’too long: Consider the brief cope. Int. J. Behav. Med. 1997, 4, 92. [Google Scholar] [CrossRef] [PubMed]
  51. Perczek, R.; Carver, C.S.; Price, A.A.; Pozo-Kaderman, C. Coping, Mood, and Aspects of Personality in Spanish Translation and Evidence of Convergence with English Versions. J. Pers. Assess. 2000, 74, 63–87. [Google Scholar] [CrossRef] [PubMed]
  52. Morán, C.; Landero, R.; Eresa González, M.Ó.N.I.C.A. COPE-28: Un análisis psicométrico de la versión en español del Brief COPE. Univ. Psychol. 2010, 9, 543–552. [Google Scholar] [CrossRef] [Green Version]
  53. Fleming, T.M.; Bavin, L.; Stasiak, K.; Hermansson-Webb, E.; Merry, S.N.; Cheek, C.; Hetrick, S. Serious games and gami-fication for mental health: Current status and promising directions. Front. Psychiatry 2017, 7, 215. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. Hamari, J.; Tuunanen, J. Player Types: A Meta-synthesis. Trans. Digit. Games Res. Assoc. 2014, 1, 29–53. [Google Scholar] [CrossRef]
Figure 1. Consistency model.
Figure 1. Consistency model.
Applsci 11 01971 g001
Figure 2. Example of a virtual agent with a secureattachment style.
Figure 2. Example of a virtual agent with a secureattachment style.
Applsci 11 01971 g002
Figure 3. Crew mood state according to valence and arousal dimensions that participant could improve at the end of each session.
Figure 3. Crew mood state according to valence and arousal dimensions that participant could improve at the end of each session.
Applsci 11 01971 g003
Figure 4. Main study variables and their outputs.
Figure 4. Main study variables and their outputs.
Applsci 11 01971 g004
Figure 5. Machine learning strategy followed for each algorithm in Table 5 and each subscale to predict. Feature selection was made and validated by cross-validation (CV). When the best features were obtained, hyperparameter tuning was carried out, also validated with CV. With the best features and hyperparameters, modelling was done, and metrics were obtained by CV.
Figure 5. Machine learning strategy followed for each algorithm in Table 5 and each subscale to predict. Feature selection was made and validated by cross-validation (CV). When the best features were obtained, hyperparameter tuning was carried out, also validated with CV. With the best features and hyperparameters, modelling was done, and metrics were obtained by CV.
Applsci 11 01971 g005
Table 1. Demographic description of participants.
Table 1. Demographic description of participants.
VariableCategoryN%
GenderMale3151%
Female3049%
Age18–35 years3151%
36–55 years3049%
Education0 years00%
6 years23%
10 years00%
12 years47%
14 years711%
16 years915%
18 years711%
+18 years3252%
Marital StatusNon-independent single1830%
Independent single1220%
Married without children813%
Married with children1830%
Divorced without children23%
Divorced with cohabiting children12%
Divorced with independent children12%
Widower/widow with independent children12%
Employment StatusEmployed2439%
Self-employed610%
Student1220%
Housewife/husband35%
Unemployed1525%
Retired12%
Annual Gross IncomeLess than 15,000 €2643%
Between 15,001 and 30,000 €2033%
Between 30,001 and 60,000 €35%
Between 60,001 and 75,000 €12%
Not reported1118%
Table 2. Competency model of attachment with indicators.
Table 2. Competency model of attachment with indicators.
Grawe DimensionUnobservable ConstructsObservable Indicators
AttachmentBeliefs and attitudes related to attachmentNumber of times participant decided to solve a situation alone
Number of times participant decided to solve a situation with someone
Number of times participant decided to delegate situation to someone else to solve
Number of times participant decided to not solve a situation
Number of times virtual agents made decisions
Childcare decision-making
Attachment goals and needs
Plans and strategies
Table 3. Competency model of self-esteem with indicators.
Table 3. Competency model of self-esteem with indicators.
Grawe DimensionUnobservable ConstructsObservable Indicators
Self-EsteemWorth: feelings about selfSelf-evaluation
Social-worth: feelings about and from othersOthers’ evaluations
Others’ support
Table 4. Competency model of self-efficacy with indicators.
Table 4. Competency model of self-efficacy with indicators.
Grawe DimensionUnobservable ConstructObservable Indicators
Self-efficacyCompetenceReaction times (ms)
Correct/incorrect game performance
Number of completed trails
Self-evaluation
Table 5. Competency model of stress avoidance with indicators.
Table 5. Competency model of stress avoidance with indicators.
Grawe DimensionUnobservable ConstructObservable Indicators
Maximization of pleasure and minimization of stress/painAvoiding or approaching pleasant or unpleasant situationsNumber of times participant decided to solve a situation alone
Number of times participant decided to ask someone to explain
Number of times participant decided to avoid a situation
Number of times participant decided to do nothing
Table 6. Machine learning algorithms used, hyperparameters tuned, and values tested.
Table 6. Machine learning algorithms used, hyperparameters tuned, and values tested.
ModelParameterValues
Conditional inference treesMaximum depth(0, 100)
Minimum sum of weights in a node to be considered for splitting(5, 100)
Random forestNumber of trees(100, 1000)
Number of variables randomly sampled as candidates at each split(5, 100)
Support vector machine (SVM)C(−10, 10)
Sigma(−10, 10)
Generalized linear model (GLM) with lasso or elasticnet regularizationAlpha(0, 1)
Table 7. Descriptive analysis of variables of interest and results of categorization. High/Total column shows balancing of both categories (perfect balancing would mean High/Total of 0.5). * Relationship questionnaire (RQ) was not numeric, so the number of subjects in each category (low for insecure, high for secure) is the original one.
Table 7. Descriptive analysis of variables of interest and results of categorization. High/Total column shows balancing of both categories (perfect balancing would mean High/Total of 0.5). * Relationship questionnaire (RQ) was not numeric, so the number of subjects in each category (low for insecure, high for secure) is the original one.
SCALEMEANSDMEDIANCATEGORIZATION
LowHighHigh/Total
Attachment subscale 137.768.853731280.47
Attachment subscale 229.926.972932270.46
Attachment subscale 337.634.723825340.58
Attachment subscale 319.523.911933260.44
Relationship(RQ)*12470.8
Self-Esteem75.2912.2676.537230.38
Self-Efficacy29.715.92828320.53
BAS Subscale 110.782.061128300.52
BAS Subscale 211.262.511130280.48
BAS Subscale 315.123.961622360.62
BIS Subscale18.763.641832260.45
Brief Cope avoidant26.413.926.529290.5
Brief Cope approach29.433.682932260.45
Table 8. Metrics of models fitted for each subscale. Dataset column reports datasets that produced the best results along with number of variables + number of demographic variables included.
Table 8. Metrics of models fitted for each subscale. Dataset column reports datasets that produced the best results along with number of variables + number of demographic variables included.
VariableDatasetModelFeatures (N)AccuracyKappaTPRTNR
Attachment Subscale 1All (51 + 6)GLMNet150.90.760.880.9
Attachment Subscale 2All (51 + 6)GLMNet140.90.740.680.81
Attachment Subscale 3All (51 + 6)GLMNet140.850.630.880.83
Attachment Subscale 4All (51 + 6)GLMNet110.880.670.930.83
RelationshipAll (51 + 6)GLMNet110.910.580.650.98
Self-esteemDecisions (42 + 6)GLMNet100.770.460.640.88
Self-reported (26 + 6)SVM110.6300.021
Characters’ status (224 + 6)GLMNet120.980.960.971
Self-efficacyAttention (201 + 6)GLMNet150.950.890.980.93
Memory (114 + 6)GLMNet140.830.650.880.81
Velocity (106 + 6)GLMNet150.870.650.870.83
Planning (88 + 6)GLMNet150.920.80.950.89
Perception (53 + 6)randomForest130.720.380.690.71
Inhibition (44 + 6)GLMNet120.820.610.840.83
Flexibility (35 + 6)GLMNet130.780.440.790.81
Interference control (6 + 6)randomForest60.580.270.680.62
All (291 + 6)GLMNet140.950.880.930.96
BAS Subscale 1All (12 + 6)SVM110.670.160.830.48
BAS Subscale 2All (12 + 6)GLMNet90.730.450.770.83
BAS Subscale 3All (12 + 6)GLMNet110.720.350.790.57
BIS SubscaleAll (12 + 6)GLMNet90.80.480.750.8
BRIEF COPE AvoidantAll (12 + 6)GLMNet90.750.50.830.72
BRIEF COPE ApproachAll (12 + 6)GLMNet90.760.430.710.85
Note: TPR: True Positive Ratio; TNR: True Negative Ratio; GLMNet: Generalized Linear Models; SVM: Support Vector Machine.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Giglioli, I.A.C.; Carrasco-Ribelles, L.A.; Parra, E.; Marín-Morales, J.; Alcañiz Raya, M. An Immersive Serious Game for the Behavioral Assessment of Psychological Needs. Appl. Sci. 2021, 11, 1971. https://doi.org/10.3390/app11041971

AMA Style

Giglioli IAC, Carrasco-Ribelles LA, Parra E, Marín-Morales J, Alcañiz Raya M. An Immersive Serious Game for the Behavioral Assessment of Psychological Needs. Applied Sciences. 2021; 11(4):1971. https://doi.org/10.3390/app11041971

Chicago/Turabian Style

Giglioli, Irene Alice Chicchi, Lucia A. Carrasco-Ribelles, Elena Parra, Javier Marín-Morales, and Mariano Alcañiz Raya. 2021. "An Immersive Serious Game for the Behavioral Assessment of Psychological Needs" Applied Sciences 11, no. 4: 1971. https://doi.org/10.3390/app11041971

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop