Towards an Affective Intelligent Agent Model for Extrinsic Emotion Regulation

Abstract

Emotion regulation is the human ability to modulate one’s or other emotions to maintain emotional well-being. Despite its importance, only a few computational models have been proposed for facilitating emotion regulation. None of them prepare a plan of all the actions necessary for emotion regulation customized to the needs of a specific individual. To address this gap, we propose a computational model for an intelligent agent which, grounded in a multidimensional emotion representation, facilitates emotion regulation in individuals. This computational model is based on J. Gross’s theoretical framework of emotion regulation. An intelligent agent selects the most appropriate regulation strategy to maintain an individual’s emotional equilibrium considering the individual’s personality traits. A dynamic planner prepares a plan of emotion regulation actions which is dynamically adapted according to the emotional changes observed in the individual after applying the previous emotion regulation actions. This refinement of the initial regulatory action plan allows the proposed emotion regulation agent to adapt the plan to the specific characteristics of the individual, facilitating the individual to improve their emotion regulation capabilities and improve their emotional health.

1. Introduction

Over the course of human evolution, emotions have played an essential role that has allowed humans to progress as a species. Emotions play an adaptive function by preparing the individual to face new situations [1]. They also have a motivational function that helps guide humans in their decision-making and behavior. Finally, emotions also play a social function by allowing other humans to know our state of mood. Given the importance of emotions for our evolution and individual and collective well-being, numerous studies have been carried out from the perspective of psychology, neuroscience, sociology, and lately also from artificial intelligence, to better understand the process of emotion generation and to modify or at least regulate them for our benefit [2].

The process of emotion generation (Figure 1) has its origin in the perception of changes in the individual or in the environment in which the individuals find themselves [3]. This change is evaluated by the individual and an activation is produced, which involves several physiological changes. From this evaluation, and depending on the social and cultural context of the individual, the external expression of the emotion appears, and this external expression can be recognized by other individuals [4].

From birth, the human being begins to learn how to have some control over the emotion generated and expressed. This process is called emotion regulation [5,6]. Emotion regulation can be informally defined as an important aspect of emotional well-being that involves the ability to manage and modify one’s or other’s emotions effectively [7]. Emotion regulation is crucial in mental health and can help individuals deal with emotional disorders such as stress, anxiety, or depression. In addition, emotion regulation has practical applications in fields such as education, marketing, and entertainment [8,9,10].

James J. Gross, a renowned psychologist, has contributed significantly to the understanding of this field through his development of the process model of emotion regulation [2]. This widely accepted theory identifies different stages in the generation of emotions that can be modulated using different strategies. In his proposal, Gross establishes a framework for classifying these strategies, that facilitates a precise identification and specification of the actions involved in emotion regulation [11].

Within the computing domain, emotion regulation is part of the broader field of affective computing. Affective computing, as an interdisciplinary domain, seeks to develop computational intelligent systems with the ability to recognize, interpret, and simulate human emotions [12].

In this article, we present an intelligent agent, which is based on the theory of emotion regulation proposed by Gross [13], capable of regulating emotions in individuals. This article is the evolution and extension of our preliminary work [14], where the algorithms and formulas used during the emotion regulation process have been improved and a complete case study has been incorporated. The proposed agent uses an arousal- and valence-based emotion representation to facilitate extrinsic emotion regulation in individuals. To this end, we propose a dynamic planner that uses user’s affective characteristics (e.g., emotional state and personality) to determine a set of personalized actions that are focused on regulating the user’s emotion toward an emotional equilibrium state.

2. Background

Let us start by defining the theoretical framework on which our intelligent agent for emotion regulation is based. Our conceptual framework for emotion regulation draws heavily from the theories advanced by James J. Gross [13] and aligns with the conceptualization and representation of emotions proposed by James A. Russel [15]. According to Gross, emotion regulation is composed of a variety of cognitive, behavioral, and physiological processes employed by individuals to control the elicitation, experience, and expression of their emotions. The ability of regulating emotions is crucial for maintaining emotional balance, promoting mental well-being, and facilitating effective decision-making in various situations individuals face. Effective emotion regulation endows individuals with the ability to adapt their emotional responses flexibly and adaptively, taking into account environmental influences and personal objectives [16,17]. This regulation process can be carried out by a individual to control or direct their own emotions, which is known as intrinsic emotion regulation or self-regulation [18]. It can also be carried out by a person to influence the emotional state of a third person, which is known as extrinsic emotion regulation [19]. The capacity for self-regulation is learned and improved throughout the human lifespan. Thus, an individual’s ability to emotionally self-regulate can increase throughout their life, especially if the individual can follow some role models or a therapist to guide this process. With continued practice of emotion regulation activities, the ability to regulate can be improved [20]. The affective agent for emotion regulation proposed in this article can be a promising alternative to facilitate this process of improving emotion regulation. Note that, although in this article we use the generic term emotion regulation, the proposal is only focused on extrinsic emotion regulation.

2.1. Process Model of Emotion Regulation

One of the most used and referenced model of emotion regulation is the Emotion Regulation Process Model proposed by the psychologist James J. Gross [5,13]. This model proposes that an individual can influence various phases of the emotion generation process through the implementation of specific actions or strategies.

The emotion regulation process is usually carried out iteratively following three steps:

• Observation: This step recognizes the current emotional state of the individual.
• Evaluation: This process involved comparing the observed emotional state with the desired target state.
• Reaction: If the desired emotional state has not been reached, some necessary adjustments must be introduced to effectively modify the emotional state.

This model of emotion regulation distinguishes between antecedent-focused regulation strategies and response-focused regulation strategies (see Figure 2). Antecedent-focused strategies for emotion regulation involve the regulation of emotions before the emotions are triggered, as noted in [21]. These strategies can be implemented by either modifying the trigger of the emotion or altering one’s cognitive processes. On the other hand, response-focused strategies are aimed at managing the response elicited by the emotion.

Gross categorizes regulation strategies into different families based on the stage of the emotion generation process they influence [5]. These families include:

• Situation selection: Strategies within situation selection focus on deciding what situations a person faces, for example, avoiding confronting situations that evoke negative emotions such as sadness. However, to apply this strategy effectively, it is necessary to have the ability to predict the emotional response that the situation will produce, which is difficult in many situations [11,22].
• Situation modification: These strategies involve altering a situation to achieve a more desirable emotional response. This type of modification pertains specifically to altering the external physical environment. Sometimes, it may be difficult to distinguish between selection and modification strategies since the changes made in one situation may be perceived as creating a new situation instead [5,11,13].
• Attentional deployment: Strategies within this family are aimed to redirect attention between elements of the external environment or between personal thoughts [23,24]. Distraction and concentration are the most common strategies. Distraction consists of redirecting attention from the emotional aspect of the situation to another, avoiding its emotional charge. Concentration would be its counterpart and refers to drawing attention to emotional features of a situation [25].
• Cognitive change: Cognitive change strategies consist in altering the individual’s evaluation or appraisal of a situation. The most commonly reported technique is reappraisal, which involves altering the individual’s internal interpretation or understanding of the situation. Another strategy is decentering, which consists of seeing an event from a broader perspective, observing one’s inner experiences as transient and separate from one’s self [26,27].
• Response modulation: Response modulation involves influencing the emotional response in its behavioral, experiential, or physiological components. A well documented strategy in this family is expressive suppression, which consists of inhibiting the externalization of emotional expressions. Exercise, sleep, and alcohol or drug use are also considered ways of response modulation.

2.2. Affective State Representation

The notion of emotional equilibrium is crucial when considering emotional stability [28]. Emotional equilibrium denotes an individual’s natural emotional state in the absence of external or internal events. This state differs from one person to another and is influenced by factors like cognitive development, personality, and past experiences.

In order to adequately represent and reason with a person’s emotional state and detect the need to use emotion regulation, it is necessary to use an emotional state representation model. One of the most commonly used methods of emotion representation is the multidimensional model proposed by J. Rusell [15]. That model represents emotions in a two-dimensional space, where valence is placed on the horizontal axis and arousal is placed on the vertical axis (see Figure 3). Valence is a subjective measure of emotional experience that indicates whether the emotion is pleasant or unpleasant to the person. Arousal, on the other hand, refers to the level of activation or stimulation a person feels in response to an emotional stimulus. In addition, a pair of arousal and valence values can be associated with a category of emotions. For example, the emotion of happiness is associated with a high level of positive valence and a low level of positive arousal. In contrast, the emotion of fear is associated with a high level of arousal and a negative valence. The measurement of a person’s valence and arousal values can be estimated in several ways. Some physiological parameters, which can be measured by smart wearables like wristbands or watches, have been proven to be effective measures of the levels of arousal and valence experienced by a person in response to an emotional stimulus [29].

3. Related Work

The capability of agents to facilitate emotion regulation is proposed as an effective approach to mitigate negative emotions that can detrimentally impact cognitive skills, such as stress and frustration. An agent with the ability to perform actions that facilitate the emotion regulation of individuals has numerous practical applications. An illustrative example of this capability is found in the works presented in [30], which examines how agents contribute to frustration management in drivers, with a consequent positive impact on reducing the risk of traffic accidents. To evaluate this effectiveness, a driving simulator, as detailed in [31], was used to test a specific agent. By using cognitive reappraisal strategies, this agent managed to decrease driver frustration and, as a result, improved driving performance.

Another example is presented in [32]. The authors propose the Help4Mood model, which incorporates an affective agent based on the FAtiMA architecture [33]. The purpose of that model is to assist patients with depression through the application of specific strategies for cognitive change and response modulation. The agent, designed to interact with the individual, uses visual cues on the screen and tests to monitor the individual’s emotional state [34]. The results indicate that affective agents can demonstrate outstanding efficacy in therapeutic contexts by providing substantial therapeutic support and facilitating effective communication with healthcare professionals. This study thus contributes to understanding the effectiveness of affective agents in enhancing psychological interventions.

Emotion regulation also facilitates the creation and strengthening of social relationships. In [35], a model for developing intelligent virtual agents that use emotion regulation to establish and strengthen social relationships is proposed. The model enables the agent to generate and reason about emotions. Emotion regulation is implemented through instrumental and relationship-oriented actions. The model was evaluated in a video game experiment concluding that agents with emotion regulation are perceived as providing more intimacy, help, emotional security, and self-validation functions.

Some approaches focus on modeling the emotion regulation strategies proposed by Gross. For example, CoMERG stands out a computational model that focuses on antecedent-focused strategies of emotion regulation [36,37]. The influence of each regulation strategy is articulated through a set of critical parameters, such as weight (indicating the impact on the emotion regulation process), emotional value (reflecting the contribution to the emotional response), willingness value (representing the willingness to modify the emotion), and personal flexibility to adapt to emotion regulation behavior. Each regulation strategy influences the adjustment between the current emotional level and the desired emotional level of the agent. These emotional levels are represented quantitatively, allowing for precise measurements and comparisons. The disparity between these quantitative variables determines the level of adjustment in the regulation process, outlining the relative influence of the different strategies in that process. Unlike the proposal we make in this article, this model specializes exclusively in emotion regulation without addressing the emotional generation process. Moreover, this proposal focuses on the simulation of emotion regulation patterns based on all the parameters mentioned above. Since it is not conceived as an agent to select and plan the best emotion regulation actions, it does not consider the planning processes of the necessary regulation actions, the replanning after the execution of each of the actions, nor the selection of the best action based on the state resulting from the execution of the previous actions.

Models based on natural language processing, especially large language models (LLMs), can benefit from using emotion regulation strategies to communicate effectively with the individual. For example, in [38], an end-to-end dialog model that uses transformers to adapt conversations to various contexts and emotional states is presented. With a specific focus on emotion regulation, the model considers contextual factors and generates empathic responses to modulate individual emotions effectively. ER-Chat was trained using the EmpatheticDialogues [39] dataset, which includes detailed emotion and intention labels. Evaluation results in experiments with the chatbot demonstrate improved performance and increased individual acceptability, highlighting the promising potential of this approach to advance emotionally aware natural language processing systems. Ni et al. [40] have performed a fine tuning of the Chinese version of ChatGPT with which they have achieved a significant improvement in the regulation of negative emotions, although they have not achieved any improvement in the regulation of positive emotions. Unlike our proposal, in this work, the process of emotion regulation is not modeled in any way and all emotion regulation relies on the effectiveness (or lack of effectiveness) of a LLM. Therefore, there are no specific actions of emotion regulation defined, nor planning. There is also no control of the result in the emotional state of each interaction during the dialogue.

In addition, chatbots have the potential to play moderator roles, promoting emotion regulation within group dynamics and consolidating themselves as valuable tools in this context. An illustrative example is GremoBot (Group emotion Bot) [41], a prototype designed to improve performance in group work by implementing reappraisal and attentional deployment strategies. GremoBot monitors sentiment and tone in the group, intervening when it detects a lack of interaction or negative sentiment. In such cases, the bot uses different techniques to realign the group’s focus and encourage a positive reinterpretation of the situation. To enhance individual engagement, predefined phrases and emojis are strategically employed, contributing to a friendlier and more effective communication environment.

As we have observed in the existing literature, several projects have been focused on the development of systems that implement emotion regulation techniques for a variety of purposes. However, when examining these proposals, it can be observed that there is a gap related to the absence of a computational framework capable of dynamically adapting regulation strategies, by planning the necessary emotion regulation actions to meet individual human needs. This critical gap highlights the need for innovative computational models that can systematically plan and personalize emotion regulation actions, addressing a significant aspect in the quest for emotional well-being.

4. Emotion Regulation Agent

This study introduces a computational model which facilitates emotion regulation in individuals (see Figure 4) employing a multidimensional emotion representation based on arousal and valence. Our model is composed of three fundamental components. An emotion recognition process, a personalized dynamic planner and a process in charge of executing the actions focused on regulating the user’s emotion.

An affective intelligent agent interacting with an individual will detect the emotions expressed by an individual. This emotion recognition process relies on the analysis of physiological data of the individual. The aim of collecting physiological data is to gather diverse indicators enabling the identification of the individual’s emotional state. It is crucial to identify techniques that offer precise data on the external expression of emotions. To minimize the impact of the physiological data collection process on the individual’s behavior, only minimally invasive techniques are considered. Among the possible techniques [42], we currently consider facial recognition, body temperature, and galvanic skin response.

Different people show different emotion regulation abilities. Similarly, different regulation actions will have different effects depending on the person performing the regulation action. Some authors indicate that a person’s regulation capacity remains fairly stable and can be considered a personality trait [43]. Whenever possible, it is important to know the personality characteristics of the individual so that the intelligent agent is able to select the emotion regulation strategies that a priori are most appropriate to the personality traits of the individual. These personality traits can be obtained through personality tests. Our proposal employs the Big Five personality model (Five-Factor-Model), also known as OCEAN [44,45]. There are several personality tests to obtain these traits. In our proposal, we have chosen the revised NEO Personality Inventory because it is one of the most commonly accepted by the psychology community [46].

Based on the emotion recognized in the individual, their personality traits, and their detected state of emotional equilibrium, the intelligent affective agent must select a set of emotion regulation actions and carry out the actions to achieve the emotion regulation of the individual and bring them back to their emotional equilibrium. This is the most essential process to be carried out by the agent. The agent use a dynamic planner of emotion regulation actions for this process. We have chosen to employ the Gross model to formulate the agent’s strategies, because this model unifies the different perspectives of emotion regulation and it has been established as the reference model. However, the emotion regulation strategies proposed by Gross are not specific enough for application in a computational model. Therefore, emotion regulation strategies should be divided into more specific sub-strategies so that they can be converted into real actions to be applied by the agent and their emotional impact can be estimated.

In our proposal, an agent has a catalog of actions A belonging to one of the five strategies defined in Section 2.1:

$$\begin{array}{cc}\hfill \mathit{Stat}=& \{\mathit{Situation}\mathit{selection},\mathit{Situation}\mathit{modification},\mathit{Attentional}\mathit{deployment},\mathit{Cognitive}\mathit{change},\hfill \\ \hfill & \mathit{Response}\mathit{modulation}\}\hfill \end{array}$$

Considering the set of strategies St, the agent will prepare a plan defined as a list of regulation actions $\left\{{\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_n\right\}$ (where ${\alpha }_i\in A$) that are sequentially executed by the executor process. A regulation action $\alpha$ consists of a tuple $〈\phi ,\Delta S〉$ where $\phi$ is a set of specific steps that must be executed to apply the regulation action $\alpha$ and $\Delta S$ is the expected modification of the recognized emotion state ($S_a$) after applying the regulation action $\alpha$ ($S_a\to S_e$). Consequently, if in an instant of time t the recognized emotional state is $S_a$, the expected resulting emotional state after applying the regulation action $\alpha$ is define as $S_{e,\alpha }=S_a+\Delta \pm \psi$ where $\psi$, called threshold of acceptance or tolerance, represents the allowed difference between the predicted emotional state after applying the emotion regulation action and the emotional state actually obtained. Circles around each state $S_{e,0},S_{e,1},\cdots S_{e,4}$ in Figure 3 represent this tolerance threshold ${\psi }_i$ for each predicted emotional state $S_{e,i}$.

Our computational model associates each emotion regulation action with the possible changes that can produce in the individual’s emotional state and the costs of applying that regulation action. An intelligent agent uses a planning algorithm to select the best strategy based on the individual’s emotional state, the effect of the regulation actions, the cost of applying each regulation action, and an equilibrium emotional state adjusted to the individual’s emotional equilibrium and personality. This is an iterative process in which a dynamic planner analyzes the actual effect that each applied emotion regulation strategy has produced in the individual until the desired equilibrium emotional state is reached.

We rely on the BDI model [47] to model the agent’s behavior. The BDI model (Beliefs-Desires-Intentions) is a conceptual framework for modeling agents based on practical reasoning. Beliefs represent the knowledge that the agent has about itself and the environment. Desires reflect the goals that the agent wishes to achieve. Finally, intentions are the goals or actions that the agent commits to perform. Algorithm 1 defines the agent’s characteristics for emotion regulation behavior.

Algorithm 1 Agent behavior

1: $B\leftarrow B_0$ 2: $I\leftarrow I_0$ 3: $S_{\epsilon }\leftarrow$get_equilibrium_state() 4: while true do 5:     $\beta \leftarrow \mathit{percept}\left(\right)$ 6:     $B\leftarrow \mathit{belief}\_\mathit{revision}(\mathit{B},\beta )$ 7:     $S_a\leftarrow$recognize_emotional_state$\left(\right)$ 8:     $D\leftarrow \mathit{options}(B,I,S_a)$ 9:     $I\leftarrow \mathit{filter}(B,D,I,S_a)$ 10:    if distance$\left(S_a-S_{\epsilon }\right)>\psi$ then 11:         $\pi \leftarrow \mathit{plan}(B,D,A,S_a,S_{\epsilon })$ 12:         while not (empty($\pi$) or succeeded($S_a$,$S_{\epsilon }$)) do 13:            $\alpha \leftarrow$ first element of $\pi$ 14:            execute($\alpha$) 15:            $\pi \leftarrow$ tail of $\pi$ 16:            $B\leftarrow \mathit{belief}\_\mathit{revision}(\mathit{B},\beta )$ 17:            $S_{a,\alpha },\leftarrow$recognize_emotional_state$\left(\right)$ 18:            $D\leftarrow \mathit{options}(B,I,S_{a,\alpha })$ 19:            $I\leftarrow \mathit{filter}(B,D,I,S_{a,\alpha })$ 20:            if not succeeded($S_{a,\alpha }$,${\mathit{S}}_e$) then 21:                $\pi \leftarrow \mathit{plan}(B,D,A,S_a,S_{\epsilon })$ 22:            end if 23:         end while 24:     end if 25: end while

Initially, the agent has a set of initial beliefs $B_0$ and initial intentions $I_0$. The function get_equilibrium_state is used to identify the equilibrium state of the user. Then, the percept method allows for identifying data from the environment in the form of perceptions and messages that the agent may receive. The belief_revision function checks the data coming from the perceptions $\beta$ and obtains and modifies the beliefs B of the agent. Subsequently, the method recognize_emotional_state obtains the individual’s current emotional state e. With this information, the desires D of an agent are obtained, which in this case are goals related to reaching the state of emotional equilibrium in the individual. The function filter filters out the intentions I that the agent can commit to. Next, if the current emotional state of the user $S_a$ differs from the equilibrium state $S_{\epsilon }$ with a threshold $\psi$, the method plan, allows to obtain a plan $\pi$ which, as previously indicated, is composed by a sequence of actions $\left\{{\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_n\right\}$. Then the execution of the plan is started by executing the actions sequentially. For each action ${\alpha }_i$ executed, the individual’s emotional state representation in the agent is updated. Finally, the succeeded method determines if the action has produced the expected emotional state, using the individual’s equilibrium state $\epsilon$ and the threshold of acceptance $\psi$.

4.1. Emotion Regulation Planner

Emotion regulation involves comparing the current emotional state with the desired emotional state (the equilibrium emotional state). If there is a discrepancy, it will be necessary to initiate a regulation process. The core of our model for the regulation process lies in the planning of the actions to be performed by the agent to apply a regulation action represented by the plan function in Algorithm 1. By dividing the final objective (final emotional state, or equilibrium emotional state) into sub-objectives (all the intermediate emotional states), the regulation process is facilitated, and the result of applying of each regulation action can be evaluated.

The planning process will begin when it is detected that the emotional state deviates from the equilibrium emotional state. Then, the aim of the agent will be to reach the emotional equilibrium again, moving the emotional state towards the equilibrium state of the individual. The planner searches the most appropriate regulation strategies taking into account the individual’s personality traits. This planner uses the individual’s emotional state and the impact of each regulation action on the individual’s arousal and valence, and selects a set of actions to be performed during the regulation process until the state of emotional equilibrium is reached.

An example of planning regulation actions to reach the equilibrium state can be seen in Figure 3. The circles represent the expected states $S_{e,i}$ after applying the regulation action ${\alpha }_i$ (with a tolerance threshold $\psi$), and the lines represent the change in the affective state resulting from the application of the regulation actions suggested by the planner.

Our proposed model uses a dynamic planner so that the process to reach the target emotional state is an iterative process in which, if any of the intermediate steps do not achieve the expected effect, a replanning process takes place to redefine the plan adding or removing actions. The success or failure of a regulation action ${\alpha }_a$ can be estimated by calculating the difference between the actual individual’s emotional state $S_{a,\alpha }$ and the expected target emotional state $S_{e,\alpha }$ resulting from the application of the action $\alpha$. This target emotional state is calculated considering the initial emotional state S represented by its two dimensions, arousal and valence, and the modification that the action was estimated to cause in these two dimensions $\Delta S$. In order to decide the success or failure of this emotion regulation action $\alpha$, the expected or target state $S_{e,\alpha }$ is calculated with a threshold $\psi$ of acceptance, or tolerance, (see Figure 3). Consequently, the function for deciding the success or failure of a regulatory action $\alpha$ is defined by the following formula:

$${\mathit{succeeded}}_{\alpha }(S_{a,\alpha },S_{e,\alpha })=d(S_{e,\alpha },S_{ϵ})\le {\psi }_{\alpha }$$

(1)

where $S_{e,\alpha }$ is the expected emotional state after applying the regulation action $\alpha$, $S_{a,\alpha }$ is the emotional state actually reached after applying the regulating action and the function d represents the euclidean distance between the states $S_{e,\alpha }$ and the individual’s equilibrium emotional state $S_{ϵ}$. Both states are represented by their dimensional components (arousal and valence).

Note that during the regulation process, the planning of each action implies not only deciding the concrete action to be performed, but also deciding when it will be performed, for how long, in what context, and what material and temporal resources will be necessary to employ. To represent these requirements, the different actions ${\alpha }_i$ to apply emotion regulation strategies are associated with a cost. This cost is an estimation of the resources needed to carry out the action associated with the regulation strategy, such as the time required for the individual to perform the action or the cost of modifying their situation. As will be described in the following sections, this cost and the utility of each action is used by the planning algorithm to select the action to be employed in each step of the emotion regulation process.

4.2. Personality Traits and Customization

It is important to keep in mind that the effects of the different regulation strategies may vary from one person to another, depending on their personality, emotional state, context, and culture. Therefore, it is essential to identify which strategies work best for an individual in each specific situation. In addition, to estimate the impact that different strategies have on the emotional state the individuals, it is important to customize the model to each particular individual.

As a first approach to customizing the system for a specific individual, in this work, we use the personality defined by the Five Factor Model (FFM) [44,45]. The FFM is a commonly used framework that describes personality along five dimensions: openness, conscientiousness, extroversion, agreeableness, and neuroticism. Each of these dimensions can influence how people experience and regulate their emotions.

Research has shown that personality traits may be related to different preferences or facility for emotion regulation strategies. People who score high on the openness trait are good at applying cognitive change strategies. In contrast, people with high conscientiousness tend to use attentional deployment strategies [48,49]. This information can be used to select specific techniques that may work best for individuals with different personality traits.

Considering this, we propose a new dynamic planner that select the best expected regulation actions according to the emotional state and personality of the user knowing the impact that each action $\alpha$ has on the user’s arousal and valence ($\Delta S_{\alpha }$) that represent the user’s emotional state, and the level of each user’s personality trait (t). Using the Formula (2), the planner chooses the action that minimize the distance between the expected achieved state after applying the action ($S_{e,\alpha }^{\prime }$), and the user’s emotional equilibrium state ($S_{ϵ}$).

$$\widehat{\alpha }=\underset{\alpha \in A}{argmax}\left(d{(S_{e,\alpha }^{\prime },S_{ϵ})}^{-1}+P_{\alpha }\right)$$

(2)

where d is the euclidean distance between the states, $S_{ϵ}$ is the equilibrium state of the user, and $S_{e,\alpha }$ is an estimation of the impact that the action will have on the user’s emotional state, calculated as:

$$S_{e,\alpha }^{\prime }=S_a+\Delta S_{\alpha }$$

(3)

$P_{\alpha }\in [-1,1]$ represents the suitability of the action $\alpha$ according to the user’s personality traits, calculated as:

$$P_{\alpha }=\frac{1}{N}·\sum _{i=0}^N{\theta }_{\alpha }·t_i$$

(4)

where N is the number of personality traits (five in the case of the OCEAN model), $t_i$ represents the value for the i-th personality trait of the user, and ${\theta }_{\alpha }$ is the correlation between the action $\alpha$ and the i-th personality trait as described in

Table 1. Correlation between the personality traits of the OCEAN model and the regulation strategies proposed by Gross in [50].

Strategy	Action $\mathit{\alpha }$	O	C	E	A	N
Situation selection	Avoidance	−	+	−	0	+
Situation modification	Self-assertion	+	+	+	−	−
Attentional deployment	Distraction	+	+	0	0	−
Cognitive Change	Reappraisal	+	0	0	0	−
Response modulation	Suppression	0	0	−	0	0

.

We estimate the individual’s personality traits by a Big Five personality traits test [46]. With this test, we customize the values of the coefficients of each personality trait t in the range from 0 to 1. For example, for the attribute of extroversion, a value of 1 means that the person is completely extroverted, while a value of 0 indicates that they are completely introverted. These weights are assigned to the different emotion regulation strategies and can be estimated by experimentation such as done in [51,52]. This customization is also improved through a learning approach, where the model continuously adapts these weights to the individual’s emotional responses and feedback. This model learns from previous experiences how different emotion regulation strategies influence the individual affective state.

4.3. Planner Improvement and Individual Personalization

The success of the emotion regulation process carried out by the intelligent agent depends to a great extent on the correct selection of the emotion regulation strategy and the specific actions to be executed. To achieve this, the agent needs to learn from the result of the executed actions so that its decisions are improved over time. Considering that the effectiveness of the regulation actions depends strongly on the unique characteristics of the individual, personalization becomes essential for this process. Personalization for each individual is achieved through a reinforcement learning strategy [53], an approach that addresses the problem within the framework of Markov Decision Processes (MDP). In this strategy, an agent accumulates rewards over time by performing actions in an environment with the goal of maximizing the total amount of rewards obtained. In this process, the agent must learn to make decisions in a dynamic environment to maximize a cumulative reward. Different algorithms can be used for this purpose. We have selected Q-learning [54], which has stood out from the rest in recent literature [55,56]. This will allow the agent to learn how to plan the best sequences of actions for a specific individual to transition from the current emotional states to the equilibrium emotional state.

The states to be considered in this learning process will be the different combinations of arousal and valence that represent the individual’s emotional state. Actions, on the other hand, constitute the set of emotion regulation actions available in the agent’s plans.

The application of Q-learning for this case involves assigning values to each possible combination of emotional states and regulation actions, constructing what is known as a Q-table. Initially, these values in the Q-table reflect the general knowledge of how specific actions may influence the emotional state and its correlation with the user’s personality attributes. These values are obtained in a first estimation and initial stage of the customization seen in the previous section and thanks to this, the learning algorithm does not start from scratch, but from a well-founded basis. The Q-table is adjusted as the agent applies different emotion regulation actions to an individual. The updating of this table follows a formula defined for Q-learning, which weights the immediate reward obtained by an action (success or failure of the regulation action) with the estimate of future reward. This approach allows the agent to learn from the long-term consequences of its decisions. This process is repeated over multiple iterations, allowing the agent to adapt and customize its decisions in response to the emotional needs of one specific individual, as depicted in the table Q. Furthermore, this learning allows to obtain estimations that take into account the emotional state from which each action or strategy is performed. This is due to the fact that the agent is learning for each of the states which are the most effective strategies to reach the marked state of equilibrium. Through this process, we are confident that the dynamic emotion regulation planner will personalize its decisions to the individual, improving the effectiveness of the emotion regulation process.

5. Case Study: Application of the Emotion Regulation Agent

To illustrate the practical application of our proposed emotion regulation agent, let us consider a hypothetical scenario involving an individual named Alex. Alex is a college student who experiences stress and anxiety during exam periods.

• Initialization: The process begins with the emotion regulation agent initializing its understanding of Alex’s emotional state. Alex takes a personality test based on the Big Five model, providing the agent with information about his personality traits. The agent also collects baseline physiological data, such as heart rate, skin conductance, and facial expressions, to understand Alex’s emotional state in a neutral context. In this way, the agent estimates Alex’s equilibrium state. As long as Alex’s is in his emotional equilibrium state, the agent will maintain this belief using the predicate:

  $$\mathit{equilibrium}\_\mathit{state}(A_{eq},V_{eq})$$

  where $A_{es}\in [-1,1]$, $V_{es}\in [-1,1]$ represent the arousal and valance of the equilibrium state ($A_{eq},V_{eq}\in S_{\epsilon }$), respectively. For instance, the the equilibrium state of Alex is set as:

  $$\mathit{equilibrium}\_\mathit{state}(0.3,0.3)$$
• Monitoring: As Alex prepares for an upcoming exam, the emotion regulation agent continuously monitors physiological indicators to recognize any changes in his emotional state. The emotion recognition module uses these data to estimate Alex’s emotional state, represented by its arousal and valence values which are internally represented as a belief using the predicate:

  $$\mathit{emotional}\_\mathit{state}(A_e,V_e)$$

  where $A_e\in [-1,1]$, $V_e\in [-1,1]$ represent the arousal and valance of the Alex’s emotional state ($A_e,V_e\in S_a$), respectively. For example, the emotional state of Alex in an instant t is:

  $$\mathit{emotional}\_\mathit{state}(-0.5,0.7)$$
  When is detected that the user is not in his equilibrium emotional state by the method presented in Equation (1), the planner will be activated and the agent will start planning actions to regulate Alex’s emotional state and keep him away from the detected anxious state. For instance, setting the threshold $\psi$ to $0.25$ and using the euclidean distance, the current emotional state of Alex is deviated from the equilibrium state in $0.89$, exceeding the established threshold $\psi$. The planner will have to establish actions that will allow Alex to return to its equilibrium state.
• Planning: The emotion regulation agent, provided with knowledge of Alex’s personality traits and detected emotional state, initiates the planning phase. It formulates a plan to help Alex regulate his emotions to achieve an affective equilibrium state. The plan includes a sequence of actions categorized into different emotion regulation strategies. For instance, using the OCEAN personality model, the personality of Alex is defined as:

  $$\begin{array}{c}\hfill \mathit{openness}(\mathit{alex},0.8)\\ \hfill \mathit{conscientiousness}(\mathit{alex},\mathit{0.6})\\ \hfill \mathit{extraversion}(\mathit{alex},\mathit{0.5})\\ \hfill \mathit{agreeableness}(\mathit{alex},\mathit{0.9})\\ \hfill \mathit{neuroticism}(\mathit{alex},\mathit{0.2})\end{array}$$
  Based on the expected effect of the different actions and preferences given by the user’s personality, the agent can make an estimation of what are the best emotion regulation actions to perform in order to help the user reach his equilibrium state.

  Table 2. Example of the values used by the dynamic planner to select the action at a given time instant.

  Action $\mathit{\alpha }$	$\mathbf{\Delta }$S		$\mathit{d}{({\mathit{S}}_{\mathit{e},\mathit{\alpha }},{\mathit{S}}_{\mathit{ϵ}})}^{-1}$	${\mathit{P}}_{\mathit{\alpha }}$	Result
  	Arousal	Valence
  Avoidance	$-0.1$	$+0.2$	$1.49$	$-0.10$	$1.39$
  Self-assertion	$+0.1$	$+0.3$	$1.41$	$0.16$	$1.57$
  Distraction	$-0.3$	$+0.2$	$1.64$	$0.24$	$\mathbf{1}.\mathbf{88}$
  Reappraisal	$-0.1$	$+0.3$	$1.71$	$0.12$	$1.83$
  Suppression	$-0.2$	$+0.1$	$1.37$	$-0.10$	$1.27$

  shows this estimation considering the personality of Alex, where the highest score value represents the best action.
  In this case, the agent has planned to perform two actions. First, it will encourage Alex to perform a distraction technique (attentional deployment strategy). Then, based on the expected emotional state after applying the first action, the next planned action is to reframe his thoughts about the exam by a reappraisal exercise (cognitive change strategy).
• Action Execution: The agent communicates with Alex, providing guidance and instructions for the planned actions. Alex, guided by the agent, tries to distract himself, diverting his attention away from stressors. Then, if the previous action was successful, the agent continues to perform the next intended action and encourages positive affirmations helping Alex reevaluate his perspective on the situation with his next exam. If any of the steps in the plan deviate from the expected response in Alex’s emotional state, then the plan would be readjusted from that point, choosing new actions based on Alex’s current emotional state. The agent will consider an action successful if the difference between the current and expected emotional state does not exceed the established threshold, and will not be considered successful in the opposite case; as can be seen in Equation (1).
• Personalization: During and after the execution of the planned actions, the emotion regulation agent continuously monitors Alex’s physiological responses and estimates his emotional state at each point. It analyzes the effectiveness of the applied strategies by comparing the actual emotional state with the expected outcomes. If a more positive emotional state is observed after the actions performed, the agent considers these actions successful and increases the probability of using such actions in the future in similar context. This is achieved by means of a Q-learning algorithm, adjusting the values of the Q-table with the feedback received and the corresponding formula of this learning algorithm.
  This learning process, which considers general knowledge derived from personality traits as a first approximation and individualized responses observed in real-time interactions to enhance personalization to the individual, ensures that the agent continuously improves its ability to assist individuals in managing their emotions effectively, contributing to long-term emotional well-being.

6. Conclusions and Future Work

In this article we have presented a preliminary work that will need to be validated by experiments conducted by a multidisciplinary team of specialists in artificial intelligence and psychology.

The agent model described in this work facilitates emotion regulation based on the arousal-valence dimensions of emotions and the process model of emotion regulation. Each regulation action is associated with the changes that can be produced in the arousal and valence values. This allows an agent to assist the individual in reaching a target affective state or maintaining emotional equilibrium. It is important to note that the effects of different actions to regulate emotions may vary from one individual to another, depending on different factors such as the individual’s personality. Therefore, our model evaluates the result of each action taken during the regulation process and determines what the next action should be to achieve emotional equilibrium. This process dynamically personalizes the individual experience and learns from previous experiences which results from applying the emotion regulation actions in a specific individual. The dynamic planner estimates the best actions for emotion regulation based on the individual personality, their current emotional state, their equilibrium emotional state, and the cost and impact on arousal-valence of each strategy. The proposed emotion regulation agent model not only can be used to apply extrinsic emotion regulation on an individual, but will also allow the individual to improve their self-regulation capacity.

Some possible real-life applications that our emotion regulation agent model can have include helping workers and students to control their stress levels by monitoring and helping them to cope with them when their levels reach peaks. Also within the mental health field, the model can be the basis for building a chatbot to help people with anxiety or depression to regulate their emotional state or cope with certain symptoms by planning and guiding the different exercises. In a similar way, it can also be used for a more general population to help them improve their mood and learn to regulate their emotions effectively. Furthermore, as emotions are intrinsic to the human being, they are transversal to many fields and the model could be adapted to them. For example, in the entertainment field, the goal can be shifted from seeking the equilibrium state to specific emotional states to achieve more realistic and personalized interactive experiences.

While the proposed model for emotion regulation provides a promising framework for assisting individuals in regulating their emotions, several areas can still be explored for future work. For instance, one area for future work is to investigate the role of context in emotion regulation. Different regulation strategies require different levels of understanding of the situation. Thus, one possible approach is to extend the current emotion regulation model to be context-aware, taking into account the specific context of the individual and selecting the most appropriate regulation strategies accordingly.

We are currently working on the specification of several aspects of our model. For example, it is necessary to define the actions to be performed or proposed by the agent to facilitate the application of each emotion regulation technique. Likewise, the participation of the therapist in charge of each person will be necessary to review and personalize the actions according to the specific characteristics of the person, such as culture, social context, cognitive conditions, etc. Simultaneously, we are extending the GenIA³ affective agent architecture [57,58] to support the planning and execution of emotion regulation actions.

The ethical and moral implications surrounding the use of intelligent agents to regulate human emotional states need to establish comprehensive guidelines, standards, and protocols to ensure the safe and responsible utilization of such systems. It is essential to balance the potential benefits of emotion regulation and the ethical challenges that may arise.