# The Right to Remember: Implementing a Rudimentary Emotive-Effect Layer for Frustration on AI Agent Gameplay Strategy

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Overview of the Love Letter Card Game

- Eliminating CardsCards that can be used, under certain conditions, to eliminate other players from the round.
- Information-gathering CardsCards that can be used to get information about the card the opponent might hold.
- Protection CardsCards that are used by players to prevent them being a legal target for other players’ card effects.
- Compelling CardsCards which serve no other function than that they compel an action of the player in the event that they are required to be discarded.

#### 2.1. Developing a Default Strategy for Play

#### 2.2. Phase-I Testing

^{6}games were played to test a number of initial hypotheses. It was reasoned, firstly, that the likelihood of a draw or tie between the two agents was so remote that it would be statistically insignificant. Secondly, assuming this to be true, it was thus reasoned that as it was possible to eliminate one opponent in the first round or, that by going first, the second player was to have access to more information for their first turn that there may be some advantage to one player or another dependent on the turn order. Our hypotheses are, thus, stated more formally below:

**Hypothesis 1**(H1).

**Hypothesis 2**(H2).

**Hypothesis 3**(H3).

^{6}games only 101 games were identified to have been a draw. The probability of a draw using the standard play strategy was then calculated using a Poisson distribution as P(D, X = 1) = 0.0001. At one-percent of one-percent we concluded that the results supported the expectations of H1.

^{6}games concluded with 537,128 player one wins to 462,771 player one losses. Using Chi-squared to compare with the reasoned expected outcomes being close to one win for one loss, X

^{2}(1, N = 10

^{6}) = 512, the resultant p-value of less than 0.00001 confirmed that this trend was statistically significant and reflected an accurate likelihood of performance based on turn order. On the basis of these results we were able to confirm H2 and reject H3 suggesting that, under ordinary circumstances, the standard play strategy yielded a probability of approximately 54% likelihood that the first player would win the round or game.

## 3. Modelling Frustration for Agent Manipulation

#### 3.1. Frustration as It Has Been Observed in Human Players

- Instinctual factors as the basis of an initial subset of data feed with other data sources into an emotive response. It should be noted that these factors can run the gambit between simple fact gathering within the game-world, the state of play, environmental factors, etc., and predetermined interpretations of values as artificial biological stimuli.
- This emotive response, which is the positioning of the Emotive Effect Layer, in turn manipulates this data so as to condition a desired behavioural effect in the state-machine.
- This behavioural effect is, itself, in the form of an emotive product, a description of an exhibited behaviour. This is achieved through use of the determination index, as described previously, and the selection of a subset of modifying coefficients.

#### 3.2. Implementing Frustration Modifiers through the Emotive Effect Layer

- (1)
- The agent, as part of the normal default strategy, first calculates the probability assessments for their own hand, the opponent’s hand in light of previous plays, and performs an analysis of what their play should be in light of this information.
- In order to perform these calculations the agent has knowledge of how many of each specific card exist within the game deck, how many of that card have already been played, as well as prior information gained from information-seeking cards.
- The probability calculations can be split into two activities. Raw calculations which occur first from knowledge of the game, and override calculations which come from turn-by-turn acquired knowledge. Thus a base probability is first calculated, and then adjusted where appropriate.
- The base, or raw, calculations are subject to alteration by associated coefficients. These coefficients represent gameplay reaction values, such as tilting effects or successive loss effects. Each of these coefficients are represented as value from 0 to 1, with 1 being the initial state and are subject to alteration. Thus this represents the agent’s ability to have their reasoning altered by gameplay events, the base probabilities becoming either more or less accurate as the case may be.

- (2)
- The agent performs their turn action.
- (3)
- Steps 1 and 2 are repeated until game end.
- (4)
- There is a game evaluation analysis.
- This analysis reviews the state of the match in terms of win/loss for the agent, the length of the game by number of terms, the trend of expected behaviour as seen in frustration profiles taken from human player observations.
- This analysis is then used to calculate agent investment and valence spike values which are, themselves, values between 0 and 1, as per the expectations of behavioural shifts seen in the real world. In effect, these are normalised values taken from data taken from the frustration, gameplay, and motivation studies cited in this paper.
- The shift of investment and valence spikes are then compared to their previous values, or 0 in the event of a first calculation.
- The determination index value is calculated as a product of these shifts in the same way as a multi-variable chain differential as used to calculate error gradients in back propagation techniques.

- (5)
- The newly-determination index value points towards a specific subset of probability affecting coefficients or behaviours, as described in Step 1, which are fed as new replacement values for those probability coefficients or describe some game action, such as ending the match before a full ten games have been completed.

## 4. Phase-II Testing

- Agent investment should steadily increase in value during match play IF the win-loss ratio for the agent remains close to one [6]. Other negative consequence effects during this eventuality should have lesser impact, as well.
- Agent investment should increase rapidly IF the game concludes with a loss, but took greater than 10 turns to complete.
- Agent investment should decrease rapidly IF the win-loss ratio for the agent reaches towards a critical point for match play.
- Agent investment should decrease rapidly IF the game concludes with a loss and in fewer than 3 turns.
- The agent’s ability to calculate probability should reduce IF the agent suffers a number of losses in quick succession, comparable with effects of “tilting” [15].
- The agent will end the match IF the agent investment value is sufficiently low (<0.15) and the win-loss ratio suggests that match play is reaching a critical point.

^{5}matches.

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Khashman, A. A Modified Backpropagation Learning Algorithm with added Emotional Coefficients. IEEE Trans. Neural Netw.
**2008**, 19, 1896–1909. [Google Scholar] [CrossRef] [PubMed] - Clark, L.; Crooks, B.; Clarke, R.; Aitken, M.; Dunn, B.D. Physiological responses to near-miss outcomes and personal control during simulated gambling. J. Gambl. Stud.
**2012**, 28, 123–137. [Google Scholar] [CrossRef] [PubMed] - Larche, C.J.; Musielak, N.; Dixon, M.J. The Candy Crush Sweet Tooth: How ‘Near-misses’ in Candy Crush Increase Frustration, and the Urge to Continue Gameplay. J. Gambl. Stud.
**2016**, 1–17. [Google Scholar] [CrossRef] [PubMed] - Palomaki, J.; Laakasuo, M.; Salmela, M. Losing More by Losing It: Poker Experience, Sensitivity to Losses and Tilting Severity. J. Gambl. Stud.
**2014**, 30, 187–200. [Google Scholar] [CrossRef] [PubMed] - Gilleade, K.M.; Dix, A. Using frustration in the design of adaptive videogames. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Singapore, 3–5 June 2005; ACM: New York, NY, USA, 2005; pp. 228–232. [Google Scholar]
- Hazlett, R.L. Measuring Emotional Valence during Interactive Experiences: Boys at Video Game Play. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Montréal, QC, Canada, 22–27 April 2006; ACM: New York, NY, USA, 2006; pp. 1023–1026. [Google Scholar]
- Zacharatos, H.; Gatzoulis, C.; Chrysanthou, Y.; Aristidou, A. Emotion Recognition for Exergames using Laban Movement Analysis. In Proceedings of the Motion in Games, Dublin, Ireland, 6–8 November 2013; ACM: New York, NY, USA, 2009; pp. 61–66. [Google Scholar]
- Boehner, K.; DePaula, R.; Dourish, P.; Sengers, P. How emotion is made and measured. Int. J. Hum. Comput. Stud.
**2007**, 65, 275–291. [Google Scholar] [CrossRef] - Canossa, A.; Badler, J.; El-Nasr, M.S.; Anderson, E. Eliciting Emotions in Design of Games—A Theory Driven Approach. In Proceedings of the 4th Workshop on Emotions and Personality in Personalized Systems (EMPIRE) 2016, Boston, MA, USA, 16 September 2016; ACM: New York, NY, USA, 2016; pp. 34–42. [Google Scholar]
- Bowman, N.D.; Weber, R.; Tamborini, R.; Sherry, J. Facilitating Game Play: How Others Affect Performance at and Enjoyment of Video Games. Media Psychol.
**2013**, 16, 39–64. [Google Scholar] [CrossRef] - Canossa, A.; Drachen, A.; Sørensen, J.R.M. Arrrgghh!!!: Blending quantitative and qualitative methods to detect player frustration. In Proceedings of the 6th International Conference on Foundations of Digital Games, Bordeaux, France, 29 June–1 July 2011; ACM: New York, NY, USA, 2011; pp. 61–68. [Google Scholar]
- Ferguson, C.J.; Olson, C.K. Friends, fun, frustration and fantasy: Child motivations for video game play. Motiv. Emot.
**2013**, 37, 154–164. [Google Scholar] [CrossRef] - Yun, C.; Shastri, D.; Pavlidis, I.; Deng, Z. O’ Game, Can You Feel My Frustration?: Improving User’s Gaming Experiences via StressCam. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; ACM: New York, NY, USA, 2009; pp. 2195–2204. [Google Scholar]
- Bakkes, S.C.J.; Spronck, P.H.M.; van den Herik, H.J. Opponent modelling for case-based adaptive game AI. Entertain. Comput.
**2009**, 1, 27–37. [Google Scholar] [CrossRef] - Mandryk, R.L.; Atkins, M.S. A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum. Comput. Stud.
**2007**, 65, 329–347. [Google Scholar] [CrossRef] - Hazlett, R.L.; Benedek, J. Measuring emotional valence to understand the user’s experience of software. Int. J. Hum. Comput. Stud.
**2007**, 65, 306–314. [Google Scholar] [CrossRef] - Lindquist, K.A.; Wager, T.D.; Kober, H.; Bliss-Moreau, E.; Barrett, L.F. The brain basis of emotion: A meta-analytic review. Behav. Brain Sci.
**2012**, 35, 121–143. [Google Scholar] [CrossRef] [PubMed] - Przybylski, A.K.; Rigby, C.S.; Ryan, R.M. A Motivational Model of Video Game Engagement. Rev. Gen. Psychol.
**2010**, 14, 154–166. [Google Scholar] [CrossRef] - Van den Hoogen, W.; Poels, K.; Ijsselsteijn, W.; de Kort, Y. Between Challenge and Defeat: Repeated Player-Death and Game Enjoyment. Media Psychol.
**2012**, 15, 443–459. [Google Scholar] [CrossRef] - Alexander, J.T.; Sear, J.; Oikonomou, A. An investigation of the effects of game difficulty on player enjoyment. Entertain. Comput.
**2013**, 4, 53–62. [Google Scholar] [CrossRef] - Bakkes, S.C.J.; Spronck, P.H.M.; van Lankveld, G. Player behavioural modelling for video games. Entertain. Comput.
**2012**, 3, 71–79. [Google Scholar] [CrossRef]

Losses | 72,497 | |
---|---|---|

Win-Loss Ratio (% of total matches) | Complete Victory (6:0) | 0.00 |

6.00 (6:1) | 0.00 | |

3.00 (6:2) | <0.01 | |

2.00 (6:3) | 3.57 | |

1.50 (6:4) | 9.25 | |

1.00 (5:5) | 14.68 | |

0.67 (4:6) | 16.24 | |

0.50 (3:6) | 42.67 | |

0.33 (2:6) | 9.38 | |

0.17 (1:6) | 4.21 | |

0.00 (0:6) | 0.00 | |

Initial Agent Investment (each match) | 0.5000 | |

Final Agent Investment (mean) | 0.2219 | |

Mean Agent Investment Shift (per game) | −0.0984 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Stallwood, J.; Ranchhod, A.
The Right to Remember: Implementing a Rudimentary Emotive-Effect Layer for Frustration on AI Agent Gameplay Strategy. *Computers* **2017**, *6*, 18.
https://doi.org/10.3390/computers6020018

**AMA Style**

Stallwood J, Ranchhod A.
The Right to Remember: Implementing a Rudimentary Emotive-Effect Layer for Frustration on AI Agent Gameplay Strategy. *Computers*. 2017; 6(2):18.
https://doi.org/10.3390/computers6020018

**Chicago/Turabian Style**

Stallwood, James, and Ashok Ranchhod.
2017. "The Right to Remember: Implementing a Rudimentary Emotive-Effect Layer for Frustration on AI Agent Gameplay Strategy" *Computers* 6, no. 2: 18.
https://doi.org/10.3390/computers6020018