Engagement-Oriented Dynamic Difficulty Adjustment

Abstract

As an emerging and lively research field, game designers are employing Dynamic Difficulty Adjustment (DDA) in Game Artificial Intelligence (Game AI) to improve player experience. Traditional DDA methods focus little on player churn, which cannot always lead to enhanced player engagement. Hence, we propose the Engagement-oriented Dynamic Difficulty Adjustment (EDDA) to meet the urgent need for a highly general and customizable solution in the game industry. EDDA directly considers players’ churn trend to ensure player engagement during gameplay. Its real-time monitoring algorithm and common parameter set are effective in quantifying and preventing player churn. We developed a prototype system integrating seven major game genres to verify the difficulty, gameplay time, and scores of the Game Engagement Questionnaire (GEQ) in multiple dimensions. EDDA has the largest mean and median of all genres in the above metrics with the highest confidence level and effect size, which demonstrates its generality and availability in improving player experience. It is fair to say that EDDA not only provides game designers with targeted player churn monitoring and intervention means, but also offers a deeper level of thinking for the generalized application of DDA and other Game AI technologies.

1. Introduction

With the continuous development of research in traditional Artificial Intelligence (AI) and the rapid development of the game industry, Game AI has emerged [1,2]. The research field of Game AI covers AI within and for games [3]. The game is an interdisciplinary field where science and art merge so that it can provide a unique research environment for AI. Furthermore, games have subjective characteristics that are difficult to regulate, especially their design and experience. Specifically, the complexity and entertainment value of games make them an ideal issue for AI, and the interaction between players and games can further push the boundaries of AI methods [4,5].

Player experience is the lifeblood of the game industry. The experience design has made Dynamic Difficulty Adjustment (DDA) a research hotspot in recent years [6,7,8,9,10,11,12,13]. DDA is a typical technology that optimizes player experience by modifying game parameters automatically. It aims to immerse players during gameplay by adapting game challenges to match player skills. Traditional difficulty curves set by game designers are not suitable for various styles of players, which leads to player churn, a severe negative signal in games [12,13,14,15,16,17]. Therefore, churn monitoring and intervention are of great value for improving player engagement to ensure an optimized experience [18,19]. However, few general DDA methods center on optimizing player engagement. Improving the customizability of algorithms and providing suitable parameter settings for different game genres in a general framework is the key to making the approach effective [3].

Herein, we propose a new method called EDDA, which stands for Engagement-oriented Dynamic Difficulty Adjustment. EDDA directly considers players’ churn trend and adjusts the game difficulty according to gameplay time in each challenge. We set four types of phases and times: sleep, active, unit, and threshold, as well as a common parameter set. The game parameters will be adjusted dynamically via the real-time monitoring of players’ gameplay time in each unit time of the corresponding challenge. In addition, we introduced the fitness based on our previous work and game design experience to measure the appropriateness of difficulty settings to adapt to different players. To evaluate the performance of EDDA and its effect on player experience, we implemented a prototype system and tested it based on the evaluation metrics of participants’ experiences. The results demonstrate that EDDA provides an effective approach to prevent player churn and optimize player experience for game designers.

The main contributions of our work are threefold:

• A usable and available real-time monitoring algorithm is designed to improve player engagement. Game designers can formulate corresponding difficulty adjustment solutions by observing players’ gameplay in different phases and times;
• A general and scalable common parameter set is integrated based on the positive and negative correlation of game parameters and difficulty. It provides an efficient means to intervene in player churn at the player, partner, opponent, and system levels;
• A prototype system is implemented to fully test EDDA. The fitness, gameplay time, and scores of the Game Engagement Questionnaire (GEQ) [20] are core metrics for evaluating game balance, player engagement, and player experience. Significant improvements in the metrics prove the ability of EDDA in improving player experience.

The remainder of the paper is organized as follows: In Section 2, we introduce the flow model, player engagement, and player churn. Section 3 elaborates on formalized definitions of core concepts and the detailed design of EDDA with adaptive parameters for player churn monitoring and intervention. The application effect of EDDA and test results of the evaluation metrics are shown and analyzed in Section 4. Finally, an overall conclusion of our contributions with limitations and future work is drawn in Section 5.

2. Core Concepts

Players constitute the core of games [21,22]. Player experience focuses on the individuals who play the game and their personal experience, which is surveyed during or after gameplay to describe the quality of player interaction [23,24,25]. An outstanding game must provide an excellent experience for players to be recognized by both players and the industry. Typically, this can be achieved by constantly keeping players in the flow state. For any genre of game, realizing the flow model based on DDA is an effective means to improve player engagement and prevent player churn.

2.1. Flow

Flow, which is the mental state in which a person performs an activity with full immersion and enjoyment [26], was proposed by Mihaly Csikszentmihalyi (1975). Due to the natural relationship between challenge and skill, it has been widely referred to across various fields. The primary goal of games is to create entertainment through intrinsic motivation, which is highly correlated with flow. As a result, flow can be refined into a map between player skill and game challenge (shown in Figure 1) [27]. Flow occurs when players face challenges that match their skills during gameplay. Players whose current skill level is certainly below the game challenge level will feel anxious, whereas challenges that are too easy will cause boredom for more skilled players. Either situation may lead to players giving up playing. As a result, the game should balance its challenges and the player’s skill to address and overcome the above situations to keep the player inside the flow channel, thus providing a better experience. As the size of the potential audience grows, designing such a balance will become increasingly difficult for game designers.

2.2. Engagement

Engagement is an essential element of the player experience, which directly determines key metrics such as sales, reviews, and retention [7,28]. It can be explained as a process whereby players engage in pursuing objectives and consequently perform a range of activities to accomplish them while being affected [8]. It is necessary to quantify player engagement, since players will not keep playing without sustained engagement.

In academia, Mayes et al. (2001) propose a construct labeled “engagement” that determines the quality of player experience [29]. They define engagement as how fun, involving, and motivating a task is and introduce the Engagement Questionnaire (EQ), with the dimensions of interest, authenticity, curiosity, involvement, and fidelity. Brockmyer et al. (2009) further propose the GEQ, developed to quantify the subjective experience of deep engagement in immersion, presence, flow, and absorption dimensions [20]. They use engagement as a generic indicator of game involvement and suggest that the four dimensions and dissociation can be conceptualized as representing the progression of deep engagement during gameplay. From then till now, engagement has constituted an important determinant of the impact of playing games.

2.3. Churn

In the game industry, churn generally refers to players leaving the game permanently due to factors such as game mechanics, charging pressure, or personal reasons, etc. [18,19]. That is, reasons such as a too high or low difficulty, more money required than expected, little free time to play, and friends who play together quitting may cause churn without exception. It follows that churn has always been a state that all game practitioners must try to avoid. Monitoring player churn in real time is a crucial task during gameplay, especially to statistically track players’ gameplay time within a fixed period. Besides monitoring, churn intervention is a further concern that aims to optimize player retention by adjusting features in games.

Considering feasibility, DDA is an effective approach to conduct intervention [30]. Game designers introduce DDA to continuously keep players engaged by balancing an accurate level of difficulty. As players may quit at any time, designing DDA algorithms based on gameplay time is the key to monitoring and intervening with players in their personalized experience.

In the flow model, challenges are instantiated by difficulty. DDA centering on optimizing player engagement provides a general solution to reduce player churn, thus enhancing the experience of various players.

3. Engagement-Oriented Dynamic Difficulty Adjustment

Engagement-oriented Dynamic Difficulty Adjustment (EDDA) is a general method to enhance player experience with high customizability. EDDA can be divided into monitoring and intervention processes. We introduce the challenge, phase, and time to partition the experience.

Any game has challenges to some extent. A game without challenges may hardly build effective interaction with players to ensure their experience. Challenges in games typically manifest in terms of difficulty. It is precisely a series of challenges of various difficulties that constitutes the mainstay of the game and concurrently maintains player interest. Tasks, goals, and puzzles are all challenges for players, with combats, competitions, and levels serving as carriers for challenges. Each challenge players face in the game is a crucial factor influencing their gameplay. Based on this, we divide the game by challenges. Multiple challenges form sleep and active phases, while each challenge consists of sleep and active times. The difficulty will be adjusted by selecting the appropriate parameters in the designed common set.

3.1. Monitoring

It is particularly important to ensure the real-time and differentiated monitoring of player engagement. We introduced a supervisory subsystem to track each player’s entire game life cycle in real time. The difference indicates whether monitoring is enabled. EDDA utilizes a lightweight design, as the subsystem only initializes monitoring when the player has passed early game. Players may have certain trials at each challenge. Typically, players will continue during the start of the game and each challenge with sufficient motivation. It is inappropriate to judge player churn during this.

The game is divided into a sleep phase and an active phase based on challenges, while the former (corresponding to the start of the game) does not monitor but the latter (the remaining game) does. The same applies to sleep time (corresponding to the first certain trials at a challenge) and active time (the remaining trials at the challenge). Among the four phases and times, the sleep, active, and threshold ones are integer multiples of unit one. The gameplay time of each unit time for players is the core monitoring metric. If the threshold metric under the corresponding condition decreases, the difficulty will be adjusted in real time. All parameters related to churn are defined as shown in

Table 1. Formalized definition of churn parameters.

Symbol	Parameter	Description
$C$	Challenge	The stage player faces difficulties during gameplay.
$P^S$	Sleep phase	The former part of the game that does not monitor churn.
$P^A$	Active phase	The latter part of the game that monitors churn in real time.
$P^U$	Unit phase	The minimum number of challenges that can be monitored.
$P^T$	Threshold phase	The number of unit phases that can be judged to be churn.
$T^S$	Sleep time	The former part of a challenge that does not monitor churn.
$T^A$	Active time	The latter part of a challenge that monitors churn in real time.
$T^U$	Unit time	The minimum time that can be monitored.
$T^T$	Threshold time	The number of unit times that can be judged to be churn.
$M$	Threshold metric	The metric that can be judged to be churn.
$F$	Fitness	The appropriateness of difficulty settings to adapt to different players.
$G$	Gameplay time	The gameplay time for the player.
$N^T$	Number of times	The number of unit times in a challenge.
$N^C$	Number of challenges	The number of challenges in the game.
$N^R$	Number of trials	The number of trials for the player during a challenge.

.

In Table 1, G^U and G^R represent G in a unit time and a trial, respectively. M is equal to G^U, as shown in (1) when the T^T of it continuously decreases within T^A in the same C concerning the churn caused by high difficulty.

$$M=G^U$$

(1)

For the churn caused by low difficulty, M should be measured by the mean of G^U in P^U of Cs, with i (from 1 to P^U), j (from 1 to $N_i^T$), and k (from 1 to P^U) as counters in (2) for the condition of P^T of it continuously decreasing within P^A in different Cs. $N_i^T$ refers to the number of unit times in the i-th C.

$$M={\displaystyle \frac{\sum _{i=1}^{P^U}\sum _{j=1}^{N_i^T}{G}_{ij}^U}{\sum _{k=1}^{P^U}{N}_k^T}}$$

(2)

3.2. Intervention

After monitoring player churn, it is crucial to adopt targeted DDA schemes based on the causes of churn. We design a parameter set that offers general adjustable DDA parameters to intervene in churn at the game and designer levels. In addition, F is proposed to measure the appropriateness of difficulty settings with and without DDA introduced.

3.2.1. Parameter Set

Players, partners, opponents, systems, scenes, and other neutral actors are all important game elements that affect player engagement. Through systematic research of all major game genres in the game industry, the more common parameters of each genre are selected. A common adjustable parameter set is constructed based on the correlation between parameters and difficulty as well as their belonging, as shown in

Table 2. Common adjustable parameter set.

Correlation	Classification	Parameter
Positive	Partner-fixed	Cooldown.
	Opponent-fixed	Number, attack, defense, health point, magic point, stamina, speed, adaptive capacity, perception range.
	Opponent-random	Luck, critical chance, hit chance, dodge chance, beneficial attribute rate.
	Neutral	Gameplay pace, difficulty seed, adverse time.
Negative	Player-random	Luck, critical chance, hit chance, dodge chance.
	Partner-fixed	Number, attack, defense, health point, magic point, stamina, speed, adaptive capacity.
	Partner-random	Luck, critical chance, hit chance, dodge chance.
	Opponent-fixed	Cooldown.
	Opponent-random	Adverse attribute rate.
	Neutral	Hint number, beneficial time.

.

The parameters in Table 2 are classified into six types: player-random, partner-fixed, partner-random, opponent-fixed, opponent-random, and neutral. It should be noted that player-fixed attributes are generally visible to players. Changing them directly will break the concealment of DDA, thereby reducing the player experience. A positive correlation means that the difficulty will increase as the parameter increases, while a negative correlation means the opposite. All parameters in the set are adjustable, but not all of them exist or are suitable for adjustment in practice. Game designers may adaptively choose appropriate parameters to implement player churn intervention on account of high or low difficulty.

Take the Role-playing Game (RPG) and Racing Game (RAC) for instance, each combat and race can be considered as a C, correspondingly. In the RPG, partners generally refer to the teammates the player meets during gameplay, while enemies in combat are typical representatives of opponents. If churn caused by high difficulty is monitored during a C, the attack, defense, health point, magic point, stamina, speed, adaptive capacity, critical chance, hit chance, and dodge chance of enemies may be decreased while increasing those of teammates. Meanwhile, it is possible to slightly increase the player’s critical, hit, and dodge to ensure concealment. Conversely, the difficulty of the current C should be increased when monitoring the churn across Cs. As for RAC, racers on the same or other team as the player are partners and opponents in races. In addition to speed, luck and cooldown are key parameters applicable to item and speed races to adjust difficulty, respectively. Game designers may improve the probability of racers in the player’s team obtaining items that benefit them to reduce difficulty. Lower intervals at which other teams can use acceleration may be set to enhance race intensity.

3.2.2. Fitness

As a new difficulty evaluation metric, F is calculated via the mean and maximum value of G^R, as shown in (3), which is equivalent to the sum of the sub-fitness of each C in the game. The counters m from 1 to N^C and n from 1 to $N_m^R$ (the number of trials for the player during the m-th challenge) are introduced.

$$F=\sum _{m=1}^{N^C}\left(1-{\displaystyle \frac{\sum _{n=1}^{N_m^R}{G}_{mn}^R}{N_m^R·\max \left\{G_{m1}^R,G_{m2}^R,\cdots ,G_{mn}^R\right\}}}\right)$$

(3)

A game of appropriate difficulty needs to adapt Cs to players of different skill levels, which explains why DDA is needed. When introducing F, G^R of each C needs to be considered. The greater the relative difference between the mean and the maximum value of G^R, the higher the controllability provided to players. That is, the difficulty setting is more appropriate and corresponds to a higher F. In addition, the sum of the sub-fitness is selected instead of the mean, aiming to amplify the differences in comparison. The comparison of F within the same game is valid, while the value of F is directly related to N^C in the game. The sensitivity of F in different genres of games to outliers varies due to the restrictive time of each C. A longer C may cause F to be more sensitive, and so does an excessively low N^C. Therefore, game designers should precisely set and divide Cs to reduce the impact of outliers.

EDDA is proposed around challenges, phases, and times with a lightweight design. The real-time monitoring algorithm, common parameter set, and fitness can ensure the entire process of monitoring and intervention of player churn. Its high adaptability allows game designers to adequately apply it across various game genres.

4. Results and Discussion

In the game industry, the primary game classification method is to classify according to its mode. As a result, the RPG, RAC, Fighting Game (FTG), Shooting Game (STG), Adventure Game (AVG), Strategy Game (SG), and Casual Game (CG) are the seven major genres widely researched and recognized by the whole industry. Based on this, a prototype system containing the above seven genres is implemented for the gameplay testing of players. Meanwhile, we introduce fitness, gameplay time, and GEQ scores as core metrics for evaluating game balance, player engagement, and player experience to verify the application effect of EDDA. Each item in the GEQ has “No”, “Maybe”, and “Yes”, three options that correspond to 0, 1, and 2 scores, respectively.

In the design of the ablation experiment (baseline (without DDA), EDDA-s (with EDDA’s monitoring module and without EDDA’s intervention module), and EDDA), three groups are selected for comparison since the intervention module cannot exist independently from the monitoring one. For EDDA-s, it adopts the same real-time monitoring algorithm as EDDA. However, it achieves DDA by directly adjusting the AI’s overall skill level instead of specific parameters in the common parameter set as EDDA. We invited 100 participants (50 males and 50 females) aged between 19 and 29 (M = 22.96 and SD = 2.72) to test the prototype system for this study. Anonymity is assured in the study. We declare that the experiment was conducted with voluntary participation and fully informed participants. All participants should experience the seven games with the baseline, EDDA-s, and EDDA. Each genre’s gameplay for each group lasted for two days, for a total of six weeks. Only after the experience of one group was finished would the next group continue. A demographic questionnaire was distributed to each participant before gameplay, while participants were to finish a GEQ corresponding to the group after two weeks of gameplay. Both games and groups were ordered by counter-balancing using random Latin squares, respectively.

Six weeks later, we collected 100 log data and GEQ samples of the 100 players. Cronbach’s alpha [31] shows strong reliability for the GEQ (α = 0.81). The players’ gameplay time in each genre was extracted from their log data to analyze and calculate fitness. The mean core metrics for the seven major genres of all players for each group are shown in Figure 2.

By validating the methods through Friedman tests [32], significant differences are found for the gameplay time (χ²(2) = 14, p < 0.001) and GEQ scores (χ²(2) = 14, p < 0.001), while for the fitness they are not (χ²(2) = 5.43, p > 0.05). As shown in

Table 3. The metrics of post hoc pairwise comparison for baseline, EDDA-s, and EDDA.

Metric	Group	M	Me	SD	z 1	p 1	δ 2
Fitness	Baseline	2.08	1.99	0.38	−1.2677	0.2049	−0.3061 †
	EDDA-s	2.36	2.40	0.57	−2.1129	0.0346 *	−0.2653 †
	EDDA	2.73	2.50	0.79	−2.1129	0.0346 *	0.4286 ‡
Gameplay time	Baseline	181.15	164.34	68.23	−2.2819	0.0225 *	−0.2653 †
	EDDA-s	214.66	190.15	71.22	−2.2819	0.0225 *	−0.2653 †
	EDDA	236.55	201.92	80.32	−2.2819	0.0225 *	0.3878 ‡
GEQ scores	Baseline	16.41	19.14	5.66	−2.2819	0.0225 *	−0.2245 †
	EDDA-s	18.78	20.89	5.81	−2.2819	0.0225 *	−0.2653 †
	EDDA	21.20	23.47	5.89	−2.2819	0.0225 *	0.5510 §

¹ The z and p values of each group are those of the Wilcoxon signed-rank test between itself and its next group. The order of groups is baseline, EDDA-s, and EDDA (loop). ² The δ of each group is calculated with itself as the first group and its next group as the second one. The order of groups is baseline, EDDA-s, and EDDA (loop). * The p value of fitness is less than 0.05, and those of gameplay time and GEQ scores fit the correction via the Holm–Bonferroni method. ^† The absolute value of δ is greater than or equal to 0.147 and less than 0.330, which indicates a small effect size. ^‡ The absolute value of δ is greater than or equal to 0.330 and less than 0.474, which indicates a medium effect size. ^§ The absolute value of δ is greater than or equal to 0.474, which indicates a large effect size.

, Wilcoxon signed-rank tests [33] based on post hoc pairwise comparison with the Holm–Bonferroni method [34] show that each paired group exhibited significant differences in gameplay time and GEQ scores, including EDDA versus EDDA-s, EDDA versus baseline, and EDDA-s versus baseline. The comparison of medians as well as means demonstrates the effectiveness of EDDA’s monitoring and intervention modules in enhancing gameplay time and GEQ scores. By contrast, fitness only performs the Wilcoxon signed-rank test. The results prove that the intervention module can improve difficulty fitness. Moreover, we utilized Cliff’s delta (δ) [35] to further measure the differences in the paired groups. The monitoring and intervention modules, respectively, show a small effect size from the results, while the two modules combined have larger ones. Thus, the significant differences in all the metrics between EDDA versus baseline are verified on all sides.

Via the systematic analysis of the experimental results, we can conclude that EDDA’s real-time monitoring algorithm and common parameter set improve player experience and engagement significantly, and that the parameter set can further ensure game balance to a certain extent. The effect sizes demonstrate that combining the algorithm and set has greater practical significance than their separate use.

Furthermore, the AVG has the highest absolute increment of all core metrics of EDDA compared to the baseline among the seven major genres in Figure 2, while the FTG, STG, and RAC have the lowest ones in fitness, gameplay time, and GEQ scores, respectively. The relative increments of the AVG in fitness, CG in gameplay time, and SG in GEQ scores are the highest, while the lowest genres are consistent with the absolute increment. It follows that EDDA is more applicable to the AVG than others. Visually, the improvements for EDDA-s and EDDA in gameplay time and GEQ scores appear consistent across genres. It is noteworthy that fitness is decreased when EDDA-s and EDDA are implemented in the FTG. The reason for this lies in the fixed shorter round duration limit of the FTG. The maximum value of gameplay time in each trial of EDDA-s and EDDA does not vary much from that of the baseline, whereas the higher mean leads to a slightly lower fitness. In terms of the descriptive statistic measures in Table 3, EDDA-s and EDDA distinctly improve the central tendency of the metrics while reducing statistical dispersion to a certain degree. Considering the overall results, EDDA’s advantages far outweigh its flaws.

5. Conclusions

In this paper, we propose EDDA, a highly general, usable, scalable, and available Engagement-oriented Dynamic Difficulty Adjustment method. As a novel DDA solution, it aims to improve player experience by adjusting the corresponding stage’s difficulty within the common parameter set based on observing players’ gameplay per unit time in real time.

EDDA is designed to center on player churn to adjust the game difficulty according to players’ gameplay time. It introduces challenge, phase, and time in sleep, active, unit, and threshold dimensions. Furthermore, a common adjustable parameter set is constructed at the player, partner, opponent, and system levels based on the positive and negative correlations of game parameters and difficulty. The experiment with the prototype system demonstrates that EDDA has significantly improved the metrics of fitness, gameplay time, and GEQ scores compared with those without DDA. The results further prove the effectiveness of our proposed method in improving player experience, player engagement, and game balance. Based on this, game designers can keep track of players’ situations during gameplay at any time and implement targeted solutions to prevent player churn while ensuring the optimal experience for players.

It should be noted that we only verified the capability among relatively young players aged between 19 and 29 due to the restrictions of the long testing time. The applicability of EDDA to players of other ages should be proven in subsequent work. In addition, although the two-day gameplay test for each genre of each group with a total of six weeks based on random Latin squares avoids potential learning or fatigue effects to a certain extent, its effectiveness for the longer testing of a single game genre needs to be further confirmed. Anyway, gameplay testing should be carried out in conditions of players with sufficient time to minimize the sensitivity of gameplay time to their strategy changes.

EDDA is highly adaptable for game content design that can be simply divided into multiple challenges, while additional algorithmic design is required for games that are difficult to modularize. When implementing EDDA, game designers may find that some established methods based on Machine Learning (ML) perform better than it in specific game genres. However, EDDA generally performs better across genres since it effectively reduces the learning costs for related personnel in design, which offers a convenient way to quickly develop highly customizable prototype systems. In reality, the method has the potential for broad applications, expanding beyond the game industry. In the future, we will further optimize the applicable stability of EDDA and explore its availability in education and training.