# Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

- Is there a trend in multi-robot operators’ behaviors?
- Can we find groups of operators according to their behaviors?
- Is it possible to predict the actions of operators?
- Can we improve multi-robot interfaces by modeling operators?

## 2. Background

#### 2.1. Classification

- K-means (KM) [29]: This is the most common centroid-based clustering algorithm and allows one to classify N observations with D dimensions into K clusters, in such a way that each observation belongs to the cluster with the nearest mean. Although the problem is computationally hard (NP-hard), there are efficient heuristic algorithms to solve it. Most of the proposed K-means algorithms initialize the centroids with a certain criterion and perform two steps iteratively, namely assignment and update. The assignment step involves the classification of observations into clusters defined by their centroids, whereas in the update stage, the clusters’ centroids are recomputed by calculating the mean of the observations within a group. The algorithm converges if there are no significant changes between iterations, but the optimality of the solution is not ensured.
- Self-Organizing Maps (SOM) [30]: This technique is based on an unsupervised neural network that is trained to produce a low-dimensional, discretized representation of the observation space. The algorithm has two modes: training and mapping. Training involves the generation of maps from the inputs by using competitive learning, while mapping refers to the classification of new inputs by means of the obtained maps. Two relevant parameters to be considered are the initialization method and the distance metric for the optimization.
- Partition Around Medoids (PAM) [31]: This is a centroid-based clustering similar to K-means, but it uses medoids instead of mean values as centers for clusters. Medoids are representative points of clusters, so the differences between observations and medoids of clusters are minimal. In contrast to means, medoids must be observations, which is useful when a mean value cannot be defined or does not represent observations. The algorithm is similar to K-means, initializing medoids with a certain pattern and making iterations of assignment and update steps.
- Hierarchical (H) [32]: This technique iteratively builds the clusters by combining similar observations (agglomerative) or separating dissimilar ones (divisive). For this purpose, two parameters are defined: metric and linkage criteria. The metric is required to compute the distances between observations and has an impact on the shapes of clusters, whereas the linkage criterion is used to compute the distances between the observation sets as a function of the distances between observation pairs.

- Calinski–Harabasz (CH) [33]: As shown by Equation (1), this is defined as the division between the overall intercluster and intracluster variances ($S{S}_{B}$ and $S{S}_{A}$), multiplied by a factor that depends on the number of observations, N, and clusters, K. The higher the CH metric, the higher the quality of a set of clusters.
- Davies–Bouldin (DB) [34]: This criterion is described by Equation (2), where K is the number of clusters, $\overline{{d}_{i}}$ is the average distance from every point in the cluster i to its centroid, $\overline{{d}_{j}}$ is the average distance from every point in the cluster j to its centroid, and ${d}_{i,j}$ is the distance between the centroids of both clusters. In contrast to CH, the lower the DB metric, the higher the quality of the clusters.

#### 2.2. Prediction

- Uniform ($U\left(x\right|a,b)$): This is a continuous probability distribution that assigns equal probability to all the values of x between a and b.
- Normal/Gaussian ($N\left(x\right|\mu ,{\sigma}^{2})$): This is a continuous probability distribution that has two parameters: a mean $\mu $ that defines its location and a variance ${\sigma}^{2}$ that defines its scale. It is widely used because many variables in nature and society are distributed normally or can be approximated to a normal distribution.
- Student’s t ($St\left(x\right|\mu ,\lambda ,\nu $): This is a continuous probability distribution that has three parameters: mean $\mu $, precision $\lambda $, and degrees of freedom $\nu $.
- Binomial ($Bin\left(x\right|n,p)$): This is a discrete probability distribution that estimates the number of successes in a set of experiments with two possible outcomes as a function of the number of experiments n and the success probability p.
- Bernoulli ($Bern\left(x\right|p)$): This is a special case of the binomial distribution with a single experiment ($n=1$), where the probability of success is p and the probability of failure is $1-p$.

## 3. Experiments

## 4. Analysis

#### 4.1. Definition of Profiles

**Number of games:**Ratio between the number of games played by a given player and the number of games played by the player who played the most. Therefore, the frequent players will have values close to 1, and the infrequent ones close to 0.**Auto. Rover/Drone:**This defines the robot that is acting autonomously from 0 (the rover is always autonomous) to 1 (the drone is always autonomous).**Drone Random/Strategy:**This indicates whether the drone is moving randomly (0), following a strategy (1), or a balance between these. This latter situation occurs when there are strategy changes during the game or when multiple games are considered.**Drone Spiral/Scan:**This indicates whether the drone is following a spiral path (0), a back and forth path (1), or a balance between them. Again, this latter situation occurs when there are strategy changes during the game or when multiple games are considered.**Rover Random/Strategy:**This indicates whether the rover is moving randomly (0), following a strategy (1), or a balance between these. Again, this latter situation occurs when there are strategy changes during the game or when multiple games are considered.**Rover Left/Right:**This indicates whether the drone is following the left walls (0), right walls (1), or a balance between them. Again, this latter situation occurs when there are strategy changes during the game or when multiple games are considered.**Strategy changes:**This shows the number of strategy changes of a player divided by the maximum number of strategy changes for all players, where 0 means that the player did not change their strategy, and 1 means that this player was the one who changed their strategy the most.**Number of touches:**Ratio between the number of times the user touched the screen and the maximum number of touches for all players, computed in the same way as the strategy changes variable.**Wall crashes:**Ratio between the number of collisions with walls by a player and the maximum number of collisions with walls by all players.**Score:**Ratio between the total time needed by a player to complete the mission and the worst time for completing the mission for all players, which is computed in the same way than the previous variables.

#### 4.2. Classification of Operators

#### 4.3. Prediction of Preferences

#### 4.4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Abbreviations

CH | Calinski–Harabasz |

DARPA | Defense Advanced Research Projects Agency |

DB | Davies–Bouldin |

H | Hierarchical |

KM | K-means |

S | Silhouette |

SOM | Self-organizing maps |

PAM | Partition around medoids |

UAV | Unmanned aerial vehicle |

USAR | Urban search and rescue |

## References

- Wong, C.Y.; Seet, G. Workload, awareness and automation in multiple-robot supervision. Int. J. Adv. Robot. Syst.
**2017**, 14. [Google Scholar] [CrossRef] [Green Version] - Lysaght, R.J.; Hill, S.G.; Dick, A.O.; Plamondon, B.D.; Linton, P.M. Operator Workload: Comprehensive Review and Evaluation of Operator Workload Methodologies (No. TR-2075-3); Analytics Inc.: Willow Grove, PA, USA, 1989. [Google Scholar]
- Moray, N. Mental Workload: Its Theory and Measurement; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013; Volume 8. [Google Scholar]
- Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Adv. Psychol.
**1988**, 52, 139–183. [Google Scholar] - Endsley, M.R. Design and evaluation for situation awareness enhancement. In Proceedings of the Human Factors Society Annual Meeting; SAGE Publications: Los Angeles, CA, USA, 1988; Volume 32, pp. 97–101. [Google Scholar]
- Roldán, J.J.; Peña-Tapia, E.; Martín-Barrio, A.; Olivares-Méndez, M.A.; Del Cerro, J.; Barrientos, A. Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction. Sensors
**2017**, 17, 1720. [Google Scholar] [CrossRef] [PubMed] - Hou, M.; Zhu, H.; Zhou, M.; Arrabito, G.R. Optimizing operator–agent interaction in intelligent adaptive interface design: A conceptual framework. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.)
**2011**, 41, 161–178. [Google Scholar] [CrossRef] - Roldán, J.J.; Olivares-Méndez, M.A.; del Cerro, J.; Barrientos, A. Analyzing and improving multi-robot missions by using process mining. Auton. Robots
**2018**, 42, 1187–1205. [Google Scholar] [CrossRef] - Yanco, H.A.; Norton, A.; Ober, W.; Shane, D.; Skinner, A.; Vice, J. Analysis of human-robot interaction at the darpa robotics challenge trials. J. Field Robot.
**2015**, 32, 420–444. [Google Scholar] [CrossRef] - Cummings, M.L.; Nehme, C.E.; Crandall, J.; Mitchell, P. Predicting operator capacity for supervisory control of multiple UAVs. In Innovations in Intelligent Machines; Springer: Berlin/Heidelberg, Germany, 2007; Volume 1, pp. 11–37. [Google Scholar]
- Hocraffer, A.; Nam, C.S. A meta-analysis of human-system interfaces in unmanned aerial vehicle (UAV) swarm management. Appl. Ergon.
**2017**, 58, 66–80. [Google Scholar] [CrossRef] - Roldán, J.J.; Peña-Tapia, E.; Garcia-Aunon, P.; del Cerro, J.; Barrientos, A. Bringing adaptive & immersive interfaces to real-world multi-robot scenarios: Application to surveillance and intervention in infrastructures. IEEE Access
**2019**. [Google Scholar] [CrossRef] - Loh, C.S.; Sheng, Y.; Ifenthaler, D. Serious Games Analytics: Methodologies for Performance Measurement, Assessment, and Improvement; Springer International Publishing: Cham, Switzerland, 2015. [Google Scholar]
- Van Ruijven, T.W.J. Serious games as experiments for emergency management research: A review. In Proceedings of the 8th International Conference on Information Systems for Crisis Response and Management (ISCRAM 2011), Lisbon, Portugal, 8–11 May 2011. [Google Scholar]
- Wood, R.T.; Griffiths, M.D.; Eatough, V. Online data collection from video game players: Methodological issues. CyberPsychology Behav.
**2004**, 7, 511–518. [Google Scholar] [CrossRef] - Rodriguez-Fernandez, V.; Ramirez-Atencia, C.; Camacho, D. A multi-uav mission planning videogame-based framework for player analysis. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 25–28 May 2015; pp. 1490–1497. [Google Scholar]
- Jerčić, P.; Hagelbäck, J.; Lindley, C. Physiological Affect and Performance in a Collaborative Serious Game Between Humans and an Autonomous Robot. In International Conference on Entertainment Computing; Springer: Cham, Switzerland, 2018; pp. 127–138. [Google Scholar]
- IJgosse, W.M.; van Goor, H.; Luursema, J.M. Saving robots improves laparoscopic performance: Transfer of skills from a serious game to a virtual reality simulator. Surg. Endosc.
**2018**, 32, 3192–3199. [Google Scholar] [CrossRef] - Bailey, K.D. Typologies and Taxonomies: An Introduction to Classification Techniques; Sage: New York, NY, USA, 1994; Volume 102. [Google Scholar]
- Eisen, M.B.; Spellman, P.T.; Brown, P.O.; Botstein, D. Cluster analysis and display of genome-wide expression patterns. Proc. Natl. Acad. Sci. USA
**1988**, 95, 14863–14868. [Google Scholar] [CrossRef] [PubMed] - Rupp, A.A.; Sweet, S.J. Analysis of Multivariate Social Science Data. 2011, pp. 686–693. Available online: https://www.tandfonline.com/doi/abs/10.1080/10705511.2011.607725?journalCode=hsem20 (accessed on 20 May 2019).
- Haldar, P.; Pavord, I.D.; Shaw, D.E.; Berry, M.A.; Thomas, M.; Brightling, C.E.; Green, R.H. Cluster analysis and clinical asthma phenotypes. Am. J. Respir. Crit. Care Med.
**2008**, 178, 218–224. [Google Scholar] [CrossRef] [PubMed] - Bensaid, A.M.; Hall, L.O.; Bezdek, J.C.; Clarke, L.P. Partially supervised clustering for image segmentation. Pattern Recognit.
**1996**, 29, 859–871. [Google Scholar] [CrossRef] - Billieux, J.; Thorens, G.; Khazaal, Y.; Zullino, D.; Achab, S.; Van der Linden, M. Problematic involvement in online games: A cluster analytic approach. Comput. Hum. Behav.
**2015**, 43, 242–250. [Google Scholar] [CrossRef] - Rodríguez-Fernández, V.; Menéndez, H.D.; Camacho, D. A study on performance metrics and clustering methods for analyzing behavior in UAV operations. J. Intell. Fuzzy Syst.
**2017**, 32, 1307–1319. [Google Scholar] [CrossRef] [Green Version] - Rodríguez-Fernández, V.; Menéndez, H.D.; Camacho, D. Automatic profile generation for uav operators using a simulation-based training environment. Prog. Artif. Intell.
**2016**, 5, 37–46. [Google Scholar] [CrossRef] - Rodríguez-Fernández, V.; Menéndez, H.D.; Camacho, D. Analysing temporal performance profiles of UAV operators using time series clustering. Expert Syst. Appl.
**2017**, 70, 103–118. [Google Scholar] [CrossRef] [Green Version] - Fahad, A.; Alshatri, N.; Tari, Z.; Alamri, A.; Khalil, I.; Zomaya, A.Y.; Bouras, A. A survey of clustering algorithms for big data: Taxonomy and empirical analysis. IEEE Trans. Emerg. Top. Comput.
**2014**, 2, 267–279. [Google Scholar] [CrossRef] - Hartigan, J.A.; Wong, M.A. Algorithm AS 136: A k-means clustering algorithm. J. R. Stat. Soc. Ser. C (Appl. Stat.)
**1979**, 28, 100–108. [Google Scholar] [CrossRef] - Kohonen, T. The self-organizing map. Proc. IEEE
**1990**, 78, 1464–1480. [Google Scholar] [CrossRef] - Kaufman, L.; Rousseeuw, P.J. Partitioning around medoids (program pam). Finding Groups in Data: An Introduction to Cluster Analysis; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1990; pp. 68–125. [Google Scholar]
- Blashfield, R.K. Mixture model tests of cluster analysis: Accuracy of four agglomerative hierarchical methods. Psychol. Bull.
**1976**, 83, 377–388. [Google Scholar] [CrossRef] - Calinski, T.; Harabasz, J. A dendrite method for cluster analysis. Commun. Stat.-Theory Methods
**1974**, 3, 1–27. [Google Scholar] [CrossRef] - Davies, D.L.; Bouldin, D.W. A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell.
**1979**, 2, 224–227. [Google Scholar] [CrossRef] - Rousseeuw, P.J. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math.
**1987**, 20, 53–65. [Google Scholar] [CrossRef] [Green Version] - Bishop, C. Pattern Recognition and Machine Learning (Information Science and Statistics); Springer: New York, NY, USA, 2007. [Google Scholar]
- Casper, J.; Murphy, R.R. Human-robot interactions during the robot-assisted urban search and rescue response at the world trade center. IEEE Trans. Syst. Man Cybern. Part B (Cybern.)
**2003**, 33, 367–385. [Google Scholar] [CrossRef] [PubMed] - Peña, D. Análisis de Datos Multivariante; McGraw-Hill: Madrid, Spain, 2002; Volume 24. [Google Scholar]

**Figure 1.**Methodology of the study: (1) operator classification, and (2) operator strategy prediction.

**Figure 2.**Screen captures of USARSIM: (

**a**) main menu, (

**b**) selection of strategy, (

**c**) control of the drone, and (

**d**) control of the rover.

**Table 1.**Standard deviations for the variables of the players (operators) and levels (missions) profiles.

Variables | Players | Levels |
---|---|---|

Number of games | 0.2888 | 0.2547 |

Auto: Rover/Drone | 0.2470 | 0.1316 |

Drone: Random/Strategy | 0.2592 | 0.0396 |

Drone: Spiral/Scan | 0.3520 | 0.1002 |

Rover: Random/Strategy | 0.3206 | 0.1055 |

Rover: Left/Right | 0.3816 | 0.0772 |

Strategy Changes | 0.1161 | 0.0869 |

Number of touches | 0.2332 | 0.1729 |

Wall crashes | 0.2463 | 0.4716 |

Score | 0.2835 | 0.0894 |

Global | 0.2815 | 0.1946 |

Rank | Mean | Linear | Quadratic |
---|---|---|---|

1 | PAM-9 (5.35) | PAM-9 (6.18) | PAM-9 (4.09) |

2 | SOM-8 (4.91) | SOM-9 (4.81) | SOM-8 (3.75) |

3 | KM-9 (3.87) | KM-9 (2.35) | KM-9 (3.56) |

4 | H-9 (2.78) | H-8 (2.04) | H-9 (1.93) |

**Table 3.**Performance of predictions with different profiles (data in rows and methods in columns). The best results are shown in bold. Note: Performance with totally random prediction was 51.84%.

Profile | Mean | Linear | Quadratic | Average |
---|---|---|---|---|

Individuals | 61.47% | 60.12% | 59.18% | 60.26% |

Clusters | 64.08% | 65.77% | 62.49% | 64.11% |

Population | 59.97% | 59.81% | 60.17% | 59,98% |

Average | 61.84% | 61.90% | 60.61% | 61.45% |

**Table 4.**Performance of predictions for operators according to their number of games: individuals (I), clusters (C), population (P), mean (M), linear (L) and quadratic (Q). The best results are shown in bold.

Players | $\mathbf{Games}\le 10$ | $\mathbf{Games}>10$ |
---|---|---|

I-M | 57.70% | 64.45% |

I-L | 56.99% | 62.06% |

I-Q | 56.64% | 62.60% |

C-M | 66.31% | 61.49% |

C-L | 66.55% | 64.84% |

C-Q | 66.71% | 58.57% |

P-M | 60.60% | 59.57% |

P-L | 60.29% | 59.43% |

P-Q | 60.61% | 59.19% |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Roldán, J.J.; Díaz-Maroto, V.; Real, J.; Palafox, P.R.; Valente, J.; Garzón, M.; Barrientos, A.
Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame. *Robotics* **2019**, *8*, 53.
https://doi.org/10.3390/robotics8030053

**AMA Style**

Roldán JJ, Díaz-Maroto V, Real J, Palafox PR, Valente J, Garzón M, Barrientos A.
Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame. *Robotics*. 2019; 8(3):53.
https://doi.org/10.3390/robotics8030053

**Chicago/Turabian Style**

Roldán, Juan Jesús, Víctor Díaz-Maroto, Javier Real, Pablo R. Palafox, João Valente, Mario Garzón, and Antonio Barrientos.
2019. "Press Start to Play: Classifying Multi-Robot Operators and Predicting Their Strategies through a Videogame" *Robotics* 8, no. 3: 53.
https://doi.org/10.3390/robotics8030053