Next Article in Journal
Low-Cost Hardware in the Loop for Intelligent Neural Predictive Control of Hybrid Electric Vehicle
Previous Article in Journal
Image Steganalysis of Low Embedding Rate Based on the Attention Mechanism and Transfer Learning
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Electronics
Volume 12
Issue 4
10.3390/electronics12040970
electronics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Novel Approach for Contextual Clustering and Retrieval of Behavior Trees to Enrich the Behavior of Social Intelligent Agents

by
Mona Jamjoom
1,
Nada Ahmed
1,*,
Safia Abbas
2,
Rania Hodhod
3,
Mohamed El-Sheikh
4 and
Zahid Ullah
5
1
Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh 11671, Saudi Arabia
2
Department of Computer Science, Faculty of Computer and Information Sciences, Ain Shams University, Cairo 11566, Egypt
3
TSYS School of Computer Science, Turner College of Business, Columbus State University, Columbus, GA 31907, USA
4
Basic Science Department, Cairo University, Cairo 12613, Egypt
5
Information Systems Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
*
Author to whom correspondence should be addressed.
Electronics 2023, 12(4), 970; https://doi.org/10.3390/electronics12040970
Submission received: 29 December 2022 / Revised: 10 February 2023 / Accepted: 12 February 2023 / Published: 15 February 2023
(This article belongs to the Section Computer Science & Engineering)
Browse Figures
Versions Notes

Abstract

:
Recently, many works have been carried out to find effective ways that can allow for plausibly effective interactions of social intelligent agents (SIAs) in unpredictable environments in a reasonable time. Behavior trees (BTs) allow for knowledge to be modeled as a graph representation and provide a way for SIAs to effectively interact with the received information. BTs can store past social experiences that can then be used by SIAs to provide adequate human-like interactions when facing new social situations (query). One challenge appears when a social agent with vast past experiences—represented as a forest of BTs—tries to retrieve a similar BT to learn from in order to provide plausible interactions in the current situation in a cost-effective manner. Cognitive scripts with their inherent temporal structure can address this challenge where they can facilitate the use of contextual retrieval techniques on BTs represented as cognitive scripts. This paper introduces novel hybrid retrieval techniques that use agglomerative hierarchical clustering (H-clustering) and similarity-based algorithms: map-and-reduce and least common parent (LCP) to effectively retrieve similar BTs to a specific query BT in a reasonable time. The model groups BTs, represented as cognitive scripts, into compact clusters that can then be used to retrieve the most similar BT to a query one in real time without noticeable delay. A comparison was done between the performance of the proposed hybrid-retrieval techniques using a semi-structured dataset of cognitive scripts. The results showed that H-clustering-map-and-reduce is more cost-effective than H-clustering-LCP as it allowed for a low average retrieval time of 8 × 10−3 s compared to 3.1 s, respectively.

1. Introduction

Modeling human behaviors is essential in real-life surroundings such as smart homes, smart traffic controllers, assistive technologies [1], and applications that require human-like interactions, such as intellectual advisers, expert systems/tutoring agents [2,3,4,5,6,7], robotic assistants [8,9,10], emotional non-player characters [11], augmented video games [3,12,13], and social agents [14].
Social intelligent agents (SIAs) are computational cognitive agents that have epistemic abilities besides the traditional ones in terms of action and communication [15]; they tend to exhibit other forms of aptitudes including tracking an agent’s mental state transition and teaching an agent to make decisions guided by social norms. SIAs can use knowledge modeled by graph structures such as finite state machines (FSM) or behavior trees (BTs) to provide human-like interactions with the received information. A big challenge is to get these models to evolve over time by learning from other graph structures that are not only from the same domain but also across different domains. AI agents that modeled their knowledge using FSM show poor scaling, reusing issues, and a lack of extendibility and adaptivity, whereas the hierarchal tree structure with the inherent state transition logic in BTs shows more reliable modularity that is flexible for the retrieval, reuse, and extendibility of knowledge [16]. This allows BTs to be an effective way to model social situations.
Despite the successful usage of BTs in AI games, nonplayer characters (NPCs), and some robotics applications [16], a challenge appears when the BTs do not adhere to certain vocabulary or include nodes that are marked as traversal points. This challenge resulted from the focus on a set of general actions associated with a specific set of events to adapt or predict the next action in most AI applications while ignoring the general or abstract view of the surrounding scene and past events or actions that had been taken by the same agent in different or similar situations. In rich environments, social agents can have vast past experiences represented as a forest of BTs, in this case, the retrieval of a similar BT to a query BT becomes of high cost and may not be as accurate as needed because of the lack of contextual information in BTs. One way to address this challenge can be through using cognitive scripts and a novel contextual retrieval technique.
Cognitive scripts are one form of memory structures that enjoy certain continuous learning abilities through their ability to evolve over time through multiple exposures to similar stimuli/behaviors in different social situations [17]. They enjoy temporal relation between subsequent events which helps with contextual retrieval of similar cognitive scripts to a query script. Accordingly, representing BTs as evolving cognitive scripts can allow for cross-domain learning by SIAs provided their ability to contextually retrieve similar scripts to enrich their experiences over time and accordingly, come up with plausible behaviors in different social situations. Despite the flexibility of cognitive scripts, the retrieval process of similar scripts from a large repository of scripts can be extensively costly in terms of time [14], which can cause latency in the responses taken by social agents.
This paper introduces two novel hybrid retrieving techniques that can be used to retrieve the most similar cognitive script to a query script in a cost-effective manner. The core of both techniques is hierarchical clustering (H-clustering) which aims to group the cognitive scripts into compact clusters. Contextual retrieval techniques (map-and-reduce and LCP) are then used to find the cluster with similar scripts and retrieve the most similar one to a query script. A comparison between the performance of the proposed techniques is presented. The results showed that both techniques are effective in retrieving the most adequate cognitive script to a query script, whereas the H-clustering-map-and-reduce shows a lower cost than H-clustering-LCP in terms of retrieval time.
The paper is organized as follows: Section 2 summarizes the literature review, Section 3 shows the transformation of BTs to the semi-structured representation of cognitive scripts (CogScrs), Section 4 provides the implementation of the retrieval techniques in detail, Section 5 presents the experimental results, and lastly, the conclusion is provided in Section 6.

2. Literature Review

Behavior trees (BTs) have drawn the attention of researchers over the years because of their capability in modeling human behavior/processes and their reusability. In [18,19], the authors presented optimization and refining approaches for modeling BTs with multiple branches that simulate human behaviors gathered by different sensors and can be used for further learning processes. In the past decade, a variety of techniques were used to automatically generate/evolve BTs or improve existing parameters within pre-defined BTs including reinforcement learning, evolution-inspired learning, and case-based reasoning. Despite the exerted effort of their work, none of these techniques were used to contextually retrieve BTs based on their similarity to a query BT in order to learn from its entailed actions and events, instead they focused on learning action by action in each step at a specific event.
In [20], an existing BT is re-ordered based on Q-values that measure the overall expected reward computed using the lowest-level sequences as actions in the reinforcement learning (RL) algorithm. In [21], the RL takes place in the Fallback nodes by executing the child with the highest Q-value. Similar ideas were built upon in [22] by combining the hierarchical RL approach MAXQ with BTs. Starting from a manual design where some Fallback nodes are marked to be improved by learning. Having found the Q-values of each child, new condition nodes were created to allow the execution of each child when that child is the best option. This line of ideas was also explored in [23] and discussed at a more general level by combining RL and BTs in [24,25]. Individual actions or conditions can also be endowed with RL capabilities, a concept that was explored in [26,27]. The latter specified learning nodes in the BT where an RL problem in terms of states, actions, and rewards was defined. Finally, an approach involving the translation of a learned RL policy into a BT was suggested by [28]. One of the main drawbacks of Q-learning-based techniques is that the simulation time for a complex game can be intractable, and the time taken to produce accurate Q-values is exponential to the state-action space of the agent [20].
BTs seem to be very well suited for evolutionary algorithms; it has been shown that locality, in terms of small changes in BTs design giving small changes in performance, is important for the performance of evolutionary algorithms [29]. Genetic programming is one form of an evolutionary algorithm that has been applied to BTs. A number of studies have presented solutions that outperform manual designs [16,30,31,32]. These methods have been used to automatically evolve new behaviors using high-level definitions given a new environment [33,34] and additional improvements have been made to increase scalability for evolving algorithms [35]. A combination of grammatical evolution with BTs and path-finding algorithms such as A* was explored in [36]. Air Combat simulators are highly dynamic environments where human behavior is often represented by rule-based scripts. Grammatical evolution has been used to create adaptive human behavior models [37] and studies have been done to improve pilot decision support during reconnaissance missions [38]. The researchers reported that it was not possible to evaluate the performance based on robustness due to the limited number of simulations. Although optimization allowed the algorithm to perform well, it caused fewer possibilities to find a desired solution to the problem [37].
Other military-related training simulators included ground unit mobility and training on how to act in dangerous situations. To reduce the need for military experts to explain to programmers how virtual soldiers should behave in a particular situation, a genetic algorithm was used to automatically generate such agents [39]. In multi-agent environments, it is often difficult to specify policies for every single agent and thus the same hand-written BTs have been used for many units. Using an evolutionary approach, individual BTs were developed for each agent, resulting in a more natural and efficient behavior of the whole team [40], but the downside of such automated techniques is the reduced designer control [41]. Another example of a multi-agent system is a robotic swarm. In this field, BTs were generated using both grammatical evolution [42] and genetic programming. In [43], the evolutionary system was based on a distributed island model. Some of the solutions generated by evolutionary algorithms can be quite large and complex. Further methods were described to address this problem but with limited success [16,44].
Another technique that was used to handle BTs is case-based reasoning (CBR). CBR is an automated decision-making process where solutions to new problems are provided through experiences from problems that were previously encountered. Following this process, rules can be constructed that may solve future problems by retrieving and reusing stored behaviors [45,46,47]. One drawback of this method is the difficulty of refining existing strategies that have been previously learned from experience [45,46], but it has been shown that some additional adaptability can be provided using RL [48].
In [49,50], authors depend on a blackboard observer decoder (BOD) that contains a collection of pairs (key, bod) called an observer’s list which is updated whenever a bod starts or stops observing the key. The execution starts by ticking a root node, and this ticking process traverses downwards through the EDBT’s nodes. This process is carried out depending on the ticking rules associated with the type of each node that is ticked. Accordingly, the events and actions are pre-determined through the blackboard, then the EDBT is used to determine the behavior step-by-step like ticking the behavior “walk” that returns either failure or success. Despite that this traditional method is suitable for manipulating NPCs’ behavior, it might not be suitable for smart autonomous agents such as robots. Since the recognition and retrieving processes for the most similar scene for the existing one and ticking the most similar BT are not considered. This retrieval process would lead to better performance especially for autonomous smart agents such as robots, because it mimics human thinking, adds more adaptive behavior, and minimizes the time taken for choosing suitable actions based on the blackboard contents.

3. From Behavior Tree to Cognitive Script

A behavior tree (BT) can be defined as a < V, E, T > tuple, where V is the set of vertices, E is the set of directed edges, and T is the root node. Similarly, cognitive scripts (CogScrs) can be defined as a < N, E, ID > tuple, where N is the set of nodes with stored events, E is the set of directed edges connecting the nodes, and ID is the set of IDs for each node in the script. The ID list allows for the selection of intermediate nodes which can also act as the start node for a new sequence of behaviors that would aid in the creation of new hybrid behaviors from pre-existing behaviors. In both BTs and CogScrs, the graph depicts a hierarchical representation where the root has no parent and the leaves have no children. The execution of BTs and CogScrs is similar in functionalities of ticking and retrieving, for more detail about the retrieval and execution of CogScrs, see [14]. An example of a behavior tree that was converted to a cognitive script is shown in Figure 1. The provided structure of cognitive script allows for improved learning and adaptive interactive processes between the SIAs and the surrounding environment.
Figure 1 shows the constructed nodes as predicates in the form of relation (parameter1, parameter2,…) and branches (directed edges that link a set of event nodes to each other) in the multi-branched stage_coach cognitive script; node 1101 can have subsequent event nodes 1102 or 1103. Each event node is represented in a predicate form “IID-verb (object, object)”, where “IID” is the index number of that node in the structured tables as shown in Table 1. All scripts are organized and stored in a semi-structured repository as illustrated in [14].
Each multi-branched cognitive script is stored in the database as a structured table that defines all parent nodes and their child nodes as shown in Table 1. For example, arrive(driver, boarding location) is mentioned twice as a parent node in Table 1 because it has two child nodes: wait(driver) and enter(passenger, stagecoach).

4. Contextual Clustering and Retrieval Approach

Contextual information retrieval based on map-and-reduce from web pages has attracted numerous researchers who seek an optimal retrieval process that can achieve high accuracy with low retrieval time [51,52]. This section explains the integration of hierarchical clustering with the map-and-reduce technique to contextually cluster cognitive scripts and retrieve the most similar ones (base scripts) to a new one (target script) that is presented by the user.

4.1. Hierarchical Clustering Based on Map-and-Reduce

The contextual clustering process is done before the retrieval process in order to organize cognitive scripts stored in a repository of scripts into a set of clusters. Each cluster has a specific vector of words as its significant centroid. The centroids are used in computing the similarity measure between the clusters and the target script in the contextual retrieving process. In this work, agglomerative hierarchical clustering was used to group cognitive scripts into distinct clusters. The hierarchical clustering process works from the bottom to the top based on similarity measures; the algorithm uses a modified version of Ward’s minimum variance method to measure the compactness of clusters by maximizing the total similarity within a cluster instead of minimizing the total distance within a cluster. Therefore, at each iteration, the two clusters with the largest similarity are merged, which ensures the compactness of the two clusters before merging them. This process continues until all the scripts are grouped in one cluster. At last, the average similarity between the clusters in each level is calculated and the level with the maximum similarity would act as the cut-off point for the desired level of clusters in the produced hierarchy as shown in Figure 2.
The main role of the map-and-reduce technique in the hierarchical clustering process is to transform each cognitive script into one vector of words associated with term frequencies (TF) as shown in Figure 3. The centroid of a cluster is then defined by the significant words in all scripts inside this cluster. Centroids are then used to find out the similarity measure between a new cognitive script (target) and each of the clusters to aid in the contextual retrieval process. The cluster with the highest similarity measure is then recognized as the chosen cluster.

4.2. The Retrieval Process for Cognitive Scripts Using H-Clustering-Map-and-Reduce

The contextual retrieval process between scripts has the following steps:
  • The user enters the (target) script that is analyzed and tokenized into initial vector form.
  • This vector is sent to the first phase in the retrieval process where the initial vector is compared with all the centroids.
  • The cluster with the most similar centroid is picked, and all the scripts in this cluster are sent back to the tokenization process to be tokenized into a set of tokens.
  • The produced tokens form the bag-of-words for each script, which are then mapped by the Mapper into a set of words with a unique index for each word.
  • The reducer groups the repeated indices into one term associated with its frequency value.
  • Finally, term frequency (TF) similarity is used to retrieve the most similar cognitive script from the chosen cluster.
A diagram that shows the flow of the retrieval process is shown in Figure 4 and the pseudo-code for the retrieval process is presented in Figure 5.

5. Experimental Results

For the purpose of this study, we adopted the corpus of social scenarios from [53]. The corpus has 131 nodes and more than 262 words. Table 2 shows the scripts’ names, the number of nodes, and the number of words in each script. All scripts were pre-processed and stored in a predicate form, see [14] for more details.
In order to provide a baseline to evaluate the performance of the proposed techniques in a plausible way, fifteen human participants were recruited via email invitations to participate in a study where they were asked to choose the most similar script to a query script. In this study, each script in the CogScr repository was used as a target script once. The results from this study is summarized in Table 3. Details of the primary study using a subset of the cognitive scripts’ repository can be found in [17].
As seen in Table 3, each script in the CogScr repository was used as a target script once, the results for the first nine scripts are obtained from [17]; however, another study for the rest of the scripts was conducted where fifteen human participants were asked to choose the most similar script to a query script. A major voting process was followed to decide on the most similar retrieved script if different scripts were chosen by the participants. The final results of similar scripts are presented in Table 3.
As stated earlier, the human participants’ choices were used as the baseline to evaluate the performance/plausibility of the proposed retrieval techniques. Both techniques, as can be seen in Table 4, have proven to be valid methods as they effectively retrieve contextually similar cognitive scripts to a query script from a set of clusters. However, the H-clustering-map-and-reduce has shown precedence over H-clustering-LCP where 60% of the retrieved scripts matched the human participants’ first choice compared to 40% for H-LCP. The time taken to load all the cognitive scripts from the database and the formation of clusters is about 2.5 s, this time can be ignored if the clusters are formed offline and are already in the working memory. The H-clustering-map-and-reduce recorded an average of 1 × 10−2 s to find the best cluster (the cluster that contains scripts that are of high similarity to the query script) for a query cognitive script, while the H-clustering-LCP recorded an average of 1 × 10−3 s to find the best cluster. On the other hand, the average time taken to retrieve the most similar script from the chosen cluster is 8 × 10−3 s compared to 3.1 s for map-and-reduce and LCP, respectively. Overall, H-clustering-map-and-reduce recorded a very low retrieval time that would allow SIAs to respond to events happening in the surrounding environment in a seamless way with no obvious latency.
None of the works in the literature review addressed the retrieval time of multi-branched BTs based on certain criteria or similarity basis. However, working with BTs, in general, has posed some challenges, such as the exponential time when re-ordering BTs based on Q-values [20,21,22,23,24,25,26,27,28], high cost of 3.6 million seconds when developing AI-bots in games using evolving behavior trees on sequential computers [29,30,31], and slow response during execution when building a case-based planner by learning and refining existing BTs [46,47,48,49]. Accordingly, the proposed technique in this paper can be seen as an advancement over existing works when it comes to time efficiency when working with multi-branched BTs.
Another positive aspect that can be attributed to the proposed technique is that it can augment some of the previous works. For example, the work done in [18] and [19] focused on building BTs by using information received from sensors to model daily life activities with no work or consideration for retrieving similar behaviors and/or prediction of next actions to create further rich related experiences. Others considered simulating human behaviors in dangerous situations for training purposes [38,39,40,41,42,43,44] without the consideration of making use of similar experiences. The proposed technique can help these works retrieve similar behavior/scenarios that can then be merged together to provide richer behaviors/scenarios in a timely-efficient manner.

6. Conclusions

AI social agents rely on representations like BTs to define their actions and behaviors in complex environments. The representation of BTs as cognitive scripts provides a way for BTs to evolve by learning from other BTs which accordingly enriches the agent’s experiences and learning abilities. Earlier works have provided novel techniques to retrieve contextually similar scripts to a query script by computing the similarity of the query script to each script in the repository. With the increase in the number and size of the scripts, the retrieval time increases exponentially. This work presents a novel hybrid retrieval process that uses agglomerative hierarchical clustering to group the cognitive scripts into compact clusters with the centroid of each cluster being used as the label of the cluster. This allows a query script to be compared to the clusters’ centroids only rather than individual scripts which reduces the time complexity tremendously. Once the cluster with similar scripts is identified, the similarity measure is computed between the query script and the member scripts in this cluster only. Two methods were used to retrieve the most similar script from the chosen cluster, map-and-reduce, and LCP methods. The results show that map-and-reduce is more cost-efficient than LCP with respect to time complexity; the average time to find the most similar script in an identified cluster is 8 × 10−3 s and 2.7 s for map-and-reduce and LCP, respectively. It is worth mentioning that the average time taken to form the clusters is 2.5 s, but this time doesn’t affect the achieved results as the clusters are formed offline and can be updated frequently when new scripts are available. The results show that the combination of agglomerative hierarchical clustering and the map-and-reduce method provides a plausible cost-efficient way to retrieve contextually similar BTs represented as cognitive scripts from a large-size repository.

Author Contributions

Conceptualization, M.E.-S., R.H., and S.A.; methodology, M.E.-S., R.H., and S.A.; software, M.E.-S. and R.H.; validation, S.A., M.J., and N.A.; formal analysis, R.H., N.A., and Z.U.; investigation, M.J., Z.U., and R.H.; resources, R.H., S.A., and N.A.; data curation, R.H. and S.A.; writing—original draft preparation, R.H., M.J., S.A., Z.U., and N.A.; writing—review and editing, M.J., S.A., and R.H.; visualization, N.A. and Z.U.; supervision, Z.U. and R.H.; project administration, M.J. and R.H.; funding acquisition, M.J., N.A., and Z.U. All authors have read and agreed to the published version of the manuscript.

Funding

Princess Nourah bint Abdulrahman University Researchers Supporting Project Number (PNURSP2023R104), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data analyzed during this study are publicly available for use from references [14,53].

Acknowledgments

The authors acknowledge Princess Nourah bint Abdulrahman University Researchers Supporting Project number (PNURSP2023R104), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Tamine-Lechani, L.; Boughanem, M.; Daoud, M. Evaluation of contextual information retrieval effectiveness: Overview of issues and research. Knowl. Inf. Syst. 2010, 24, 1–34. [Google Scholar] [CrossRef]
  2. Ullah, Z.; Jamjoom, M. An intelligent approach for Arabic handwritten letter recognition using convolutional neural network. PeerJ Comput. Sci. 2022, 8, e995. [Google Scholar] [CrossRef] [PubMed]
  3. Azarnov, D.A.; Chubarov, A.A.; Samsonovich, A.V. Virtual actor with social-emotional intelligence. Procedia Comput. Sci. 2018, 123, 76–85. [Google Scholar] [CrossRef]
  4. Ullah, Z.; Lajis, A.; Jamjoom, M.; Altalhi, A.; Saleem, F. Bloom’s taxonomy: A beneficial tool for learning and assessing students’ competency levels in computer programming using empirical analysis. Comput. Appl. Eng. Educ. 2020, 28, 1628–1640. [Google Scholar] [CrossRef]
  5. Ullah, Z.; Saleem, F.; Jamjoom, M.; Fakieh, B. Reliable prediction models based on enriched data for identifying the mode of childbirth by using machine learning methods: Development study. J. Med. Internet Res. 2021, 23, e28856. [Google Scholar] [CrossRef] [PubMed]
  6. Jamjoom, M.; Alabdulkreem, E.; Hadjouni, M.; Karim, F.; Qarh, M. Early Prediction for At-Risk Students in an Introductory Programming Course Based on Student Self-Efficacy. Informatica 2021, 45, 6. [Google Scholar] [CrossRef]
  7. Ullah, Z.; Jamjoom, M. A deep learning for Alzheimer’s stages detection using brain images. Comput. Mater. Contin. 2022, 74, 1457–1473. [Google Scholar] [CrossRef]
  8. Chella, A.; Lanza, F.; Seidita, V. Decision process in human-agent interaction: Extending JASON reasoning cycle. In International Workshop on Engineering Multi-Agent Systems; Springer: Cham, Switzerland, 2018; pp. 320–339. [Google Scholar]
  9. Pilato, G.; Vassallo, G.; Augello, A.; Vasile, M.; Gaglio, S. Expert chat-bots for cultural heritage. Intell. Artif. 2005, 2, 25–31. [Google Scholar]
  10. Augello, A.; Pilato, G.; Gaglio, S. Intelligent advisor agents in distributed environments. In Information Retrieval and Mining in Distributed Environments. Studies in Computational Intelligence; Soro, A., Vargiu, E., Armano, G., Paddeu, G., Eds.; Springer: Berlin, Germany, 2011; Volume 324, pp. 109–124. [Google Scholar]
  11. Tikhomirova, D.V.; Chubarov, A.A.; Samsonovich, A.V. Empirical and modeling study of emotional state dynamics in social videogame paradigms. Cogn. Syst. Res. 2020, 60, 44–56. [Google Scholar] [CrossRef]
  12. Augello, A.; Pilato, G.; Gaglio, S. A conversational agent to support decisions in SimCity like games. Proc. IEEE ICSC 2009, 2009, 367–372. [Google Scholar]
  13. Monceaux, J.; Becker, J.; Boudier, C.; Mazel, A. Demonstration—First Steps in Emotional Expression of the Humanoid Robot Nao. In Proceedings of the ICMI-MLMI’09, Cambridge, MA, USA, 2–4 November 2009; pp. 235–236, ACM 978-1-60558-772-1/09/11. [Google Scholar]
  14. Abbas, S.; Hodhod, R.; El-Sheikh, M. Retrieval of behavior trees using map-and-reduce technique. Egypt. Inform. J. 2022, 23, 55–64. [Google Scholar] [CrossRef]
  15. Allahbakhsh, M.; Ignjatovic, A.; Benatallah, B.; Foo, N.; Bertino, E. Representation and querying of unfair evaluations in social rating systems. Comput. Secur. 2014, 41, 68–88. [Google Scholar] [CrossRef]
  16. Colledanchise, M.; Parasuraman, R.; Ögren, P. Learning of behavior trees for autonomous agents. IEEE Trans. Games 2019, 11, 183–189. [Google Scholar] [CrossRef]
  17. Hodhod, R.; Magerko, B.; Gawish, M. Pharaoh: Context-Based Structural Retrieval of Cognitive Scripts. Int. J. Inf. Retr. Res. 2013, 2, 58–71. [Google Scholar] [CrossRef]
  18. Bouchard, B.; Bouchard, K.; Gaboury, S.; Francillette, Y. Modeling Human Activities Using Behaviour Trees in Smart Homes. In Proceedings of the PETRA ’18: The 11th PErvasive Technologies Related to Assistive Environments Conference, Corfu, Greece, 26–29 June 2018; pp. 67–74. [Google Scholar]
  19. Francillette, Y.; Boucher, E.; Bier, N.; Lussier, M.; Bouchard, K.; Belchior, P.; Gaboury, S. Modeling the behavior of persons with mild cognitive impairment or Alzheimer’s for intelligent environment simulation. User Model User Adapt. Interact. 2020, 30, 895–947. [Google Scholar] [CrossRef]
  20. Dey, R.; Child, C. QL-BT: Enhancing behavior tree design and implementation with Q-learning. In Proceedings of the IEEE Conference on Computational Intelligence in Games (CIG), Niagara Falls, ON, Canada, 11–13 August 2013; IEEE: Niagara Falls, ON, Canada, 2013; pp. 1–8. [Google Scholar] [CrossRef] [Green Version]
  21. Fu, Y.; Qin, L.; Yin, Q. A reinforcement learning behavior tree framework for game AI. In Proceedings of the 2016 International Conference on Economics, Social Science, Arts, Education and Management Engineering, Huhhot, China, 30–31 July 2016; Atlantis Press: Huhhot, China, 2016. [Google Scholar] [CrossRef]
  22. Zhang, Q.; Sun, L.; Jiao, P.; Yin, Q. Combining behavior trees with MAXQ learning to facilitate CGFs behavior modeling. In Proceedings of the 4th International Conference on Systems and Informatics (ICSAI), Hangzhou, China, 11–13 November 2017; pp. 525–531. [Google Scholar] [CrossRef]
  23. Zhu, X. Behavior tree design of intelligent behavior of non-player character (NPC) based on Unity3D. J. Intell. Fuzzy Syst. 2019, 37, 6071–6079. [Google Scholar] [CrossRef]
  24. Hölzl, M.; Gabor, T. Reasoning and learning for awareness and adaptation. Software Engineering for Collective Autonomic Systems: The ASCENS Approach. In Lecture Notes in Computer Science; Wirsing, M., Hölzl, M., Koch, N., Mayer, P., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 249–290. [Google Scholar] [CrossRef]
  25. Hölzl, M.; Gabor, T. Continuous collaboration: A case study on the development of an adaptive cyber-physical system. In Proceedings of the IEEE/ACM 1st International Workshop on Software Engineering for Smart CyberPhysical Systems, Florence, Italy, 17 May 2015; pp. 19–25. [Google Scholar] [CrossRef]
  26. Kartasev, M. Integrating Reinforcement Learning into Behavior Trees by Hierarchical Composition. Ph.D. Thesis, KTH Royal Institute of Technology, Stockholm, Sweden, 2019. [Google Scholar]
  27. Pereira, R.d.P.; Engel, M.P. A framework for constrained and adaptive behavior-based agents. arXiv 2015, arXiv:1506.02312. [Google Scholar]
  28. Banerjee, B. Autonomous acquisition of behavior trees for robot control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3460–3467. [Google Scholar] [CrossRef]
  29. Rothlauf, F.; Oetzel, M. On the locality of grammatical evolution. In European Conference on Genetic Programming; Springer: Berlin/Heidelberg, Germany, 2006; pp. 320–330. [Google Scholar]
  30. Lim, C.-U.; Baumgarten, R.; Colton, S. Evolving behaviour trees for the commercial game DEFCON. In Applications of Evolutionary Computation; Springer: Berlin/Heidelberg, Germany, 2010; Volume 6024, pp. 100–110. [Google Scholar] [CrossRef]
  31. Perez, D.; Nicolau, M.; O’Neill, M.; Brabazon, A. Evolving behavior trees for the Mario AI competition using grammatical evolution. Applications of Evolutionary Computation. In Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; pp. 123–132. [Google Scholar]
  32. Hoff, J.W.; Christensen, J.H. Evolving Behaviour Trees: Automatic Generation of AI Opponents for Real-Time Strategy Games. Ph.D. Thesis, NTNU, Tronheim, Norway, 2016. [Google Scholar]
  33. Estgren, M.; Jansson, E.S.V. Behaviour Tree Evolution by Genetic Programming; Linköping University: Linkoping, Sweden, 2017. [Google Scholar]
  34. Paduraru, C.; Paduraru, M. Automatic Difficulty Management and Testing in Games Using a Framework Based on Behavior Trees and Genetic Algorithms. arXiv 2019, arXiv:1909.04368. [Google Scholar]
  35. Zhang, Q.; Xu, K.; Jiao, P.; Yin, Q. Behavior modeling for autonomous agents based on modified evolving behavior trees. In Proceedings of the IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS), Enshi, China, 25–27 May 2018; pp. 1140–1145. [Google Scholar] [CrossRef]
  36. Nicolau, M.; Perez-Liebana, D.; O’Neill, M.; Brabazon, A. Evolutionary behavior tree approaches for navigating platform games. IEEE Trans Comput. Intell. AI Games 2017, 9, 227–238. [Google Scholar] [CrossRef] [Green Version]
  37. Yao, J.; Huang, Q.; Wang, W. Adaptive human behavior modeling for air combat simulation. In Proceedings of the IEEE/ACM 19th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), Chengdu, China, 14–16 October 2015; pp. 100–103. [Google Scholar] [CrossRef]
  38. Eilert, P. Learning Behaviour Trees for Simulated Fighter Pilots in Airborne Reconnaissance Missions: A Grammatical Evolution Approach. Ph.D. Thesis, Linköping University, Linkoping, Sweden, 2019. [Google Scholar]
  39. Berthling-Hansen, G.; Morch, E.; Løvlid, A.R.; Gundersen, E.O. Automating behaviour tree generation for simulating troop movements (poster). In Proceedings of the IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA), Boston, MA, USA, 11–14 June 2018; pp. 147–153. [Google Scholar] [CrossRef]
  40. Neupane, A. Emergence of Collective Behaviors in hub-Based Colonies Using Grammatical Evolution and Behavior Trees. Ph.D. Thesis, Brigham Young University, Prover, UT, USA, 2019. [Google Scholar]
  41. Schwab, P.; Hlavacs, H. Capturing the essence: Towards the automated generation of transparent behavior models. In Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference, Santa Cruz, CA, USA, 14–18 November 2015. [Google Scholar] [CrossRef]
  42. Neupane, A.; Goodrich, M. Designing emergent swarm behaviors using behavior trees and grammatical evolution. In Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, Montreal, QC, Canada, 13–17 May 2019; Volume 4, pp. 2138–2140. [Google Scholar]
  43. Jones, S.; Studley, M.; Hauert, S.; Winfield, A. Evolving Behaviour Trees for Swarm Robotics. In Distributed Autonomous Robotic Systems. Springer Proceedings in Advanced Robotics; Springer: Cham, Switzerland, 2018; Volume 6. [Google Scholar] [CrossRef]
  44. Hallawa, A.; Schug, S.; Iacca, G.; Ascheid, G. Evolving instinctive behaviour in resource-constrained autonomous agents using grammatical evolution. Applications of Evolutionary Computation. In Lecture Notes in Computer Science; Castillo, A.P., Jiménez Laredo, J.L., Fernández de Vega, F., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 369–383. [Google Scholar] [CrossRef]
  45. Palma, R.; González-Calero, P.A.; Gómez-Martín, M.A.; Gómez-Martín, P.P. Extending case-based planning with behavior trees. In Proceedings of the Twenty-Fourth International FLAIRS Conference, Palm Beach, FL, USA, 18–20 May 2011. [Google Scholar]
  46. Palma, R.; Sánchez-Ruiz, A.A.; Gómez-Martín, M.A.; Gómez-Martín, P.P.; González-Calero, P.A. Combining expert knowledge and learning from demonstration in real-time strategy games. In Case-Based Reasoning Research and Development; Lecture Notes in Computer Science; Ram, A., Wiratunga, N., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 181–195. [Google Scholar]
  47. Flórez-Puga, G.; Gómez-Martín, M.; Díaz-Agudo, B.; González-Calero, P.A. Dynamic expansion of behaviour trees. In Proceedings of the Fourth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment; AIIDE’08. AAAI Press: Stanford, CA, USA, 2008; pp. 36–41. [Google Scholar]
  48. Zhang, Q.; Yin, Q.; Xu, K. Towards an integrated learning framework for behavior modeling of adaptive CGFs. In Proceedings of the 9th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 10–11 December 2016; Volume 2, pp. 7–12. [Google Scholar] [CrossRef]
  49. Agis, R.A.; Gottifredi, S.; García, A.J. An event-driven behavior trees extension to facilitate non-player multi-agent coordination in video games. Expert Syst. Appl. 2020, 155, 113457. [Google Scholar] [CrossRef]
  50. Iovino, M.; Scukins, E.; Styrud, J.; Ögren, P.; Smith, C. A survey of Behavior Trees in robotics and AI. Robot. Auton. Syst. 2022, 154, 104096. [Google Scholar] [CrossRef]
  51. Liao, J.; Zhao, Y.; Long, S. A parallel algorithm adapted for mining big data. In Proceedings of the IEEE Workshop on Electronics, Computer and Applications, Ottawa, ON, Canada, 8–9 May 2014; pp. 564–568. [Google Scholar]
  52. Hashem, I.A.T.; Chang, V.; Anuar, B.N.; Adewole, K.; Yaqoob, I.; Gani, A.; Chiroma, H. The role of big data in smart city. Int. J. Inf. Manag. 2016, 36, 748–758. [Google Scholar] [CrossRef] [Green Version]
  53. Permar, J.; Magerko, B. A Conceptual Blending Approach to the Generation of Cognitive Scripts for Interactive Narrative. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Virtual, 11–15 October 2021; Volume 9, pp. 44–50. [Google Scholar] [CrossRef]
Electronics 12 00970 g001 550
Figure 1. Cognitive script for stagecoach behavior.
Figure 1. Cognitive script for stagecoach behavior.
Electronics 12 00970 g001
Electronics 12 00970 g002 550
Figure 2. Hierarchical clustering algorithm and cut-off point.
Figure 2. Hierarchical clustering algorithm and cut-off point.
Electronics 12 00970 g002
Electronics 12 00970 g003 550
Figure 3. Hierarchical clustering algorithm based on map-and-reduce.
Figure 3. Hierarchical clustering algorithm based on map-and-reduce.
Electronics 12 00970 g003
Electronics 12 00970 g004 550
Figure 4. H-clustering-map-and-reduce retrieval process.
Figure 4. H-clustering-map-and-reduce retrieval process.
Electronics 12 00970 g004
Electronics 12 00970 g005 550
Figure 5. Contextual retrieval algorithm based on H-clustering-map-and-reduce technique.
Figure 5. Contextual retrieval algorithm based on H-clustering-map-and-reduce technique.
Electronics 12 00970 g005
Table
Table 1. The predicate form of the stagecoach cognitive script.
Table 1. The predicate form of the stagecoach cognitive script.
Parent NodeChild Node
IIDFormIIDForm
1101arrive(driver, boarding location)1102wait(driver)
1101arrive(driver, boarding location)1103enter(passenger, stagecoach)
1102wait(driver)1103enter(passenger, stagecoach)
1103enter(passenger, stagecoach)1104return(passenger, stagecoach)
1103enter(passenger, stagecoach)1105tell(passenger, driver, white house)
1104return(passenger, stagecoach)1105tell(passenger, driver, white house)
1105tell(passenger, driver, white house)1106depart(driver, passenger)
1106depart(driver, passenger)1107converse(driver, passenger)
1106depart(driver, passenger)1108arrival_notice(driver, passenger)
1107converse(driver, passenger)1108arrival_notice(driver, passenger)
1108arrival_notice(driver, passenger)1109arrive(driver, passenger, white house)
Table
Table 2. The stored script details.
Table 2. The stored script details.
IDScriptCodeNodesWords
1car-buyingS-11634
2classicalS-21732
3concertS-32038
4cinemaS-41320
5stadiumS-51525
6pharmacyS-61324
7restaurantS-72144
8apprehendingS-859
9order-a-drinkS-91017
10stagecoachS-10919
11tour-busS-11821
12cat-ownerS-121324
13cattle-driveS-131121
14derrick-roseS-141015
15bullfightS-15814
Table
Table 3. Most similar scripts from human perspective.
Table 3. Most similar scripts from human perspective.
Query Script (Target)Human Participants (Retrieved Script)
apprehendingNone
car_buyingPharmacy
CinemaStadium
classicalConcert
concertClassical
order_a_drinkRestaurant
pharmacyorder_a_drink
restaurant order_a_drink
stadiumCinema
stagecoachtour-bus
tour-busStagecoach
cat-ownercattle-drive
cattle-drivecat-owner
derrick-roseBullfight
Bullfightderrick-rose
Table
Table 4. Retrieved scripts based on H-Map-Reduce and H-LCP.
Table 4. Retrieved scripts based on H-Map-Reduce and H-LCP.
Query ScriptH-Map and ReduceH-LCP
Retrieved ScriptRetrieving TimeRetrieved ScriptRetrieving TIME
apprehendingcar_buying0.00700car_buying3.36573
car_buyingOrder_a_drink0.00800PHARMACY *3.33244
CinemaSTADIUM *0.00500STADIUM *3.41039
ClassicalCONCERT *0.00700CONCERT *3.59187
ConcertCLASSICAL *0.00746CLASSICAL *3.50024
Order_a_drinkRESTAURANT *0.00800PHARMACY3.33244
PharmacyRESTAURANT0.00632RESTAURANT3.21454
RestaurantORDER_a_DRINK *0.00711PHARMACY3.57359
StadiumCONCERT0.00600CONCERT3.37495
stage_coachcinema0.00703CAR_BUYING3.17081
tour_busCAR_BUYING0.00500CAR_BUYING3.16260
cat_ownerCattle_drive *0.00400CATTLE_DRIVE *1.61438
cattle_driveCAT_OWNER *0.00300CAT_OWNER *1.57649
derrick_roseBULL_FIGHT *0.02100STADIUM3.37459
BullfightDERRICK_ROSE *0.02100STADIUM3.37459
Average Retrieval Time 0.00819 3.13131
* denotes that the retrieved script matches the human participants’ best choice.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Jamjoom, M.; Ahmed, N.; Abbas, S.; Hodhod, R.; El-Sheikh, M.; Ullah, Z. A Novel Approach for Contextual Clustering and Retrieval of Behavior Trees to Enrich the Behavior of Social Intelligent Agents. Electronics 2023, 12, 970. https://doi.org/10.3390/electronics12040970

AMA Style

Jamjoom M, Ahmed N, Abbas S, Hodhod R, El-Sheikh M, Ullah Z. A Novel Approach for Contextual Clustering and Retrieval of Behavior Trees to Enrich the Behavior of Social Intelligent Agents. Electronics. 2023; 12(4):970. https://doi.org/10.3390/electronics12040970

Chicago/Turabian Style

Jamjoom, Mona, Nada Ahmed, Safia Abbas, Rania Hodhod, Mohamed El-Sheikh, and Zahid Ullah. 2023. "A Novel Approach for Contextual Clustering and Retrieval of Behavior Trees to Enrich the Behavior of Social Intelligent Agents" Electronics 12, no. 4: 970. https://doi.org/10.3390/electronics12040970

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop