Algorithms

Research

Jump to: Review

17 pages, 1380 KiB

Open AccessArticle

Assessing Environmental Influences on Flash Storage for Vehicle Computing: A Quantitative and Analytical Investigation

by Ying He, Donger Chen, Isabella Xu, Wang Feng, Qing Yang, Ted Tsao and Song Fu

Algorithms 2025, 18(5), 269; https://doi.org/10.3390/a18050269 - 5 May 2025

Viewed by 164

Abstract

As automotive technology advances, ensuring efficient and reliable vehicle storage systems becomes increasingly important to vehicle edges. Environmental factors, like extreme cold or intense heat, can greatly affect how well these critical components function. In this paper, we study the effects of different [...] Read more.

As automotive technology advances, ensuring efficient and reliable vehicle storage systems becomes increasingly important to vehicle edges. Environmental factors, like extreme cold or intense heat, can greatly affect how well these critical components function. In this paper, we study the effects of different temperatures on flash-based vehicle storage systems, especially how these conditions impact data storage workloads, machine learning workloads, and vehicle edge computing by analyzing the read and write performance of car flash memory. Our approach combines environmental simulations, performance testing, and data analysis to examine how temperature changes affect the performance and reliability of vehicle storage. By testing conditions from standard room temperatures to extreme heat, this study explores how such environments influence the speed, dependability, and overall functionality of flash memory in automotive systems. The results show detailed relationships between temperature changes and the speed (throughput) and delay (latency) in flash storage, identifying areas where these systems may be vulnerable or where improvements could be made. Understanding these dynamics is essential for improving the durability and flexibility of automotive storage systems and vehicle edges in various environmental conditions. We have refined our conclusions to note that while our findings provide insights into temperature-related performance shifts, they represent one piece of a broader set of design considerations for engineers and manufacturers. Rather than offering definitive guidance for policymakers, our findings primarily help illustrate potential thermal vulnerabilities, informing ongoing work toward more robust and reliable vehicle storage systems. As the automotive industry continues to innovate, this study offers an initial foundation for future developments in vehicle storage technology. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

21 pages, 3748 KiB

Open AccessArticle

Machine Learning for Decision Support and Automation in Games: A Study on Vehicle Optimal Path

by Gonçalo Penelas, Luís Barbosa, Arsénio Reis, João Barroso and Tiago Pinto

Algorithms 2025, 18(2), 106; https://doi.org/10.3390/a18020106 - 15 Feb 2025

Viewed by 689

Abstract

In the field of gaming artificial intelligence, selecting the appropriate machine learning approach is essential for improving decision-making and automation. This paper examines the effectiveness of deep reinforcement learning (DRL) within interactive gaming environments, focusing on complex decision-making tasks. Utilizing the Unity engine, [...] Read more.

In the field of gaming artificial intelligence, selecting the appropriate machine learning approach is essential for improving decision-making and automation. This paper examines the effectiveness of deep reinforcement learning (DRL) within interactive gaming environments, focusing on complex decision-making tasks. Utilizing the Unity engine, we conducted experiments to evaluate DRL methodologies in simulating realistic and adaptive agent behavior. A vehicle driving game is implemented, in which the goal is to reach a certain target within a small number of steps, while respecting the boundaries of the roads. Our study compares Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) in terms of learning efficiency, decision-making accuracy, and adaptability. The results demonstrate that PPO successfully learns to reach the target, achieving higher and more stable cumulative rewards. Conversely, SAC struggles to reach the target, displaying significant variability and lower performance. These findings highlight the effectiveness of PPO in this context and indicate the need for further development, adaptation, and tuning of SAC. This research contributes to developing innovative approaches in how ML can improve how player agents adapt and react to their environments, thereby enhancing realism and dynamics in gaming experiences. Additionally, this work emphasizes the utility of using games to evolve such models, preparing them for real-world applications, namely in the field of vehicles’ autonomous driving and optimal route calculation. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

18 pages, 3207 KiB

Open AccessArticle

Imitating Human Go Players via Vision Transformer

by Yu-Heng Hsieh, Chen-Chun Kao and Shyan-Ming Yuan

Algorithms 2025, 18(2), 61; https://doi.org/10.3390/a18020061 - 24 Jan 2025

Viewed by 756

Abstract

Developing AI algorithms for the game of Go has long been a challenging task. While tools such as AlphaGo have revolutionized gameplay, their focus on maximizing win rates often leads to moves that are incomprehensible to human players, limiting their utility as training [...] Read more.

Developing AI algorithms for the game of Go has long been a challenging task. While tools such as AlphaGo have revolutionized gameplay, their focus on maximizing win rates often leads to moves that are incomprehensible to human players, limiting their utility as training aids. This work introduces a novel approach to bridge this gap by leveraging a Vision Transformer (ViT) to develop an AI model that achieves professional-level play while mimicking human decision-making. Using a dataset from the KGS Go server, our ViT-based model achieves 51.49% accuracy in predicting expert moves with a simple feature set. Comparative analysis against CNN-based models highlights the ViT’s superior performance in capturing patterns and replicating expert strategies. These findings establish ViTs as promising tools for enhancing Go training by aligning AI strategies with human intuition. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

23 pages, 2643 KiB

Open AccessArticle

An Efficient Optimization of the Monte Carlo Tree Search Algorithm for Amazons

by Lijun Zhang, Han Zou and Yungang Zhu

Algorithms 2024, 17(8), 334; https://doi.org/10.3390/a17080334 - 1 Aug 2024

Viewed by 1412

Abstract

Amazons is a computerized board game with complex positions that are highly challenging for humans. In this paper, we propose an efficient optimization of the Monte Carlo tree search (MCTS) algorithm for Amazons, fusing the ‘Move Groups’ strategy and the ‘Parallel Evaluation’ optimization [...] Read more.

Amazons is a computerized board game with complex positions that are highly challenging for humans. In this paper, we propose an efficient optimization of the Monte Carlo tree search (MCTS) algorithm for Amazons, fusing the ‘Move Groups’ strategy and the ‘Parallel Evaluation’ optimization strategy (MG-PEO). Specifically, we explain the high efficiency of the Move Groups strategy by defining a new criterion: the winning convergence distance. We also highlight the strategy’s potential issue of falling into a local optimum and propose that the Parallel Evaluation mechanism can compensate for this shortcoming. Moreover, We conducted rigorous performance analysis and experiments. Performance analysis results indicate that the MCTS algorithm with the Move Groups strategy can improve the playing ability of the Amazons game by 20–30 times compared to the traditional MCTS algorithm. The Parallel Evaluation optimization further enhances the playing ability of the Amazons game by 2–3 times. Experimental results show that the MCTS algorithm with the MG-PEO strategy achieves a 23% higher game-winning rate on average compared to the traditional MCTS algorithm. Additionally, the MG-PEO Amazons program proposed in this paper won first prize in the Amazons Competition at the 2023 China Collegiate Computer Games Championship & National Computer Games Tournament. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

14 pages, 777 KiB

Open AccessArticle

Evaluating the Expressive Range of Super Mario Bros Level Generators

by Hans Schaa and Nicolas A. Barriga

Algorithms 2024, 17(7), 307; https://doi.org/10.3390/a17070307 - 11 Jul 2024

Viewed by 1887

Abstract

Procedural Content Generation for video games (PCG) is widely used by today’s video game industry to create huge open worlds or enhance replayability. However, there is little scientific evidence that these systems produce high-quality content. In this document, we evaluate three open-source automated [...] Read more.

Procedural Content Generation for video games (PCG) is widely used by today’s video game industry to create huge open worlds or enhance replayability. However, there is little scientific evidence that these systems produce high-quality content. In this document, we evaluate three open-source automated level generators for Super Mario Bros in addition to the original levels used for training. These are based on Genetic Algorithms, Generative Adversarial Networks, and Markov Chains. The evaluation was performed through an Expressive Range Analysis (ERA) on 200 levels with nine metrics. The results show how analyzing the algorithms’ expressive range can help us evaluate the generators as a preliminary measure to study whether they respond to users’ needs. This method allows us to recognize potential problems early in the content generation process, in addition to taking action to guarantee quality content when a generator is used. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

22 pages, 3492 KiB

Open AccessArticle

Optimizing Reinforcement Learning Using a Generative Action-Translator Transformer

by Jiaming Li, Ning Xie and Tingting Zhao

Algorithms 2024, 17(1), 37; https://doi.org/10.3390/a17010037 - 16 Jan 2024

Viewed by 4769

Abstract

In recent years, with the rapid advancements in Natural Language Processing (NLP) technologies, large models have become widespread. Traditional reinforcement learning algorithms have also started experimenting with language models to optimize training. However, they still fundamentally rely on the Markov Decision Process (MDP) [...] Read more.

In recent years, with the rapid advancements in Natural Language Processing (NLP) technologies, large models have become widespread. Traditional reinforcement learning algorithms have also started experimenting with language models to optimize training. However, they still fundamentally rely on the Markov Decision Process (MDP) for reinforcement learning, and do not fully exploit the advantages of language models for dealing with long sequences of problems. The Decision Transformer (DT) introduced in 2021 is the initial effort to completely transform the reinforcement learning problem into a challenge within the NLP domain. It attempts to use text generation techniques to create reinforcement learning trajectories, addressing the issue of finding optimal trajectories. However, the article places the training trajectory data of reinforcement learning directly into a basic language model for training. Its aim is to predict the entire trajectory, encompassing state and reward information. This approach deviates from the reinforcement learning training objective of finding the optimal action. Furthermore, it generates redundant information in the output, impacting the final training effectiveness of the agent. This paper proposes a more reasonable network model structure, the Action-Translator Transformer (ATT), to predict only the next action of the agent. This makes the language model more interpretable for the reinforcement learning problem. We test our model in simulated gaming scenarios and compare it with current mainstream methods in the offline reinforcement learning field. Based on the presented experimental results, our model demonstrates superior performance. We hope that introducing this model will inspire new ideas and solutions for combining language models and reinforcement learning, providing fresh perspectives for offline reinforcement learning research. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

18 pages, 967 KiB

Open AccessArticle

Reducing Q-Value Estimation Bias via Mutual Estimation and Softmax Operation in MADRL

by Zheng Li, Xinkai Chen, Jiaqing Fu, Ning Xie and Tingting Zhao

Algorithms 2024, 17(1), 36; https://doi.org/10.3390/a17010036 - 16 Jan 2024

Cited by 2 | Viewed by 2236

Abstract

With the development of electronic game technology, the content of electronic games presents a larger number of units, richer unit attributes, more complex game mechanisms, and more diverse team strategies. Multi-agent deep reinforcement learning shines brightly in this type of team electronic game, [...] Read more.

With the development of electronic game technology, the content of electronic games presents a larger number of units, richer unit attributes, more complex game mechanisms, and more diverse team strategies. Multi-agent deep reinforcement learning shines brightly in this type of team electronic game, achieving results that surpass professional human players. Reinforcement learning algorithms based on Q-value estimation often suffer from Q-value overestimation, which may seriously affect the performance of AI in multi-agent scenarios. We propose a multi-agent mutual evaluation method and a multi-agent softmax method to reduce the estimation bias of Q values in multi-agent scenarios, and have tested them in both the particle multi-agent environment and the multi-agent tank environment we constructed. The multi-agent tank environment we have built has achieved a good balance between experimental verification efficiency and multi-agent game task simulation. It can be easily extended for different multi-agent cooperation or competition tasks. We hope that it can be promoted in the research of multi-agent deep reinforcement learning. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

27 pages, 893 KiB

Open AccessArticle

Official International Mahjong: A New Playground for AI Research

by Yunlong Lu, Wenxin Li and Wenlong Li

Algorithms 2023, 16(5), 235; https://doi.org/10.3390/a16050235 - 28 Apr 2023

Cited by 5 | Viewed by 4928

Abstract

Games have long been benchmarks and testbeds for AI research. In recent years, with the development of new algorithms and the boost in computational power, many popular games played by humans have been solved by AI systems. Mahjong is one of the most [...] Read more.

Games have long been benchmarks and testbeds for AI research. In recent years, with the development of new algorithms and the boost in computational power, many popular games played by humans have been solved by AI systems. Mahjong is one of the most popular games played in China and has been spread worldwide, which presents challenges for AI research due to its multi-agent nature, rich hidden information, and complex scoring rules, but it has been somehow overlooked in the community of game AI research. In 2020 and 2022, we held two AI competitions of Official International Mahjong, the standard variant of Mahjong rules, in conjunction with a top-tier AI conference called IJCAI. We are the first to adopt the duplicate format in evaluating Mahjong AI agents to mitigate the high variance in this game. By comparing the algorithms and performance of AI agents in the competitions, we conclude that supervised learning and reinforcement learning are the current state-of-the-art methods in this game and perform much better than heuristic methods based on human knowledge. We also held a human-versus-AI competition and found that the top AI agent still could not beat professional human players. We claim that this game can be a new benchmark for AI research due to its complexity and popularity among people. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

22 pages, 1927 KiB

Open AccessArticle

Measuring the Non-Transitivity in Chess

by Ricky Sanjaya, Jun Wang and Yaodong Yang

Algorithms 2022, 15(5), 152; https://doi.org/10.3390/a15050152 - 28 Apr 2022

Cited by 13 | Viewed by 3947

Abstract

In this paper, we quantify the non-transitivity in chess using human game data. Specifically, we perform non-transitivity quantification in two ways—Nash clustering and counting the number of rock–paper–scissor cycles—on over one billion matches from the Lichess and FICS databases. Our findings indicate that [...] Read more.

In this paper, we quantify the non-transitivity in chess using human game data. Specifically, we perform non-transitivity quantification in two ways—Nash clustering and counting the number of rock–paper–scissor cycles—on over one billion matches from the Lichess and FICS databases. Our findings indicate that the strategy space of real-world chess strategies has a spinning top geometry and that there exists a strong connection between the degree of non-transitivity and the progression of a chess player’s rating. Particularly, high degrees of non-transitivity tend to prevent human players from making progress in their Elo ratings. We also investigate the implications of non-transitivity for population-based training methods. By considering fixed-memory fictitious play as a proxy, we conclude that maintaining large and diverse populations of strategies is imperative to training effective AI agents for solving chess. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

23 pages, 591 KiB

Open AccessArticle

Research and Challenges of Reinforcement Learning in Cyber Defense Decision-Making for Intranet Security

by Wenhao Wang, Dingyuanhao Sun, Feng Jiang, Xingguo Chen and Cheng Zhu

Algorithms 2022, 15(4), 134; https://doi.org/10.3390/a15040134 - 18 Apr 2022

Cited by 14 | Viewed by 6839

Abstract

In recent years, cyber attacks have shown diversified, purposeful, and organized characteristics, which pose significant challenges to cyber defense decision-making on internal networks. Due to the continuous confrontation between attackers and defenders, only using data-based statistical or supervised learning methods cannot cope with [...] Read more.

In recent years, cyber attacks have shown diversified, purposeful, and organized characteristics, which pose significant challenges to cyber defense decision-making on internal networks. Due to the continuous confrontation between attackers and defenders, only using data-based statistical or supervised learning methods cannot cope with increasingly severe security threats. It is urgent to rethink network defense from the perspective of decision-making, and prepare for every possible situation. Reinforcement learning has made great breakthroughs in addressing complicated decision-making problems. We propose a framework that defines four modules based on the life cycle of threats: pentest, design, response, recovery. Our aims are to clarify the problem boundary of network defense decision-making problems, to study the problem characteristics in different contexts, to compare the strengths and weaknesses of existing research, and to identify promising challenges for future work. Our work provides a systematic view for understanding and solving decision-making problems in the application of reinforcement learning to cyber defense. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

Review

Jump to: Research

27 pages, 363 KiB

Open AccessReview

Techniques and Paradigms in Modern Game AI Systems

by Yunlong Lu and Wenxin Li

Algorithms 2022, 15(8), 282; https://doi.org/10.3390/a15080282 - 12 Aug 2022

Cited by 11 | Viewed by 10538

Abstract

Games have long been benchmarks and test-beds for AI algorithms. With the development of AI techniques and the boost of computational power, modern game AI systems have achieved superhuman performance in many games played by humans. These games have various features and present [...] Read more.

Games have long been benchmarks and test-beds for AI algorithms. With the development of AI techniques and the boost of computational power, modern game AI systems have achieved superhuman performance in many games played by humans. These games have various features and present different challenges to AI research, so the algorithms used in each of these AI systems vary. This survey aims to give a systematic review of the techniques and paradigms used in modern game AI systems. By decomposing each of the recent milestones into basic components and comparing them based on the features of games, we summarize the common paradigms to build game AI systems and their scope and limitations. We claim that deep reinforcement learning is the most general methodology to become a mainstream method for games with higher complexity. We hope this survey can both provide a review of game AI algorithms and bring inspiration to the game AI community for future directions. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

43 pages, 565 KiB

Open AccessReview

A Review: Machine Learning for Combinatorial Optimization Problems in Energy Areas

by Xinyi Yang, Ziyi Wang, Hengxi Zhang, Nan Ma, Ning Yang, Hualin Liu, Haifeng Zhang and Lei Yang

Algorithms 2022, 15(6), 205; https://doi.org/10.3390/a15060205 - 13 Jun 2022

Cited by 21 | Viewed by 10110

Abstract

Combinatorial optimization problems (COPs) are a class of NP-hard problems with great practical significance. Traditional approaches for COPs suffer from high computational time and reliance on expert knowledge, and machine learning (ML) methods, as powerful tools have been used to overcome these problems. [...] Read more.

Combinatorial optimization problems (COPs) are a class of NP-hard problems with great practical significance. Traditional approaches for COPs suffer from high computational time and reliance on expert knowledge, and machine learning (ML) methods, as powerful tools have been used to overcome these problems. In this review, the COPs in energy areas with a series of modern ML approaches, i.e., the interdisciplinary areas of COPs, ML and energy areas, are mainly investigated. Recent works on solving COPs using ML are sorted out firstly by methods which include supervised learning (SL), deep learning (DL), reinforcement learning (RL) and recently proposed game theoretic methods, and then problems where the timeline of the improvements for some fundamental COPs is the layout. Practical applications of ML methods in the energy areas, including the petroleum supply chain, steel-making, electric power system and wind power, are summarized for the first time, and challenges in this field are analyzed. Full article

(This article belongs to the Special Issue Algorithms for Games AI)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Algorithms for Games AI

Share This Special Issue

Special Issue Editors

Special Issue Information

Benefits of Publishing in a Special Issue

Published Papers (12 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI