Skip to Content

86 Results Found

  • Review
  • Open Access
14 Citations
4,147 Views
53 Pages

18 February 2025

Reinforcement Learning (RL) is a widely researched area in artificial intelligence that focuses on teaching agents decision-making through interactions with their environment. A key subset includes multi-armed bandit (MAB) and stochastic continuum-ar...

(This article belongs to the Special Issue Advances in Statistical AI and Causal Inference)
  • Article
  • Open Access
9 Citations
7,088 Views
25 Pages

9 January 2025

Non-stationary multi-armed bandit (MAB) problems have recently attracted extensive attention. We focus on the abruptly changing scenario where reward distributions remain constant for a certain period and change at unknown time steps. Although Thomps...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
14 Citations
7,708 Views
35 Pages

25 July 2025

Background/Objectives: Decision-analytic Bayesian approaches are ideally suited for designing clinical trials. They have been used increasingly over the last 30 years in developing medical devices and drugs. A prototype trial is a bandit problem in w...

(This article belongs to the Section Clinical Research Methods)
  • Review
  • Open Access
27 Citations
11,913 Views
27 Pages

28 March 2023

The restless multi-armed bandit problem is a paradigmatic modeling framework for optimal dynamic priority allocation in stochastic models of wide-ranging applications that has been widely investigated and applied since its inception in a seminal pape...

(This article belongs to the Special Issue Advances in Markovian Dynamic and Stochastic Optimization Models in Diverse Application Areas)
  • Article
  • Open Access
36 Citations
8,874 Views
16 Pages

7 December 2016

How can we harness nature’s power for computations? Our society comprises a collection of individuals, each of whom handles decision-making tasks that are abstracted as computational problems of finding the most profitable option from a set of option...

(This article belongs to the Special Issue Natural Computation: Attempts in Reconciliation of Dialectic Oppositions)
  • Article
  • Open Access
452 Views
41 Pages

A Novel Binary Hunger Games Search Algorithm with Data-Driven Repair for the Set Covering Problem

  • Broderick Crawford,
  • Hugo Caballero,
  • Gino Astorga,
  • Felipe Cisternas-Caneo and ,
  • Alan Baeza,
  • Pablo Puga Lucero,
  • Giovanni Giachetti and
  • Ricardo Soto
  • + 3 authors+ 0 author

8 September 2026

Solving problems associated with the efficient distribution and organization of resources has generated increasing interest in the scientific community. One of the most commonly used approaches consists of approximate solution techniques, which have...

(This article belongs to the Special Issue Next-Generation Bio-Inspired Algorithms: Theory, Design, and Performance)
  • Article
  • Open Access
4 Citations
1,882 Views
29 Pages

This work presents an artificial intelligence technique to minimise path planning computer processing time for successful GMA-DED 3D printings. An advanced version of the Pixel space-filling-based strategy family is proposed and developed, using, ori...

  • Article
  • Open Access
10 Citations
4,799 Views
14 Pages

Exploring Multi-Armed Bandit (MAB) as an AI Tool for Optimising GMA-WAAM Path Planning

  • Rafael Pereira Ferreira,
  • Emil Schubert and
  • Américo Scotti

Conventional path-planning strategies for GMA-WAAM may encounter challenges related to geometrical features when printing complex-shaped builds. One alternative to mitigate geometry-related flaws is to use algorithms that optimise trajectory choices&...

(This article belongs to the Special Issue Advances in Directed Energy Deposition Additive Manufacturing)
  • Article
  • Open Access
1,190 Views
23 Pages

Evidence-Driven Simulated Data in Reinforcement Learning Training for Personalized mHealth Interventions

  • Juan Carlos Caro,
  • Giorgio Galgano,
  • Melissa Muñoz,
  • Jorge Díaz Ramírez and
  • Jorge Maluenda

2 April 2026

Physical inactivity is a major preventable cause of non-communicable disease and premature mortality. Mobile health interventions can promote physical activity, but their effectiveness depends on the ability to adapt to user’s context and motiv...

(This article belongs to the Special Issue Health Informatics: Human Health and Health Care Services)
  • Article
  • Open Access
2 Citations
1,047 Views
17 Pages

An Intelligent Machine Learning-Driven Solving Framework for Capacitated Vehicle Routing

  • Hajar Bideq,
  • Khaoula Ouaddi,
  • Rachid Ellaia and
  • Agnès Gorge

Despite recent advancements in solving the Capacitated Vehicle Routing Problem (CVRP), state-of-the-art learning-based methods remain hindered by costly offline training, while classical population solvers rely on implicit mechanisms and rigid parame...

(This article belongs to the Special Issue Advances in Mathematical Models and Computational Intelligence for Transportation System Planning and Management)
  • Article
  • Open Access
14 Citations
3,928 Views
25 Pages

Combinatorial MAB-Based Joint Channel and Spreading Factor Selection for LoRa Devices

  • Ikumi Urabe,
  • Aohan Li,
  • Minoru Fujisawa,
  • Song-Ju Kim and
  • Mikio Hasegawa

26 July 2023

Long-Range (LoRa) devices have been deployed in many Internet of Things (IoT) applications due to their ability to communicate over long distances with low power consumption. The scalability and communication performance of the LoRa systems are highl...

(This article belongs to the Section Communications)
  • Feature Paper
  • Article
  • Open Access
28 Citations
13,703 Views
22 Pages

LLM-Informed Multi-Armed Bandit Strategies for Non-Stationary Environments

  • J. de Curtò,
  • I. de Zarzà,
  • Gemma Roig,
  • Juan Carlos Cano and ,
  • Pietro Manzoni and
  • Carlos T. Calafate
  • + 1 author+ 0 author

In this paper, we introduce an innovative approach to handling the multi-armed bandit (MAB) problem in non-stationary environments, harnessing the predictive power of large language models (LLMs). With the realization that traditional bandit strategi...

(This article belongs to the Special Issue Zero-Shot Learning in Natural Language Processing and Its Applications)
  • Article
  • Open Access
6 Citations
2,536 Views
21 Pages

Non-Myopic Beam Scheduling for Multiple Smart-Target Tracking in Phased Array Radar Networks

  • Yuhang Hao,
  • Zengfu Wang,
  • José Niño-Mora,
  • Jing Fu and ,
  • Quan Pan and
  • Min Yang
  • + 1 author+ 0 author

4 December 2024

This paper addresses beam scheduling for tracking multiple smart targets in phased array radar networks, aiming to mitigate the performance degradation in previous myopic scheduling methods and enhance the tracking performance, which is measured by a...

(This article belongs to the Section Radar Sensors)
  • Feature Paper
  • Article
  • Open Access
10 Citations
7,292 Views
20 Pages

3 January 2025

The multi-armed bandit (MAB) problem is a foundational model for sequential decision-making under uncertainty. While MAB has proven valuable in applications such as clinical trials and online advertising, traditional formulations have limitations; sp...

  • Article
  • Open Access
6 Citations
5,021 Views
21 Pages

15 December 2020

The Whittle index for restless bandits (two-action semi-Markov decision processes) provides an intuitively appealing optimal policy for controlling a single generic project that can be active (engaged) or passive (rested) at each decision epoch, and...

(This article belongs to the Special Issue Applied Probability)
  • Article
  • Open Access
3 Citations
2,708 Views
24 Pages

The widespread adoption of distributed energy resources (DERs) leads to resource redundancy in grid operation and increases computation complexity, which underscores the need for effective resource management strategies. In this paper, we present a n...

(This article belongs to the Section Power Electronics)
  • Feature Paper
  • Article
  • Open Access
6 Citations
2,940 Views
31 Pages

18 July 2022

This paper considers what we propose to call multi-gear bandits, which are Markov decision processes modeling a generic dynamic and stochastic project fueled by a single resource and which admit multiple actions representing gears of operation natura...

(This article belongs to the Section D1: Probability and Statistics)
  • Article
  • Open Access
3 Citations
2,568 Views
38 Pages

17 July 2024

We study stochastic linear contextual bandits (CB) where the agent observes a noisy version of the true context through a noise channel with unknown channel parameters. Our objective is to design an action policy that can “approximate” th...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
9 Citations
2,631 Views
18 Pages

Budgeted Bandits for Power Allocation and Trajectory Planning in UAV-NOMA Aided Networks

  • Ramez Hosny,
  • Sherief Hashima,
  • Ehab Mahmoud Mohamed,
  • Rokaia M. Zaki and
  • Basem M. ElHalawany

7 August 2023

On one hand combining Unmanned Aerial Vehicles (UAVs) and Non-Orthogonal Multiple Access (NOMA) is a remarkable direction to sustain the exponentially growing traffic requirements of the forthcoming Sixth Generation (6G) networks. In this paper, we i...

(This article belongs to the Section Drone Communications)
  • Feature Paper
  • Article
  • Open Access
7 Citations
3,882 Views
20 Pages

Bandit Algorithm Driven by a Classical Random Walk and a Quantum Walk

  • Tomoki Yamagami,
  • Etsuo Segawa,
  • Takatomo Mihana,
  • André Röhm and ,
  • Ryoichi Horisaki and
  • Makoto Naruse
  • + 1 author+ 0 author

25 May 2023

Quantum walks (QWs) have a property that classical random walks (RWs) do not possess—the coexistence of linear spreading and localization—and this property is utilized to implement various kinds of applications. This paper proposes RW- an...

(This article belongs to the Section Quantum Information)
  • Article
  • Open Access
900 Views
26 Pages

21 May 2026

While many multi-armed bandit algorithms assume that rewards for all arms are constant across rounds, this assumption does not hold in many real-world scenarios. This paper considers the setting of recovering bandits, where the reward depends on the...

(This article belongs to the Section Evolutionary Algorithms and Machine Learning)
  • Systematic Review
  • Open Access
960 Views
25 Pages

Adaptive decision-making has emerged as an important direction in audio classification. Reinforcement learning (RL) and multi-armed bandit (MAB) methods offer principled frameworks for sequential and localized decision-making. However, despite growin...

(This article belongs to the Special Issue Artificial Intelligence for Signal, Image, and Multimodal Data Processing: Algorithms, Models, and Knowledge Extraction)
  • Article
  • Open Access
1 Citations
4,353 Views
22 Pages

28 February 2018

Audit mechanisms frequently take place in the context of repeated relationships between auditor and auditee. This paper focuses attention on the insurance fraud problem in a setting where insurers repeatedly verify claims satisfied by service provide...

  • Article
  • Open Access
2 Citations
2,295 Views
18 Pages

18 April 2024

The multi-armed bandit (MAB) problem is a typical problem of exploration and exploitation. As a classical MAB problem, the stochastic multi-armed bandit (SMAB) is the basis of reinforcement learning recommendation. However, most existing SMAB and MAB...

(This article belongs to the Section Computing and Artificial Intelligence)
  • Article
  • Open Access
1 Citations
4,147 Views
13 Pages

Maximum Entropy Exploration in Contextual Bandits with Neural Networks and Energy Based Models

  • Adam Elwood,
  • Marco Leonardi,
  • Ashraf Mohamed and
  • Alessandro Rozza

18 January 2023

Contextual bandits can solve a huge range of real-world problems. However, current popular algorithms to solve them either rely on linear models or unreliable uncertainty estimation in non-linear models, which are required to deal with the exploratio...

(This article belongs to the Topic Machine and Deep Learning)
  • Article
  • Open Access
2 Citations
3,364 Views
36 Pages

29 December 2020

We consider the multi-armed bandit problem with penalties for switching that include setup delays and costs, extending the former results of the author for the special case with no switching delays. A priority index for projects with setup delays tha...

(This article belongs to the Special Issue Stochastic Models with Applications)
  • Article
  • Open Access
48 Citations
5,374 Views
19 Pages

UAV Trajectory Optimization in a Post-Disaster Area Using Dual Energy-Aware Bandits

  • Amr Amrallah,
  • Ehab Mahmoud Mohamed,
  • Gia Khanh Tran and
  • Kei Sakaguchi

26 January 2023

Over the past few years, with the rapid increase in the number of natural disasters, the need to provide smart emergency wireless communication services has become crucial. Unmanned aerial Vehicles (UAVs) have gained much attention as promising candi...

(This article belongs to the Special Issue New Methods and Applications for UAVs)
  • Article
  • Open Access
45 Citations
9,982 Views
22 Pages

19 September 2011

Due to the influence of unpredictable random events, the processing time of each operation should be treated as random variables if we aim at a robust production schedule. However, compared with the extensive research on the deterministic model, the...

  • Article
  • Open Access
18 Citations
5,576 Views
18 Pages

26 November 2021

This paper presents innovative reinforcement learning methods for automatically tuning the parameters of a proportional integral derivative controller. Conventionally, the high dimension of the Q-table is a primary drawback when implementing a reinfo...

(This article belongs to the Special Issue Design and Control of Electrical Machines)
  • Article
  • Open Access
3,885 Views
19 Pages

20 July 2020

In this paper, we consider techniques for establishing lower bounds on the number of arm pulls for best-arm identification in the multi-armed bandit problem. While a recent divergence-based approach was shown to provide improvements over an older gap...

(This article belongs to the Special Issue Information Theory in Machine Learning and Data Science II)
  • Article
  • Open Access
3 Citations
4,589 Views
15 Pages

We consider a risk-aware multi-armed bandit framework with the goal of avoiding catastrophic risk. Such a framework has multiple applications in financial risk management. We introduce a new conditional value-at-risk (CVaR) estimation procedure combi...

(This article belongs to the Section Risk)
  • Article
  • Open Access
12 Citations
4,923 Views
14 Pages

An Anti-Jamming Hierarchical Optimization Approach in Relay Communication System via Stackelberg Game

  • Zhibin Feng,
  • Guochun Ren,
  • Jin Chen,
  • Chaohui Chen and ,
  • Xiaoqin Yang,
  • Yijie Luo and
  • Kun Xu
  • + 2 authors+ 0 author

14 August 2019

In this paper, we study joint relay selection and the power control optimization problem in an anti-jamming relay communication system. Considering the hierarchical competitive relationship between a user and jammer, we formulate the anti-jamming pro...

(This article belongs to the Section Electrical, Electronics and Communications Engineering)
  • Article
  • Open Access
4 Citations
3,265 Views
29 Pages

Natural disasters (e.g., floods, earthquakes) significantly impact citizens, economies, and the environment worldwide. Due to their sudden onset, devastating effects, and high uncertainty, it is crucial for emergency departments to take swift action...

(This article belongs to the Section Applied Science and Humanities for Smart Cities)
  • Article
  • Open Access
7 Citations
4,261 Views
12 Pages

14 October 2019

There is strong support for formative assessment inclusion in learning processes, with the main emphasis on corrective feedback for students. However, traditional testing and Computer Adaptive Testing can be problematic to implement in the classroom....

(This article belongs to the Special Issue Smart Learning)
  • Feature Paper
  • Article
  • Open Access
10 Citations
10,304 Views
17 Pages

15 November 2017

In recent years, tools from information theory have played an increasingly prevalent role in statistical machine learning. In addition to developing efficient, computationally feasible algorithms for analyzing complex datasets, it is of theoretical i...

(This article belongs to the Special Issue Information Theory in Machine Learning and Data Science)
  • Article
  • Open Access
4 Citations
5,759 Views
8 Pages

Decentralized Blind Spectrum Selection in Cognitive Radio Networks Considering Handoff Cost

  • Yongqun Chen,
  • Huaibei Zhou,
  • Ruoshan Kong,
  • Li Zhu and
  • Huaqing Mao

Due to the spectrum varying nature of cognitive radio networks, secondary users are required to perform spectrum handoffs when the spectrum is occupied by primary users, which will lead to a handoff delay. In this paper, based on the multi-armed band...

  • Article
  • Open Access
2 Citations
2,670 Views
14 Pages

1 January 2023

We present REGA, a new adaptive-sampling-based algorithm for the control of finite-horizon Markov decision processes (MDPs) with very large state spaces and small action spaces. We apply a variant of the ϵ-greedy multiarmed bandit algorithm to...

(This article belongs to the Special Issue Feature Paper Special Issue: Reinforcement Learning)
  • Article
  • Open Access
488 Views
33 Pages

4 August 2026

This paper studies the Vehicle Routing Problem with Simultaneous Pickup and Delivery and Time Windows (VRPSPDTW). The problem arises from reverse logistics, last-mile distribution, and circular logistics, where vehicles must serve delivery and pickup...

(This article belongs to the Special Issue Metaheuristic Algorithms, 2nd Edition)
  • Article
  • Open Access
2 Citations
357 Views
25 Pages

Resource Allocation via Bayesian Optimization in Wasserstein Spaces vs. Semi-Bandit Feedback

  • Antonio Candelieri,
  • Francesco Archetti,
  • Iman Seyedi and
  • Andrea Ponti

Sequential resource allocation has long been a central problem in operations research, yet ongoing technological developments, particularly in cloud and high-performance computing and in multi-channel marketing, are giving rise to new structural cons...

(This article belongs to the Section Artificial Intelligence and Multi-Agent Systems)
  • Article
  • Open Access
24 Citations
4,605 Views
19 Pages

25 November 2021

Modern wireless networks are notorious for being very dense, uncoordinated, and selfish, especially with greedy user needs. This leads to a critical scarcity problem in spectrum resources. The Dynamic Spectrum Access system (DSA) is considered a prom...

(This article belongs to the Special Issue Unmanned Aerial Vehicle (UAV)-Enabled Wireless Communications and Networking)
  • Article
  • Open Access
4 Citations
1,642 Views
17 Pages

29 May 2025

This paper investigates data transmission in an Internet of Things (IoT) network, where multiple devices send environmental data to a remote base station through an unmanned aerial vehicle (UAV) relay. The UAV serves as an airborne intermediary that...

(This article belongs to the Special Issue Task-Oriented Communications in Industrial IoT: Age of Information and Beyond)
  • Article
  • Open Access
4 Citations
2,809 Views
19 Pages

6 January 2025

Artificial intelligence plays an indispensable role in improving productivity and promoting social development, and causal discovery is one of the extremely important research directions in this field. Acyclic directed graphs (DAGs) are the most comm...

(This article belongs to the Special Issue Causal Graphical Models and Their Applications)
  • Article
  • Open Access
613 Views
17 Pages

Spectral Efficiency Beamforming Scheme for UAV MIMO Communication via Budgeted Combinatorial Multi-Armed Bandit

  • Jing Gao,
  • Yunxing Xiang,
  • Yunchao Song,
  • Jing Zhu and ,
  • Jun Wang,
  • Xiaohui You,
  • Ge Wang and
  • Tianbao Gao
  • + 3 authors+ 0 author

6 December 2025

Unmanned aerial vehicles (UAVs) equipped with antenna arrays can deliver high-capacity, high-throughput, and low-latency communication services. Considering a UAV-assisted mmWave multi-input and multi-output (MIMO) system, a two-stage beamforming sch...

(This article belongs to the Special Issue Massive MIMO and Other Advanced Antenna Techniques for 5G/6G Wireless Systems)
  • Article
  • Open Access
1 Citations
3,749 Views
17 Pages

Sampling the Variational Posterior with Local Refinement

  • Marton Havasi,
  • Jasper Snoek,
  • Dustin Tran,
  • Jonathan Gordon and
  • José Miguel Hernández-Lobato

8 November 2021

Variational inference is an optimization-based method for approximating the posterior distribution of the parameters in Bayesian probabilistic models. A key challenge of variational inference is to approximate the posterior with a distribution that i...

(This article belongs to the Special Issue Probabilistic Methods for Deep Learning)
  • Article
  • Open Access
3 Citations
2,414 Views
15 Pages

29 August 2023

As a key 5G technology, massive multiple-input multiple-output (MIMO) can effectively improve system capacity and reduce latency. This paper proposes a user scheduling and spectrum allocation method based on combinatorial multi-armed bandit (CMAB) fo...

(This article belongs to the Section Communications)
  • Article
  • Open Access
17 Citations
3,668 Views
18 Pages

Distribution of Multi MmWave UAV Mounted RIS Using Budget Constraint Multi-Player MAB

  • Ehab Mahmoud Mohamed,
  • Mohammad Alnakhli,
  • Sherief Hashima and
  • Mohamed Abdel-Nasser

Millimeter wave (mmWave), reconfigurable intelligent surface (RIS), and unmanned aerial vehicles (UAVs) are considered vital technologies of future six-generation (6G) communication networks. In this paper, various UAV mounted RIS are distributed to...

(This article belongs to the Section Networks)
  • Article
  • Open Access
2 Citations
1,820 Views
20 Pages

MAB-Based Online Client Scheduling for Decentralized Federated Learning in the IoT

  • Zhenning Chen,
  • Xinyu Zhang,
  • Siyang Wang and
  • Youren Wang

18 April 2025

Different from conventional federated learning (FL), which relies on a central server for model aggregation, decentralized FL (DFL) exchanges models among edge servers, thus improving the robustness and scalability. When deploying DFL into the Intern...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
765 Views
18 Pages

16 October 2025

The calculation of Total Transfer Capability (TTC) for transmission corridors serves as the foundation for security region determination and electricity market transactions. However, existing TTC methods often neglect corridor correlations, leading t...

(This article belongs to the Special Issue Advances in Power System Dynamics, Stability, Control and Dispatch with Large-Scale Renewable Energy Penetrated, 2nd Edition)
  • Article
  • Open Access
10 Citations
3,340 Views
21 Pages

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

  • Zhiyu Chen,
  • Hongxi Zhou,
  • Siyuan Du,
  • Jiayan Liu and ,
  • Luyang Zhang and
  • Qi Liu
  • + 1 author+ 0 author

With the construction and development of the smart grid, the power business puts higher requirements on the communication capability of the network. In order to improve the energy efficiency of the space–air–ground-integrated power three-...

(This article belongs to the Special Issue 5G and 6G Wireless Systems: Challenges, Insights, and Opportunities)
  • Article
  • Open Access
458 Views
25 Pages

1 June 2026

In edge-intelligent systems, efficient resource management and task scheduling are critical but challenging due to the dynamic and heterogeneous nature of edge nodes (e.g., IoT devices, drones). We model this dynamic resource allocation challenge as...

(This article belongs to the Special Issue Symmetry and Asymmetry in Embedded Systems)

of 2