Skip to Content

Algorithms, Volume 15, Issue 3

2022 March - 30 articles

Cover Story: Reinforcement learning (RL) with sparse rewards is still an open challenge. Classic methods rely on learning via extrinsic rewards, and in situations where these are sparse, the agent may not learn at all. Similarly, if the agent gets rewards that create suboptimal modes of the objective function, it will prematurely stop exploring. Recent methods add intrinsic rewards to encourage exploration, but they lead to a non-stationary target for the Q-function. In this paper, we present a novel approach that (1) plans exploration far into the future using a long-term visit count and (2) decouples exploration and exploitation by learning a separate function. We also propose new environments for benchmarking exploration in RL. Results show that our approach outperforms existing methods. View this paper
  • Issues are regarded as officially published after their release is announced to the table of contents alert mailing list .
  • You may sign up for email alerts to receive table of contents of newly released issues.
  • PDF is the official format for papers published in both, html and pdf forms. To view the papers in pdf format, click on the "PDF Full-text" link, and use the free Adobe Reader to open them.

Articles (30)

  • Article
  • Open Access
7 Citations
4,280 Views
19 Pages

21 March 2022

In this article, multi-fidelity kriging and sparse polynomial chaos expansion (SPCE) surrogate models are constructed. In addition, a novel combination of the two surrogate approaches into a multi-fidelity SPCE-Kriging model will be presented. Accura...

  • Article
  • Open Access
8 Citations
6,826 Views
21 Pages

Key Concepts, Weakness and Benchmark on Hash Table Data Structures

  • Santiago Tapia-Fernández,
  • Daniel García-García and
  • Pablo García-Hernandez

21 March 2022

Most computer programs or applications need fast data structures. The performance of a data structure is necessarily influenced by the complexity of its common operations; thus, any data-structure that exhibits a theoretical complexity of amortized c...

  • Article
  • Open Access
6 Citations
4,538 Views
16 Pages

19 March 2022

Topology optimization offers a possibility to derive load-compliant structures. These structures tend to be complex, and conventional manufacturing offers only limited possibilities for their production. Additive manufacturing provides a remedy due t...

  • Article
  • Open Access
11 Citations
5,689 Views
18 Pages

Evolutionary Optimization of Spiking Neural P Systems for Remaining Useful Life Prediction

  • Leonardo Lucio Custode,
  • Hyunho Mo,
  • Andrea Ferigo and
  • Giovanni Iacca

19 March 2022

Remaining useful life (RUL) prediction is a key enabler for predictive maintenance. In fact, the possibility of accurately and reliably predicting the RUL of a system, based on a record of its monitoring data, can allow users to schedule maintenance...

  • Article
  • Open Access
2 Citations
3,112 Views
30 Pages

A Dynamic Distributed Deterministic Load-Balancer for Decentralized Hierarchical Infrastructures

  • Spyros Sioutas,
  • Efrosini Sourla,
  • Kostas Tsichlas,
  • Gerasimos Vonitsanos and
  • Christos Zaroliagis

18 March 2022

In this work, we propose D3-Tree, a dynamic distributed deterministic structure for data management in decentralized networks, by engineering and extending an existing decentralized structure. Conducting an extensive experimental study, we verify tha...

  • Article
  • Open Access
18 Citations
8,308 Views
18 Pages

Prediction of Harvest Time of Apple Trees: An RNN-Based Approach

  • Tiago Boechel,
  • Lucas Micol Policarpo,
  • Gabriel de Oliveira Ramos,
  • Rodrigo da Rosa Righi and
  • Dhananjay Singh

18 March 2022

In the field of agricultural research, Machine Learning (ML) has been used to increase agricultural productivity and minimize its environmental impact, proving to be an essential technique to support decision making. Accurate harvest time prediction...

  • Article
  • Open Access
5 Citations
4,274 Views
16 Pages

15 March 2022

Vibration signal analysis is the most common technique used for mechanical vibration monitoring. By using vibration sensors, the fault prognosis of rotating machinery provides a way to detect possible machine damage at an early stage and prevent prop...

  • Article
  • Open Access
23 Citations
4,975 Views
19 Pages

11 March 2022

The COVID-19 epidemic has highlighted the significance of sanitization and maintaining hygienic access to clean water to reduce mortality and morbidity cases worldwide. Diarrhea is one of the prevalent waterborne diseases caused due to contaminated w...

  • Article
  • Open Access
2 Citations
3,554 Views
16 Pages

Mean Estimation on the Diagonal of Product Manifolds

  • Mathias Højgaard Jensen and
  • Stefan Sommer

10 March 2022

Computing sample means on Riemannian manifolds is typically computationally costly, as exemplified by computation of the Fréchet mean, which often requires finding minimizing geodesics to each data point for each step of an iterative optimizat...

  • Article
  • Open Access
8 Citations
4,079 Views
14 Pages

Analysis of Explainable Goal-Driven Reinforcement Learning in a Continuous Simulated Environment

  • Ernesto Portugal,
  • Francisco Cruz,
  • Angel Ayala and
  • Bruno Fernandes

9 March 2022

Currently, artificial intelligence is in an important period of growth. Due to the technology boom, it is now possible to solve problems that could not be resolved previously. For example, through goal-driven learning, it is possible that intelligent...

  • Article
  • Open Access
2 Citations
4,128 Views
24 Pages

Kleene Algebra to Compute Invariant Sets of Dynamical Systems

  • Thomas Le Mézo,
  • Luc Jaulin,
  • Damien Massé and
  • Benoit Zerr

8 March 2022

In this paper, we show that a basic fixed point method used to enclose the greatest fixed point in a Kleene algebra will allow us to compute inner and outer approximations of invariant-based sets for continuous-time nonlinear dynamical systems. Our c...

  • Article
  • Open Access
2 Citations
3,448 Views
17 Pages

8 March 2022

At present, the unsupervised visual representation learning of the point cloud model is mainly based on generative methods, but the generative methods pay too much attention to the details of each point, thus ignoring the learning of semantic informa...

  • Article
  • Open Access
11 Citations
4,962 Views
16 Pages

8 March 2022

Agricultural machinery rental is a new service form that uses big data in agriculture to improve the utilization rate of agricultural machinery and promote the development of the agricultural economy. To realize agricultural machinery scheduling opti...

  • Article
  • Open Access
7 Citations
4,178 Views
32 Pages

A Seed-Guided Latent Dirichlet Allocation Approach to Predict the Personality of Online Users Using the PEN Model

  • Saravanan Sagadevan,
  • Nurul Hashimah Ahamed Hassain Malim and
  • Mohd Heikal Husin

8 March 2022

There is a growing interest in topic modeling to decipher the valuable information embedded in natural texts. However, there are no studies training an unsupervised model to automatically categorize the social networks (SN) messages according to pers...

  • Article
  • Open Access
2 Citations
3,208 Views
14 Pages

8 March 2022

Prediction of intrinsic disordered proteins is a hot area in the field of bio-information. Due to the high cost of evaluating the disordered regions of protein sequences using experimental methods, we used a low-complexity prediction scheme. Sequence...

  • Article
  • Open Access
13 Citations
6,650 Views
12 Pages

Dynamic Layout Design Optimization to Improve Patient Flow in Outpatient Clinics Using Genetic Algorithms

  • Jyoti R. Munavalli,
  • Shyam Vasudeva Rao,
  • Aravind Srinivasan and
  • Frits Van Merode

6 March 2022

Evolutionary algorithms, such as genetic algorithms have been used in various optimization problems. In this paper, we propose to apply this algorithm to obtain the layout design/redesign in order to improve the patient flow in an outpatient clinic....

  • Article
  • Open Access
10 Citations
6,361 Views
16 Pages

3 March 2022

Nowadays, eye fatigue is becoming more common globally. However, there was no objective and effective method for eye fatigue detection except the sample survey questionnaire. An eye fatigue detection method by machine learning based on the Single-Cha...

  • Article
  • Open Access
9 Citations
4,081 Views
12 Pages

1 March 2022

Detecting insulators on a power transmission line is of great importance for the safe operation of power systems. Aiming at the problem of the missed detection and misjudgment of the original feature extraction network VGG16 of a faster region-convol...

  • Article
  • Open Access
2 Citations
4,405 Views
17 Pages

Predicting Dynamic User–Item Interaction with Meta-Path Guided Recursive RNN

  • Yi Liu,
  • Chengyu Yin,
  • Jingwei Li,
  • Fang Wang and
  • Senzhang Wang

28 February 2022

Accurately predicting user–item interactions is critically important in many real applications, including recommender systems and user behavior analysis in social networks. One major drawback of existing studies is that they generally directly...

  • Review
  • Open Access
5 Citations
4,788 Views
14 Pages

Non-Invasive Systems and Methods Patents Review Based on Electrocardiogram for Diagnosis of Cardiovascular Diseases

  • Nellyzeth Flores,
  • Marco A. Reyna,
  • Roberto L. Avitia,
  • Jose Antonio Cardenas-Haro and
  • Conrado Garcia-Gonzalez

28 February 2022

Cardiovascular disease (CVD) is a global public health problem. It is a disease of multifactorial origin, and with this characteristic, having an accurate diagnosis of its incidence is a problem that health personnel face every day. That is why havin...

  • Article
  • Open Access
6 Citations
4,793 Views
44 Pages

Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning

  • Simone Parisi,
  • Davide Tateo,
  • Maximilian Hensel,
  • Carlo D’Eramo,
  • Jan Peters and
  • Joni Pajarinen

28 February 2022

Reinforcement learning with sparse rewards is still an open challenge. Classic methods rely on getting feedback via extrinsic rewards to train the agent, and in situations where this occurs very rarely the agent learns slowly or cannot learn at all....

  • Article
  • Open Access
1 Citations
4,019 Views
20 Pages

An Effective Algorithm for Finding Shortest Paths in Tubular Spaces

  • Dang-Viet-Anh Nguyen,
  • Jérôme Szewczyk and
  • Kanty Rabenorosoa

25 February 2022

We propose a novel algorithm to determine the Euclidean shortest path (ESP) from a given point (source) to another point (destination) inside a tubular space. The method is based on the observation data of a virtual particle (VP) assumed to move alon...

  • Article
  • Open Access
7 Citations
4,342 Views
37 Pages

25 February 2022

The Expectation Maximisation (EM) algorithm is widely used to optimise non-convex likelihood functions with latent variables. Many authors modified its simple design to fit more specific situations. For instance, the Expectation (E) step has been rep...

  • Article
  • Open Access
11 Citations
5,783 Views
16 Pages

Prediction of Injuries in CrossFit Training: A Machine Learning Perspective

  • Serafeim Moustakidis,
  • Athanasios Siouras,
  • Konstantinos Vassis,
  • Ioannis Misiris,
  • Elpiniki Papageorgiou and
  • Dimitrios Tsaopoulos

24 February 2022

CrossFit has gained recognition and interest among physically active populations being one of the most popular and rapidly growing exercise regimens worldwide. Due to the intense and repetitive nature of CrossFit, concerns have been raised over the p...

  • Article
  • Open Access
3 Citations
4,112 Views
35 Pages

Partitioning of Transportation Networks by Efficient Evolutionary Clustering and Density Peaks

  • Pamela Al Alam,
  • Joseph Constantin,
  • Ibtissam Constantin and
  • Clelia Lopez

24 February 2022

Road traffic congestion has became a major problem in most countries because it affects sustainable mobility. Partitioning a transport network into homogeneous areas can be very useful for monitoring traffic as congestion is spatially correlated in a...

  • Review
  • Open Access
43 Citations
10,983 Views
19 Pages

Machine Learning in Cereal Crops Disease Detection: A Review

  • Fraol Gelana Waldamichael,
  • Taye Girma Debelee,
  • Friedhelm Schwenker,
  • Yehualashet Megersa Ayano and
  • Samuel Rahimeto Kebede

24 February 2022

Cereals are an important and major source of the human diet. They constitute more than two-thirds of the world’s food source and cover more than 56% of the world’s cultivatable land. These important sources of food are affected by a varie...

  • Article
  • Open Access
38 Citations
7,192 Views
18 Pages

Machine Learning-Based Monitoring of DC-DC Converters in Photovoltaic Applications

  • Marco Bindi,
  • Fabio Corti,
  • Igor Aizenberg,
  • Francesco Grasso,
  • Gabriele Maria Lozito,
  • Antonio Luchetta,
  • Maria Cristina Piccirilli and
  • Alberto Reatti

23 February 2022

In this paper, a monitoring method for DC-DC converters in photovoltaic applications is presented. The primary goal is to prevent catastrophic failures by detecting malfunctioning conditions during the operation of the electrical system. The proposed...

  • Article
  • Open Access
4 Citations
4,226 Views
13 Pages

Reinforcement Learning for Mean-Field Game

  • Mridul Agarwal,
  • Vaneet Aggarwal,
  • Arnob Ghosh and
  • Nilay Tiwari

22 February 2022

Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously. After taking an action, the state of every agent updates to the next state, and...

  • Article
  • Open Access
10 Citations
4,968 Views
15 Pages

22 February 2022

A domain that has gained popularity in the past few years is personalized advertisement. Researchers and developers collect user contextual attributes (e.g., location, time, history, etc.) and apply state-of-the-art algorithms to present relevant ads...

XFacebookLinkedIn
Algorithms - ISSN 1999-4893