Next Article in Journal
Olfactory Detection of Toluene by Detection Rats for Potential Screening of Lung Cancer
Next Article in Special Issue
Identifying and Estimating the Location of Sources of Industrial Pollution in the Sewage Network
Previous Article in Journal
Adaptive Floor Cleaning Strategy by Human Density Surveillance Mapping with a Reconfigurable Multi-Purpose Service Robot
Previous Article in Special Issue
Evolutionary Algorithm for Improving Decision Tree with Global Discretization in Manufacturing
Article

Distributed Spectrum Management in Cognitive Radio Networks by Consensus-Based Reinforcement Learning

1
Artificial Intelligence Department, Vlatacom Institute, 11070 Belgrade, Serbia
2
Faculty of Technical Sciences, Singidunum University, 11000 Belgrade, Serbia
3
COPELABS, Universidade Lusófona de Humanidades e Tecnologias, 1749-024 Lisbon, Portugal
4
Department of Information Technologies, College of Applied Technical Sciences, 37000 Kruševac, Serbia
5
Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal
6
Faculty of Information Technology and Engineering, University Union Nikola Tesla, 11158 Belgrade, Serbia
*
Author to whom correspondence should be addressed.
This paper is an extended version of our paper published in: Dejan Dašić; Miljan Vučetić; Miroslav Perić; Marko Beko; Miloš S. Stanković. Cooperative Multi-Agent Reinforcement Learning for Spectrum Management in IoT Cognitive Networks. In Proceedings of the 10th International Conference on Web Intelligence, Mining and Semantics, Biarritz, France, 30 June–3 July 2020.
Academic Editor: Vassilis Plagianakos
Sensors 2021, 21(9), 2970; https://doi.org/10.3390/s21092970
Received: 26 March 2021 / Revised: 16 April 2021 / Accepted: 21 April 2021 / Published: 23 April 2021
(This article belongs to the Special Issue Machine Learning Applied to Sensor Data Analysis)
In this paper, we propose a new algorithm for distributed spectrum sensing and channel selection in cognitive radio networks based on consensus. The algorithm operates within a multi-agent reinforcement learning scheme. The proposed consensus strategy, implemented over a directed, typically sparse, time-varying low-bandwidth communication network, enforces collaboration between the agents in a completely decentralized and distributed way. The motivation for the proposed approach comes directly from typical cognitive radio networks’ practical scenarios, where such a decentralized setting and distributed operation is of essential importance. Specifically, the proposed setting provides all the agents, in unknown environmental and application conditions, with viable network-wide information. Hence, a set of participating agents becomes capable of successful calculation of the optimal joint spectrum sensing and channel selection strategy even if the individual agents are not. The proposed algorithm is, by its nature, scalable and robust to node and link failures. The paper presents a detailed discussion and analysis of the algorithm’s characteristics, including the effects of denoising, the possibility of organizing coordinated actions, and the convergence rate improvement induced by the consensus scheme. The results of extensive simulations demonstrate the high effectiveness of the proposed algorithm, and that its behavior is close to the centralized scheme even in the case of sparse neighbor-based inter-node communication. View Full-Text
Keywords: multi-agent reinforcement learning; consensus algorithm; cognitive radio networking; joint spectrum sensing and channel selection; distributed policy evaluation; distributed Q-learning; off-policy temporal difference multi-agent reinforcement learning; consensus algorithm; cognitive radio networking; joint spectrum sensing and channel selection; distributed policy evaluation; distributed Q-learning; off-policy temporal difference
Show Figures

Figure 1

MDPI and ACS Style

Dašić, D.; Ilić, N.; Vučetić, M.; Perić, M.; Beko, M.; Stanković, M.S. Distributed Spectrum Management in Cognitive Radio Networks by Consensus-Based Reinforcement Learning. Sensors 2021, 21, 2970. https://doi.org/10.3390/s21092970

AMA Style

Dašić D, Ilić N, Vučetić M, Perić M, Beko M, Stanković MS. Distributed Spectrum Management in Cognitive Radio Networks by Consensus-Based Reinforcement Learning. Sensors. 2021; 21(9):2970. https://doi.org/10.3390/s21092970

Chicago/Turabian Style

Dašić, Dejan, Nemanja Ilić, Miljan Vučetić, Miroslav Perić, Marko Beko, and Miloš S. Stanković 2021. "Distributed Spectrum Management in Cognitive Radio Networks by Consensus-Based Reinforcement Learning" Sensors 21, no. 9: 2970. https://doi.org/10.3390/s21092970

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop