Spectrum Sensing in Cognitive Radio Internet of Things: State-of-the-Art, Applications, Challenges, and Future Prospects

Abstract

The proliferation of Internet of Things (IoT) devices due to remarkable developments in mobile connectivity has caused a tremendous increase in the consumption of broadband spectrums in fifth generation (5G) mobile access. In order to secure the continued growth of IoT, there is a need for efficient management of communication resources in the 5G wireless access. Cognitive radio (CR) is advanced to maximally utilize bandwidth spectrums in the radio communication network. The integration of CR into IoT networks is a promising technology that is aimed at productive utilization of the spectrum, with a view to making more spectral bands available to IoT devices for communication. An important function of CR is spectrum sensing (SS), which enables maximum utilization of the spectrum in the radio networks. Existing SS techniques demonstrate poor performance in noisy channel states and are not immune from the dynamic effects of wireless channels. This article presents a comprehensive review of various approaches commonly used for SS. Furthermore, multi-agent deep reinforcement learning (MADRL) is proposed for enhancing the accuracy of spectrum detection in erratic wireless channels. Finally, we highlight challenges that currently exist in SS in CRIoT networks and further state future research directions in this regard.

1. Introduction

Wireless communication capacity is tremendously expanding due to the surge in the number of IoT devices requiring high data volumes. The fifth generation (5G) spectrum band is occupied daily with new IoT devices. Spectrum management becomes essential for continued communication of IoT devices. Cognitive radio (CR) is introduced to identify the idle spectrum of primary users (PUs) that can be assigned to secondary users (SUs) for communication, without interfering with the operation of the PUs [1]. PUs are the licensed and legitimate users of the spectral band. They have exclusive rights to the spectrum while SUs (IoT devices) are opportunistic users that feast on the idle bands of the PUs [2,3]. Spectrum sensing (SS) is an important aspect of CRs, which improves spectrum utilization in a wireless network. It routinely monitors the PUs for spectral holes that can be exploited by the SUs for communication. SS in the Cognitive Radio Internet of Things (CRIoT) network is faced with challenges such as dynamism in network environments, noise, interference, and fluctuations in signal strength. These factors impede signal detection in radio networks. Thus, it is essential for SS algorithms to learn the intricate patterns of the network environment, be robust to noise, and exhibit high memory in order to access idle spectrum of the PU [3,4,5,6].

Frantic efforts have been made by numerous researchers to develop novel sensing techniques for solving spectrum deficit problems in CRIoT networks. Among them are the conventional techniques such as energy detection (ED), which although simple to implement, struggles in noisy environments. Matched filter (MF) requires the knowledge of the environment as priori [7]. Cyclostationary feature detection (CFD) is less affected by noise but requires high computational resources. Pilot-based techniques demand constant transmission of pilot signals to detect the availability of the unused spectrum [8,9]. Statistical models like the Bayesian technique and hypothesis evaluation utilize prior knowledge of the PU to make an informed decision on the existence of a vacant spectrum [7]. Interestingly, the application of machine learning (ML) algorithms has attracted the interest of the authors in [10,11,12,13,14,15,16,17] for providing solutions to SS problems. The computational convergence and accuracy of the supervised ML methods are dependent on the quantity of datasets that are utilized for training the models, which is high in most cases. Other notable research works on SS are those due to the authors in [18,19,20,21,22,23,24,25,26], which are summarized in

Table 1. Summary of research/review articles on SS in the CRIoT network.

Reference	Type of the Article	SS Methods Adopted	Contribution/Findings	Remarks/Limitations
[18]	Research	Statistical method	The authors introduce a statistical covariance technique for SS, which is shown to outperform ED and CFD schemes.	The performance of the proposed technique degrades at low value of the threshold.
[19]	Research	Genetic algorithm	Genetic algorithm is proposed to optimize the sensing time and to reduce the probability of mis-detection.	It requires knowledge of the PU and channel condition, which may not be available prior to the design of the IoT network.
[20]	Research	CFD	The authors show that the probability of detecting the PU in a radio network is high if MIMO antennas are used.	The work fails to account for the imperfection in the wireless channel and does not demonstrate the application of the proposed technique in a real-world scenario.
[21]	Research	Pietra–Ricci index detection	A novel Pietra–Ricci index detector (statistical model) is utilized to detect the presence of the PU. The proposed technique is shown to exhibit less computational complexity when compared with CFD.	Goodness of fit test, which is critical for evaluating the performance of a statistical model is not considered by the authors.
[22]	Research	Multi-user antenna design system	The authors propose multi-user antenna system to detect the presence of the PU in a noisy region, where ED and CFD consider such conditions as signifying the absence of the PU signal.	The proposed technique falters at high-noise regimes of the wireless channel.
[23]	Research	PSO	The authors propose PSO to show that detection threshold, swarm size, and the number of iterations impact on spectrum detection and spectral efficiency of the wireless network.	The authors fail to validate the proposed technique by comparing its performance with the state-of-the-art techniques.
[24]	Research	Linear combination scheme	Proposes linear combination scheme for SS in slow, block, and fast-fading environments. The superiority of the proposed model to conventional linear combination schemes is demonstrated.	Mobility of wireless channel, which is synonymous with real-life applications is overlooked by the authors for ease of analysis.
[25]	Research	Soft combination scheme	The authors propose soft combination scheme to reduce the time required for detecting the vacant spectrum in the noisy conditions of the wireless channel. The superiority of the proposed technique to ED is demonstrated.	The performance of the proposed scheme degrades at high SNR values.
[26]	Review	__	Focusses the discussion on half and full duplex CR as well as SS in a wireless sensor network. CR applications in 5G and beyond network are discussed. Furthermore, the paper discusses challenges that need to be addressed in SS in the latter part of the article.	Discussion on a ML-based solution to SS problems in the CRIoT network is not presented.
[27]	Review	ED, CFD, MF, and wave form detection are discussed	Discussion is centered on the utilization of ED, CFD, MF, and waveform detection methods for reducing sensing time in CR networks.	ML-based solution to SS problems is not discussed.
This paper	Review	MF, CFD, Pietra–Ricci detection scheme and ML algorithms are discussed	Provides an overview of SS techniques employed in CRIoT networks. The techniques considered are MF, CFD, the Pietra–Ricci detection scheme, and ML algorithms. Proposes MADRL for SS in CRIoT networks. Furthermore, an overview of the studies on the application of ML algorithms is presented. In addition, challenges in implementation and open areas for further exploration are highlighted and discussed.

. The authors in [18] introduce a technique based on statistical covariance of the received signal, which performs better than ED. The performance of the proposed technique degrades at a low value of the threshold. A genetic algorithm is proposed by the authors in [19] to optimize the sensing time and to reduce the probability of miss detection. The technique requires prior knowledge of the PU and channel condition, which may not be available before the communication system is designed. In [20], the authors improve the accuracy of spectrum detection using CFD and a maximum ratio combining (MRC) diversity scheme. The research reveals that the probability of detecting the PU in a radio network is high if MIMO antennas are used. The work fails to account for imperfections in the wireless channel and does not demonstrate the application of the proposed technique in a real-world scenario. In [21], the author introduces a Pietra–Ricci index detection scheme for detecting the presence and the absence of the PU in a radio network. The detector shows better performance than traditional techniques like ED, CFD, and MF. A goodness of fit test, which is critical for evaluating the performance of statistical models, is not considered by the authors.

In [22], the authors propose a multi-user antenna system to detect the presence of the PU in noisy channel conditions, where ED and CFD consider such conditions as signifying the absence of the PU. The proposed technique falters at high-noise regimes of the wireless channel. In [23], particle swarm optimization (PSO) is utilized for optimizing the allocation of the spectrum to the SUs. The authors fail to validate the proposed technique by comparing its performance with the state-of-the-art techniques. In [24], the authors propose a linear combination scheme for SS in slow, block, and fast-fading environments. The scheme utilizes the mean and variance of the received signal to detect the unused spectrum band of the PU. Simulation results demonstrate the superiority of the proposed model to conventional linear combination schemes. Mobility of the wireless channel, which is synonymous with real-life applications, is overlooked by the authors for ease of analysis. In [25], the authors introduce a soft combination scheme for SS in a cooperative network by applying a likelihood ratio test to detect the PU in an environment, where the SNR is below −5 dB and whose performance is shown to be better than ED. The scheme utilizes less sensing overhead, but the performance of the soft combination scheme degrades at high values of SNR. In [26], a survey paper is presented on the latest achievements in SS. The authors discuss the use of CR in 5G and beyond broadband service. The authors end the paper by highlighting the challenges for future research. However, the review paper does not discuss ML-based solutions to SS problems in the CRIoT network.

Similarly, the authors in [27] present various approaches to the reduction in sensing time in CR networks. The proposed techniques for the task include ED, CFD, MF, and waveform detection schemes. ML-based solutions to SS problems are not discussed. This paper presents an overview of SS techniques employed in CRIoT networks. Comprehensive discussion is presented on an ML-based solution to SS problems. The article proposes multi-agent deep reinforcement learning (MADRL) for coping with the challenges posed by wireless channels and for enhancing spectrum detection results. Furthermore, challenges in implementation and prospects that may be explored as future research are highlighted and discussed.

The organization of the article is as follows: Section 1 introduces the paper; Section 2 focuses on the description of the CRIoT network; and Section 3 discusses SS in CRIoT networks. Section 4 describes ML-based techniques widely adopted for SS, which also features discussion on the proposed MADRL. An overview of the application of ML for SS in CRIoT networks is presented in Section 5, while the importance of SS in 5G and beyond networks is highlighted in Section 6. Lessons learnt and challenges of SS in CRIoT networks are highlighted in Section 7 and Section 8, respectively. Prospects that are expected to be addressed as future research are provided in Section 9 while we draw conclusions in Section 10.

2. Cognitive Radio Internet of Things

IoT has become a household name in recent years, owing to the numerous applications it performs. IoT interconnects everything via the internet. It provides a platform for sensors, humans, cars, machines, and communication gadgets, etc., to communicate and connect. It fosters smart things such as smart health, smart agriculture, smart health, smart transportation, and smart building. The literature has documented various applications of IoT [28,29,30,31,32,33,34,35,36]. Figure 1 presents a smart world that is supported by ubiquitous 5G and beyond broadband internet access. The rapid growth of IoT has put enormous pressure on the radio spectrum in 5G wireless access. The quantity of spectral data that is processed daily has escalated [37] and there is increasing demand for more spectrums to cater for emerging IoT devices. CR is put forward to proffer solutions to spectrum deficit problems in wireless networks. The combination of CR and IoT is intended to bridge the spectrum gap in the radio networks by identifying spectral holes or unoccupied spectrums of primary communication such as mobile phones, Wi-Fi nodes, and WiMax (PUs), which can be used by IoT devices for communication. The research on CR has become a hot topic in academic and industrial arenas. This is due to its potential in supporting and sustaining the operation of IoT devices [38]. The front runner in this research is Mitola and his associate who introduced CR in 1999 for effective management and better utilization of the radio spectrum [39]. The advantages of CR in the IoT network include the following:

(i)

Handling the dynamic nature of the network environment: The wireless channel in which IoT devices operate changes dynamically, with the PU signal varying over time and space. Large-scale and flat fading, shadowing, and noise are common occurrences in the network environment. CR has the capability to intelligently switch IoT devices to the frequency band of wireless channel with less noise, coping with the stochastic nature of the network environment. In other words, CR can adapt to spatial temporal characteristics of wireless channels.

(ii)

Utilization of idle and underutilized spectrum: CR senses, adapts, and accesses spectral bands that are not used by the PUs, culminating in better utilization of the spectrum. CR also ensures that the SU hops to another frequency band if its occupation of certain band affects the operation of the PU. This prevents interference and disruption of the operation of the PU [2,3].

(iii)

Spectrum sharing: Another important aspect of CR is spectrum sharing, where the unused spectra are shared among the SUs without posing danger to the primary communication [3].

(iv)

Varying link capacity resistance: With CR, SUs can change to the frequency band with higher link capacity when there are changes in the channel’s condition and the operation of the PU is endangered.

3. SS in CRIoT Networks

SS in CRIoT networks takes different forms, as illustrated in Figure 2, which includes non-cooperative, cooperative, and interference-based approaches. In non-cooperative SS (NCSS), individual SU accesses the PU independently and there is no cooperation between the SUs. This approach is affected by shadowing, fading, and noise interference. Cooperative SS (CSS) is divided into centralized and distributed CSS. In centralized CSS, several SUs send sensing signals to the PU to obtain detection information [40].

The information received is combined at the central location, called the control center, as illustrated in Figure 3. The final decision is made by comparing detection information with the preset threshold. The PU is said to be active and the band is occupied if the detection information at the control center is greater than the threshold, otherwise, the band is vacant.

In the distributed CSS, there is no control center, each SU shares its detection information and interprets the decisions of other SUs. It utilizes large communication overheads and the approach burdens the communication network [41]. In interference-based SS, the PU’s signal is detected by checking its interference with the SU. It is assumed that if one of the SUs interferes with the PU signal, such SU is within the communication range of the PU and the PU is detected [41,42]. If there is no interference, the SU is not within the communication bandwidth of the PU. This indicates that the PU band is free. Generally, CSS improves the accuracy of signal detection, owing to spatial diversity of the location of the SUs. Unlike NCSS, CSS is robust to shadowing, fading, and changes in channel condition. CSS has received major attention in the literature on account of its robustness to the dynamic nature of wireless channels.

The techniques commonly used for SS include ED, CFD, MF, Pietra–Ricci index detection, and ML-based techniques. These methods are applicable in any of the SS approaches and are presented in what follows.

3.1. Energy Detection (ED)

ED has received much attention due to its ease of implementation. It utilizes moderate computational effort and does not require prior knowledge of the PU [8]. The average energy of the received signal may be computed via Equation (1) [8] as

$$E={\displaystyle \frac{1}{M}}{\displaystyle \sum _{m=1}^M{\left|X\left(m\right)\right|}^2}$$

(1)

where E is the energy of the received signal, $X\left(m\right)$ is the received signal, and $m=1,2,\cdots ,M$ is the length of the received signal. In implementing ED for sensing the PU, the null, and alternative hypotheses are formulated as Equation (2):

$$\left[\begin{array}{l}{H}_0:X\left(m\right)=V(m)\\ H_1:X\left(m\right)=H\ast Y\left(m\right)+V(m)\end{array}\right.$$

(2)

for which, $V\left(m\right)$ is the Additive White Gaussian Noise (AWGN), $H$ is the channel matrix of the wireless medium, consisting of the channel gain that is independently and identically distributed, and $Y\left(m\right)$ is the transmitted symbol from the PU. The null hypothesis $H_0$, indicates the absence of the PU when noise is received, while $H_1$ indicates that the PU is active and present. ED compares the detection information (energy of received signal) with a threshold, forming a decision rule as Equation (3):

$$\left[\begin{array}{l}{H}_0:E<threshold,PU is inactive\\ H_1:E>threshold,PU is active\end{array}\right.$$

(3)

It is implied in Equation (3) that the PU is active if the detection information is greater than the threshold while the PU is inactive if the detection information is lower than the threshold.

The performance of SS techniques is evaluated over a number of metrics including probability of detection $P_D$, probability of missed detection $P_{MD}$, probability of false alarm $P_{FA}$, and accuracy. $P_D$ refers to the probability of asserting that the PU is active when it is actually transmitting. This parameter is expected to be high to minimize interference with the operation of the PU. $P_{MD}$ refers to the probability of declaring that the PU is absent, when in actual fact, it is transmitting. The value of $P_{MD}$ should be minimal in order to prevent interference between the PUs and the SUs. $P_{FA}$ is concerned with the chances of declaring that the PU is active, when in actual fact, its band is unoccupied. The value of $P_{FA}$ is expected to be insignificant to avoid loss of spectral bands that may be utilized by SUs for communication. Accuracy refers to the condition that the PU is detected to be active and its band is actually occupied, as well as the condition that the PU is detected to be inactive, when its band is free. For ED, $P_D$ and $P_{FA}$ are expressed by Equations (4) and (5) [8] as

$$P_D=\mathbb{Q}\left({\displaystyle \frac{{\lambda }_{ED}-M(1+\alpha )}{\sqrt{2M\left(1+{\lambda }_{ED}\right)}}}\right)$$

(4)

$$P_{FA}=\mathbb{Q}\left({\displaystyle \frac{{\lambda }_{ED}-M{\sigma }_v^2}{\sqrt{2M{\sigma }_v^4}}}\right)$$

(5)

where $\mathbb{Q}$ is the Gaussian function, ${\sigma }_v$ is the noise variance, $\alpha$ is the signal-to-noise ratio (SNR), and ${\lambda }_{ED}$ is the sensing threshold, which admits expression of the form given by [8] as

$${\lambda }_{ED}=\left({\mathbb{Q}}^{-1}\left(P_{FA}\right)\sqrt{2M}+M\right){\sigma }_v^2$$

(6)

In Equation (6), it is seen that ${\lambda }_{ED}$ is a function of noise power, probability of false alarm, and the length of the transmitted signal from the PU. ED is affected by constant changes in wireless channels and suffers a dip in performance at low SNR profiles of the wireless medium. The demonstration of ED for SS has been presented by the authors in [8,9,43].

3.2. Cyclostationary Feature Detection (CFD)

An alternative spectrum detector is CFD. It senses the PU by exploiting the periodic features of the modulated signal. In CFD, it is assumed that the noise is a stationary signal without correlation, while the modulated signal is cyclostationary with spatial correlation due to the redundancy of the signal periodicity [41]. CFD is robust to noise and it is less affected by fading and shadowing [8]. The cyclic autocorrelation function of the received signal admits an expression of the form given by Equation (7) [44] as

$$A_{xx}=\underset{T\to \infty }{\lim }{\displaystyle \frac{1}{T}}{\displaystyle \underset{-T/2}{\overset{T/2}{\int }}x\left(t+\frac{\tau }{2}\right)}\ast x\left(t-{\displaystyle \frac{\tau }{2}}\right)$$

(7)

where $T$ is the period, $x\left(t\right)$ is the received signal, $\tau$ is the time delay, $\ast$ is the symbol for correlation, and ${\mathrm{A}}_{\mathrm{xx}}$ is the autocorrelation function. The spectral correlation density, obtained from the Fourier transform of Equation (7), admits expression of the form given by Equation (8) [44] as

$$S_{xx}={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{\mathrm{A}}_{\mathrm{xx}}{e}^{-j2\pi f\tau }d\tau }$$

(8)

The received signal exhibits cyclostationarity if ${\mathrm{A}}_{\mathrm{xx}}$ is not equal to zero, else, the received signal exhibits stationarity [44]. When ${\mathrm{A}}_{\mathrm{xx}}$ is equal to zero, it is believed that noise is transmitted from the PU. CFD is implemented by forming the null and alternative hypotheses, as demonstrated by Equation (9):

$$\left[\begin{array}{l}{H}_0:x(t)=n(\mathrm{t})\\ H_1:x\left(t\right)=H\ast y\left(t\right)+n(\mathrm{t})\end{array}\right.$$

(9)

in which $n\left(t\right)$ is the noise, $y\left(t\right)$ is the transmitted signal from the PU, and $H$ is the channel matrix. The availability and the absence of the PU is known via decision rule, expressed by Equation (10) as

$$\left[\begin{array}{l}{H}_0:A_{xx}<threshold,PU is inactive\\ H_1:A_{xx}>threshold,PU is active\end{array}\right.$$

(10)

It is seen in Equation (10) that the PU is sensed to be active if the autocorrelation function is greater than the threshold, else, the PU is inactive. An illustration of SS via CFD is depicted in Figure 4.

CFD is computationally expensive and its implementation takes longer time [8,43,44]. The application of CFD for SS has been presented by the authors in [8,9,20,23,43,44,45].

3.3. Matched Filter (MF)

MF requires information about the PU a priori for implementation. Information, such as packet format and modulation type and order, is critical for sensing the spectral hole of the PU [41]. The method converges faster than CFD and consumes less computational resources [46]. MF can be described by the expression of the form given by Equation (11) as

$$Y\left[n\right]={\displaystyle \sum h\left(n-k\right)}x\left(n\right)$$

(11)

$x\left(n\right)$ is the transmitted signal from the PU, which is convolved with the channel impulse response that is delayed in k unit time, and $Y\left[n\right]$ is the test static that is used to make decision on the availability of the PU. The null and alternative hypotheses for MF are expressed by Equation (12) as

$$\left[\begin{array}{l}{H}_0:y(\mathrm{n})=w(\mathrm{n})\\ H_1:y\left(n\right)=h(\mathrm{n})\ast x\left(n\right)+w(\mathrm{n})\end{array}\right.$$

(12)

where $w\left(n\right)$ is the noise. The test statistic is compared with the threshold and the decision is formed such that Equation (13) holds as follows:

$${\left[\begin{array}{l}{\mathrm{H}}_0:\mathrm{Y}\left[\mathrm{n}\right]<\mathrm{threshold},\mathrm{PU}\mathrm{is}\mathrm{absent}\\ {\mathrm{H}}_1:\mathrm{Y}\left[\mathrm{n}\right]>\mathrm{threshold},\mathrm{PU}\mathrm{is}\mathrm{present}\end{array}\right.}^{\circ }$$

(13)

An illustration of this process is shown in Figure 5. $P_D$ and $P_{FA}$ are expressed by Equations (14) and (15) [46] as

$$P_D=\mathbb{Q}\left({\displaystyle \frac{{\lambda }_{MF}-E)}{\sqrt{E{\sigma }_w^2}}}\right)$$

(14)

$$P_{FA}=\mathbb{Q}\left({\displaystyle \frac{{\lambda }_{MF}-E)}{{\sigma }_w^{2}E}}\right)$$

(15)

for which, ${\lambda }_{MF}$ is the sensing threshold, E is the energy of the PU signal, ${\sigma }_w^2$ is the noise variance, and $\mathbb{Q}$ remains as defined earlier. The sensing threshold admits an expression of the form given by (16) [46] as

$${\lambda }_{MF}={\mathbb{Q}}^{-1}\left(P_{FA}\right)\sqrt{{\sigma }_w^{2}E}$$

(16)

MF requires prior knowledge of the PU, and it exhibits poor results if the information from the PU is not accurate [8,47]. The application of MF for SS has been presented by the authors in [8,46,47,48,49].

3.4. Pietra–Ricci Index Detection

Guimaraes in 2020 adapted the Pietra–Ricci index commonly used in economics and social sciences for detecting the vacant spectrum of the PU in a cooperative radio network [21]. Just like ED, CFD, and MF, the detection performance of this technique is tied to a threshold. The detection information (test static) of the Pietra–Ricci index scheme is expressed by Equation (17) [21] as

$$T={\displaystyle \frac{2{\displaystyle \sum _{j=1}^{n^2}\left|y_j\right|}}{{\displaystyle \sum _{j=1}^{n^2}\left|y_j-\overline{y}\right|}}}$$

(17)

in which $T$ is the test static, $y_j$ is the $j^{-th}$ element of the column vector of the received signal, $n$ is the number of SUs involved in SS, and $\overline{y}=1/n^2{\displaystyle \sum _{j=1}^{n^2}{y}_j}$. The method detects the presence and the absence of the PU by comparing $T$ with the threshold, where the PU is assumed to be present and active if $T$ is greater than the threshold, otherwise, the PU is absent and inactive. A demonstration of Pietra–Ricci index detection for SS can be found in [21]. The performance of this technique depends on the value of the threshold. Poor choice of the threshold may give false pretenses about the condition of the PU.

The weaknesses of traditional techniques, which include poor performance in low SNR, demand for partial or full knowledge of the PU, and high sensing time rejuvenate the interest of researchers to consider ML algorithms, to reduce spectrum detection time and enhance the accuracy of spectrum detection results without prior knowledge of the PU. ML techniques devoted to SS are discussed in what follows.

4. ML-Based SS Techniques

ML is adopted in the modern era to address problems in health, education, disaster management, agriculture, school management, and fraud detection [50,51,52,53,54,55,56,57,58]. Its use case has also been extended to telecommunication to address spectrum deficits in CRIoT networks, where SS is treated as a classification problem. The focus of this section is to furnish the readers with ML techniques suitable for SS in CRIoT networks. It is worthy to note that we do not provide an in-depth analysis of ML models because a comprehensive report on them can be found in [59,60,61,62]. ML is divided into various classes, as illustrated in Figure 6. These classes include unsupervised learning, supervised learning, reinforcement learning, and deep learning [63].

4.1. Supervised Learning

In this ML technique, an algorithm is utilized to learn from labeled data for predicting an outcome. The labeled data consist of input–output pairs, which enables the supervised learning algorithm to learn mapping functions that can be applied to raw, unseen, and unlabeled inputs. Common applications of supervised ML are image recognition and detection, spam filtering, classification, and regression analysis [64,65]. Examples of supervised ML algorithms are support vector machine (SVM), logistic regression (LR), decision tree, random forest (RF), Naïve Bayes (NB), and K-nearest neighbor (KNN), etc.

4.1.1. Support Vector Machine (SVM)

In SVM, kernel functions convert the input datasets into high-dimensional spaces. Flat subspace (hyperplane) is utilized to separate data into different classes. This makes it easier to execute classification and regression tasks. Kernels that are used in SVM include linear, radial basis, cubic, and quadratic functions. A linear kernel function is expressed in a form given by Equation (18) [66] as

$$L\left(E\right)=w^{T}E+a$$

(18)

where $L\left(·\right)$ represents the linear kernel function, $w$ is the weight vector, $a$ is the bias, $E$ is the energy of the received signal that is being sensed, and ${\left(·\right)}^T$ denotes the transpose of a vector. The output of the SVM may be expressed by Equation (19) [66] as

$$\underset{w\in R^d}{\min }{‖w‖}^2+C{\displaystyle \sum _{n=1}^N\max \left(0,1-x_{i}L\left(E_i\right)\right)}$$

(19)

where in Equation (19), an optimization problem is solved over a given weight $w$ to produce output $x_i$ for $i^{-th}$ input element, $C$ is the regularization constant, and $E_i$ is the detection information for $i^{-th}$ element of the input dataset. In an SVM, the PU signal is detected if $E_i$ exceeds the threshold, otherwise, the PU is inactive [67]. The authors in [67] demonstrate that an SVM exhibits better accuracy than the Gaussian Mixture Model (GMM) in sensing the spectral hole of the PU.

4.1.2. Logistic Regression

This technique predicts the probability that an input dataset belongs to a specific class. By using a sigmoid linear function, the algorithm categorizes a real-valued set of independent variables into probability values of 0 and 1. The input features of a given dataset may be expressed in matrix form as Equation (20):

$$Y=\left(\begin{array}{ccc}{y}_{11}& \cdots & y_{1n}\\ ⋮& \ddots & ⋮\\ y_{m1}& \cdots & y_{mn}\end{array}\right)$$

(20)

The independent variable X, consisting of binary values is given by Equation (21) as

$$X=\left[\begin{array}{l}0 if class 1\\ 1 if class 2\end{array}\right.$$

(21)

By applying a linear function to the input variable Y, we have an expression of the form given by Equation (22) as

$$G=\left({\displaystyle \sum _{k=1}^N{w}_k{y}_k}\right)+a$$

(22)

where $y_k$ is the $k^{-th}$ observation data of Y, $w_k=\left[w_1,w_2,w_3\cdots ,w_N\right]$ is the weight, and $a$ is the bias. The algorithm employs a sigmoid function to transform G into probability values of 0 and 1, which are used to predict the presence and the absence of the PU. That is,

$$\sigma \left(G\right)={\displaystyle \frac{1}{1+e^{-G}}}$$

(23)

where $\sigma \left(G\right)$ is the linear sigmoid function on G. It is deduced from Equation (23) that $\sigma \left(G\right)$ tends towards 1 as $G\to \infty$ and $\sigma \left(G\right)$ tends towards 0 as $G\to -\infty$. This means that the boundary of $\sigma \left(G\right)$ is between 0 and 1. The probability of belonging to a class may be expressed by Equation (24) as:

$$P\left(X=1\right)=\sigma \left(G\right),P\left(X=0\right)=1-\sigma \left(G\right)$$

(24)

4.1.3. Decision Tree

In this learning scheme, input datasets are partitioned based on different features into a tree-like form. The tree has four nodes: the root node, the branches, the internal node, and the leaf node. The root node is the starting point of the tree, which represents the features of the dataset. The branch connects the nodes. It represents possible values that a node can predict. The internal node represents further decisions based on the previous branch. The leaf node provides the final outcome of the classification. The authors in [66] build the decision tree by using Dichotomiser 3 (ID3), where the gain and entropy function are employed as metrics to classify the training data for SS.

4.1.4. Random Forest (RF)

This is an ML algorithm that utilizes an ensemble of decision trees to predict an outcome. It handles complex datasets and provides high accuracy of data classification. It is a popular choice for both regression and classification work. It creates random vectors, where each tree in the forest is trained on a random subset of the input data. This helps to minimize overfitting and enhance generalization. Each tree contributes to make a decision. It combines the decisions of multiple trees to predict an outcome [68], having an edge over decision trees. The prediction of the presence and the absence of the PU using RF may be expressed in a form given by Equation (25) as:

$$Outcome={\displaystyle \frac{1}{\mathrm{A}}}{\displaystyle \sum _{n=1}^A{f}_n\left[E\right]}$$

(25)

where E is the detection information concerning the status of the PU, $f_n$ is the classification function, and A is the number of trees that are involved in the decision.

4.1.5. K-Nearest Neighbor (KNN)

In this learning model, classification is performed based on the similarity between the instances of input datasets, rather than using the underlying features of the input data. It is assumed in many quarters to be a lazy learner. It makes predictions by memorizing the input data. The norm of the instances of the testing and training data may be expressed by Equation (26) as:

$$X={‖{\displaystyle \sum \left(f_m-f_n\right)}‖}^2$$

(26)

in which $X$ is the similarity that defines the Euclidean distance between the instances of testing data $f_n$, and instances of training data $f_m$. Given a positive integer $K$, whose value depends on the experimentation and cross validation, the algorithm identifies $K$ points in the training data that are closest to the testing data.

4.1.6. Naïve Bayes

This learning technique utilizes Bayes’ theorem to predict the likelihood that a data point belongs to a specific class, assuming that all the features of the data are unrelated or independent. The Bayes’ theorem for calculating the posterior probability may be defined by Equation (27) as

$$P\left(b/a\right)={\displaystyle \frac{P\left(a/b\right).P\left(b\right)}{P\left(a\right)}}$$

(27)

where $P\left(b/a\right)$ is the posterior probability; the probability of $b$, given features of $a$, $P\left(a/b\right)$ is the likelihood that defines the probability of features $a$, given class $b$; $P\left(b\right)$ is the prior probability of class $b$ and $P\left(a\right)$ is the marginal likelihood. In using the model for SS, class $b$ is assumed to represent the detection information concerning the absence and the presence of the PU and $a$ represents the SU that is observing the PU. Using the Naïve assumption, where each SU in IoT network is independent, it is written that

$$P\left(a_1,a_2,a_3,\cdots ,a_n/b\right)=P\left(a_1/b\right)·P\left(a_2/b\right)\cdots P\left(a_n/b\right)$$

(28)

Consequently, Bayes’ theorem of Equation (27), transforms to expression of the form given by Equation (29) as

$$P\left(b/a_1,a_2,a_3,\cdots ,a_n\right)={\displaystyle \frac{P\left(b\right)·{\displaystyle \prod _{i=1}^{n}P\left(a_i/b\right)}}{P\left(a_1\right),P\left(a_2\right),\cdots P\left(a_n\right)}}$$

(29)

The model provides information about the PU by calculating the highest posterior probability of each state of the PU, using Equation (30) that is written as

$$\widehat{b}=\underset{b}{\arg \max }\left(P\left(b\right)·{\displaystyle \prod _{i=1}^{n}P\left(a_i/b\right)}\right)$$

(30)

The state of the PU with the highest probability is chosen as the one representing the current status of the PU.

4.2. Unsupervised Learning

In unsupervised ML, an algorithm is utilized to learn from unlabeled data without a target or expected output. The aim of unsupervised learning is to enunciate hidden patterns, structures, and relationships within the data. Common tasks in unsupervised ML are clustering, dimensionality reduction, and anomaly detection as well as associate rule learning. It has useful applications in image analysis and fraud detection, among others. Its accuracy is low because of the absence of input and output data pairs; it requires large computational resources [69]. Examples are K-means clustering, Bayesian learning, principal component analysis (PCA), and independent component analysis (ICA).

4.2.1. K-Means Clustering

This learning model divides unlabeled data into K-point clusters. It assigns each data point in the cluster to the nearest centroid based on the chosen Euclidean distance. The centroids are taken as the mean of all the data points in the cluster. This process is repeated until all the cluster assignments no longer change or a certain number of iterations is reached. In determining the value of K, the elbow technique is utilized. The authors in [70] have demonstrated the application of K-means clustering for SS in the radio network.

4.2.2. Bayesian Learning

This is an unsupervised ML technique that learns hidden patterns in unlabeled data. Bayesian learning determines the likelihood function that quantifies the probability of observing the data, given the chosen model and its parameters. It assigns data points to different clusters and identifies patterns based on the learned model parameters. The GMM is a powerful probabilistic model that is used in Bayesian learning to define data features such as the number of clusters or the connection between the variables, cluster’s shape, and location. With the GMM, each data point is assigned to the cluster with the highest probability. The algorithm utilizes Bayes’ theorem to understand the pattern and the distribution of the data. The parameters of the GMM are refined iteratively by using the posterior distribution from previous iteration as the prior for the subsequent iteration. Through this process, the algorithm learns and improves its understanding of the underlying features of the input datasets.

4.3. Reinforcement Learning (RL)

In reinforcement learning (RL), the agent explores the environment, makes decisions and receives feedback in the form of a reward or penalty, learning through trial and error. RL is classified into the Markov decision process (MDP), multi-armed bandit, dynamic programming, and temperature difference learning [71]. We do not wish to repeat the discussion of these classes because a comprehensive report on them can be found in [72,73,74,75,76,77,78,79,80]. The following terms are pertinent to the understanding of RL and its deployment for SS.

(i)

Agent: This is a device that explores the environment.

(ii)

Environment: This refers to the circumstances in which the agent operates, providing states and actions.

(iii)

State: This describes the current status of the environment as seen by the agent.

(iv)

Action: It refers to the choices that are available to the agent in an environment.

(v)

Reward: It denotes the feedback received based on the actions of the agent.

(vi)

Policy: This is the strategy that guides the agent’s behavior. It maps states to actions.

(vii)

Value function: This measures the cumulative reward an agent receives by following a particular policy.

(viii)

Model: This is a representation of the environment that the agent explores to forecast the outcome of its actions.

The basic algorithm for training RL is presented in Algorithm 1.

Algorithm 1: Algorithm for training R.

#step 1: start #step 2: define the environment #step 3: state the reward #step 4: define the agent #step 5: train or validate the agent #step 6: implement the policy #step 7: end

An illustration of MDP for identifying idle spectrums of the PU is presented in Figure 7. In the illustration, an agent (the SU) in a particular state $S_i$, takes action ${\mathrm{A}}_i$, to access the spectral band of the PU (environment). Consequent upon this task, the agent receives feedback in the form of reward $R_i$. The reward measures the effect of the selected action of the SU’s state [81]. The state transitions to the next state $S_{i+1}$, and the SU receives the reward $R_{i+1}$, for the action of the new state following a policy $\beta$. The policy determines how the SU takes an action in different states. The SU continues to interact with the PU and receives the reward for every action taken. Through this process, the SU learns the characteristics of the environment and observes whether the PU band is vacant or not. The reward may be modeled to minimize spectrum sensing time.

The cumulative reward of the state $S$ under policy $\beta$ when agent’s first state is $S_i$, is defined by [82] as Equation (31):

$$v_{\beta }={\mathrm{E}}_{\beta }\left[{\displaystyle \sum _{i=0}^{\infty }{\gamma }^{i}R\left(S_i,{\mathrm{A}}_{i,}{S}_{i+1}\right)\left|S_i=S\right.}\right]$$

(31)

where $\gamma$ is the discount rate, which measures agent’s discounted future reward.

The objective of RL is to find an optimal policy that generates maximum reward for every action taken by the agent. Thus, the action value, also referred to as the Q-value can be expressed by Equation (32) [82] as:

$${\mathbb{Q}}_{\beta }\left(S,\mathrm{A}\right)={\mathrm{E}}_{\beta }\left[{\displaystyle \sum _{i=0}^{\infty }{\gamma }^{i}R\left(S_i,{\mathrm{A}}_{i,}{S}_{i+1}\right)\left|S_i=S,{\mathrm{A}}_i=\mathrm{A}\right.}\right]$$

(32)

Equation (32) represents the expected value for the agent’s action $\mathrm{A}$, under state $S$. The optimal policy, given a set of action values of a particular state, is defined by Equation (33) as:

$${\beta }^{\ast }\left(S\right)=\underset{\mathrm{A}}{\arg \max }\left\{{\mathbb{Q}}_{\beta }^{\ast }\left(S,\mathrm{A}\right)\right\}$$

(33)

where ${\beta }^{\ast }\left(S\right)$ is the optimal policy of the state S and ${\mathbb{Q}}_{\beta }^{\ast }\left(S,\mathrm{A}\right)$ is the optimal Q-value function, which is given by Equation (34) as:

$${\mathbb{Q}}_{\beta }^{\ast }\left(S,\mathrm{A}\right)=\mathrm{P}\left(S\prime /\mathrm{S},\mathrm{A}\right){\displaystyle \sum _{S\prime }R\left(S,\mathrm{A},S\prime \right)}+v_{{\beta }^{\ast }}\left(S\prime \right)$$

(34)

The cumulative reward of the agent in state S taking action A under optimal policy, admits an expression of the form given by Equation (35) [82] as:

$$v_{{\beta }^{\ast }}\left(S\right)=\underset{\mathrm{A}}{\arg \max {\mathbb{Q}}_{\beta }^{\ast }\left(S,\mathrm{A}\right)}=\underset{\mathrm{A}}{\arg \max } \mathrm{P}\left(S\prime /\mathrm{S},\mathrm{A}\right)\left({\displaystyle \sum _{S\prime }R\left(S,\mathrm{A},S\prime \right)}+v_{{\beta }^{\ast }}\left(S\prime \right)\right)$$

(35)

The algorithms for maximizing the cumulative reward in the MDP may be model-based or model-free. The former requires not only high computational operation but also perfect knowledge of the environment, which may not be available when the communication link is designed. The applications of model-based algorithms are limited [83] and are rarely used for SS. Examples of algorithms that fall under this category are Dyna, model-based policy optimizer (MBPO), Dreamer, and probabilistic inference for learning control (PILCO). Model-free algorithms are commonly used, owing to their faster rate of convergence and ease of implementation [84]. They require less computational overheads and plan their next action by learning from the environment [84]. Examples are Q-learning, actor–critic, state–action–reward–state–action (SARSA), the Monte Carlo model, and the policy gradient method. These learning models are also classified as temperature difference algorithms [83]. To receive the maximum reward in Q-learning for example, the agent takes random actions and updates the Q-value function of each action by using expression of the form given by Equation (36) [82] as:

$$\mathbb{Q}\left(S^i,{\mathrm{A}}^i\right)\leftarrow \left(1-\delta \right)\mathbb{Q}\left(S^i,{\mathrm{A}}^i\right)+\delta \left(R^{i+1}+\gamma \underset{\mathrm{A}}{\max } \mathbb{Q}\left(S^{i+1},{\mathrm{A}}^i\right)\right)$$

(36)

in which $\delta$ is the learning rate.

RL is useful for SS in CRIoT networks because it requires little training data for learning the dynamic change in status of the primary channel and coping with the non-stationarity of wireless channel.

4.4. Deep Learning (DL)

DL utilizes an artificial neural network (ANN) with many layers to analyze and learn complex patterns in input data. Figure 8 shows an ANN with zero hidden layers that accepts a number of inputs $\left(x_1,x_2,x_3,\dots ,x_n\right)$, sums the weighted input and bias $b$, and passes the result as input argument to an activation function, to give output $y$. This process may be described by the expression of the form given by Equation (37) [85] as:

$$y=\phi \left(\left[w_1,w_2,w_3,\dots ,w_n\right]\left[\begin{array}{l}{x}_1\\ x_2\\ x_3\\ ⋮\\ x_n\end{array}\right]+b\right)=\phi \left({\displaystyle \sum _{n=1}^N{w}_n{x}_n}+b\right)$$

(37)

Provided that $\phi$ symbolizes activation function, $b$ is the bias and $w$ is the synaptic weight.

The weights of the ANN are updated by minimizing the error between the network output and the expected output or target, using the mean square error (MSE), stochastic gradient descent (SGD), and root mean square propagation (RMSProp.) [86]. Vanishing gradient, overfitting, and slow convergence occur when an ANN with many hidden layers is utilized to handle a complex task. This necessitates the need for DL architecture, consisting of multiple hidden layers to overcome the limitations of the ANN. A DNN avoids overfitting by training some randomly selected nodes instead of the entire network. This is called dropout. DL allows 20%, 25%, and 50% dropout in the hidden layers, which in turn reduces the time required in training the network. DL is divided into supervised and unsupervised learning as well as deep reinforcement learning (DRL). Supervised DL is subdivided into multilayer perceptron (MLP), convolutional neural network (CNN), and recurrent neural network (RNN). An example of unsupervised DL is the autoencoder.

4.4.1. Multilayer Perceptron

This is a feed-forward DNN that consists of fully connected layers. An illustration of MLP architecture for SS is shown in Figure 9. The illustration depicts a CRIoT network with channel coefficient $\left(H_1,H_2,H_3,\cdots ,H_N\right)$ from the PU to the SUs $\left(S{U}_1,S{U}_2,S{U}_3,\cdots ,S{U}_N\right)$. The control center combines sensing information from the SU, which serves as input into the MLP architecture. After repeated process of activation at different hidden layers and updating the weights, the network generates the corresponding output $y$ that provides information about the presence and the absence of the PU in a wireless network. MLP suffers from vanishing gradient and overfitting due to a large number of hidden layers in the network architecture. A number of solutions have been proposed to address this problem, including early stoppage of the training, dropout, and regularization [63].

4.4.2. Convolutional Neural Network

MLP falters when it is utilized to handle huge and nuanced input data like image signals. It experiences the problem of dimensionality due to full connection of neurons from one layer to the other [63]. These concerns birth the introduction of a CNN to extract features in complex signals for classification and regression tasks. A CNN consists of convolution, pooling, and fully connected layers. The convolution layer employs kernels to extract the feature map in the input data. The pooling layer reduces the spatial dimension of the extracted feature map while the fully connected layer performs classification or regression duties. The output layer generates the output of the network. CNNs find useful applications in image classification, object detection, image denoising, image recognition, and segmentation. Popular types are residual neural networks (ResNet) AlexNet, VGG-16, GoogleNet, and LeNet [87,88,89,90]. CNNs capture the spatial characteristics of the PU signal by recognizing the intricate pattern of the radio spectrum, improving spectrum detection in the wireless network [8]. It combats the vanishing gradient of MLP through normalized initialization and batch normalization. To overcome overfitting and achieve good performance, CNNs require high amounts of labeled training data. The training process involves high-density calculations in each layer of the CNN and gradients are determined for every parameter using backpropagation [87]. Given that a training session may involve thousands of iterations on a large quantity of training datasets, the entire computational complexity can be immense for generating SS results in an IoT network.

4.4.3. Recurrent Neural Network

In this learning method, the temporal features of input datasets are captured. It excels in handling tasks that require memory or the need for recalling past information. In RNN architecture, the output is fed back into the network as an input, creating a closed-loop system. This allows the architecture to have memory about previous inputs. The memory characteristic of an RNN suits the model for handling sequential data in time series form [9]. Owing to its ability to recall past events, it is a valuable tool in predicting historical data concerning the present and the future spectrum usage of the PU in CRIoT networks [9]. The author in [91] describes the output of RNNs as Equation (38):

$$y_i={\phi }_y\left(D_y{h}_i+k_i\right)$$

(38)

where $y$ is the output vector, $\phi$ denotes activation function, $k$ is the bias vector, $D$ is the weighted matrix for hidden to output layer connection, and $h$ represents the hidden layer vector, expressed by Equation (39) [92] as:

$$h_i={\phi }_y\left(W_h{a}_i+U_h{h}_{i-1}+k_i\right)$$

(39)

where $W$ is the weighted matrix for hidden to hidden connection, $U$ is the weighted matrix for input to hidden layer connection, and $a$ is the input vector. Examples of RNNs are LSTM, gated recurrent unit (GRU), and bidirectional RNN. RNNs require a high volume of datasets for training. The computational complexity of RNNs is high because of two main reasons: the difficulties associated with training a large number of datasets and the sequential manner of processing the data, which limits parallel processing. The application of LSTM for SS has been demonstrated by the authors in [92]. To the best of the authors’ knowledge, no work has been reported in the literature on the application of GRU and bidirectional RNN for SS.

4.4.4. Autoencoder

This is an unsupervised DL technique that learns the essential features in input dataset by projecting the input data into smaller dense structures called latent spaces. It learns to extract essential features from the input data by training the model to reduce the error between the input and the output [93]. The autoencoder architecture for SS in the cooperative IoT network is shown in Figure 10. It consists of the encoder, the bottleneck, and the decoder. The encoder compresses the spectrum’s information from the control center into the latent space. The bottleneck captures the essential feature map in the encoder’s output while the decoder is used to determine whether the PU band is occupied or not. It produces detection information concerning the status of the PU. Other applications of autoencoders include anomaly detection, image denoising, feature learning, and dimension reduction [93]. Variational, sparse, convolutional, and denoising autoencoders are the common types. The demonstration of the autoencoder for SS has been presented by the authors in [93,94]. In [93], a variational autoencoder is utilized for SS, where the test statistic is created from the signal sample.

4.4.5. Multi-Agent Deep Reinforcement Learning (MADRL)

Supervised, unsupervised, and deep learning models exhibit poor performance when deployed to handle tasks involving making complex decisions by employing few datasets. DRL is introduced to provide solutions to complex decision-making problems [95,96] with few training data. It combines DL and RL to perform functions such as game playing, robotics, finance, healthcare, and smart grid monitoring, among others [95,96,97]. Figure 11 illustrates the proposed multi-agent deep reinforcement learning (MADRL) architecture that fuses MLP (DL) and multiple-agent RL for SS. MLP is utilized to extract features in the input data, which will be transformed into more explicit representations using any of the following, Rectified Linear (ReLU), sigmoid, and tanh as the activation function. The hidden layer will transform the input data through weighted sums and non-linear mapping, where each neuron detects a specific pattern or feature of the training data. In this way, MLP extracts complex relationships and patterns in the training data. Multi- agent RL utilizes multiple agents (SUs) to learn, interact with the PU (environment), take action, and observe the resulting state and reward. The agents learn value functions or policies for maximizing the cumulative reward. Q-learning will be used for optimizing the policy or value function [98,99,100]. The importance of multi-agent RL in the architecture is to recognize the non-stationarity in the status of the PU (environment) and generate scalable and optimal output that converges faster than single-agent RL [101,102]. Thus, the proposed MADRL architecture will leverage on the excellent data processing ability of MLP to extract several features of the PUs in a dense IoT network, where many IoT devices transmit and receive, and also deploy multi-agent RL, for sensing, and to enable IoT devices to occupy the vacant spectrum of the PUs without delay and hassle.

Table 2. Comparison of the strengths and weaknesses of traditional and ML-based SS techniques.

SS Techniques	Strength	Weakness
ED	Implementation is easy and no prior information about the PU is required.	Exhibits poor performance at low SNR.
CFD	Robust to noise, fading, and shadowing.	Computationally intensive and takes longer time to sense vacant spectrum.
MF	Demonstrates robust performance at low SNR profile of wireless medium.	Requires information about the PU, which may not be available prior to the design of the IoT network
Pietra–Ricci Index detector	Prior information about the PU is not required.	Performance depends on the value of the threshold. Poor choice of the value of the threshold gives false pretense about the PU.
Supervised Learning and Unsupervised Learning	Learns features of the PU from a given dataset without requiring either partial or full knowledge of the PU.	Large quantity of datasets is required and the accuracy of spectrum detection depends on selected features.
Deep Learning (CNN, MLP, and LSTM)	Demonstrates good performance at low SNR.	High quantity of datasets is needed. High computational complexity is due to large hidden layers involved in training and the manner of processing the data.
Reinforcement Learning	Few datasets are required for implementation. Resilient performance at low SNR and it requires less time for sensing vacant spectrum.	High computing power is needed for implementation.

summarizes the strengths and weaknesses of various approaches to SS earlier discussed.

5. Review of the Application of ML for SS in Cognitive Radio Networks

The attention is shifted here to the review of various contributions on the application of ML for SS in CRIoT networks. These studies are reported by the authors in [66,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122] and are summarized in

Table 3. Summary of the contributions on the application of ML for SS in CRIoT networks.

Reference	ML Techniques	Class of the ML Techniques	Cognitive Radio Network	Main Findings	Limitations
[66]	RF, SVM, decision tree, Naïve Bayes, KNN, LR, and ANN	Supervised ML	Cooperative	The authors demonstrate that the performance of RF is superior to SVM, decision tree, Naïve Bayes, KNN, LR, and ANN.	All ML techniques considered as candidates for sensing exhibit high $P_{FA}$, which is undesirable in SS.
[103]	MLP, SVM, and NB	Supervised DL and supervised ML	Cooperative	MLP exhibits better performance in terms of training time and spectrum detection capability.	Attention is not paid to the assessment of the models in the non-stationary condition of the wireless channel, where SU and PU are mobile. The research is limited to the case of a single PU and three SUs.
[104]	Hybrid CNN-RNN and transfer learning	Supervised DL	Cooperative	Achieves high $P_D$ and low $P_{FA}$.	The study incurs high computational complexity that may limit its adoption in large-scale networks.
[105]	RNN and CNN	Supervised DL	Cooperative	CNN and RNN outperform traditional techniques in terms of $P_D$, $P_{FA}$ and bit error rate.	The models require large datasets for training and are susceptible to overfitting.
[106]	LSTM	Supervised DL	Cooperative	Higher spectrum detection accuracy and robust performance in low SNR regime of the wireless channel are achieved.	Large training datasets are needed, and relatively high detection time is required for implementing the ML technique.
[107]	ResNet50	Supervised DL	Non-cooperative	The rate of detection of PUs in the CRIoT network improves with the reduction in the level of noise in the network.	Optimization of the model to reduce computational time at high SNR values is not reported.
[108]	ANN	Supervised ML	Non-cooperative	High detection rate at high SNR and robust performance at low SNR are demonstrated. The model is superior to ED and improved ED.	Single PU and single SU are considered for investigation but a practical wireless communication system has many PUs and SUs.
[109]	K-means clustering	Unsupervised ML	Cooperative	Achieves resilient detection performance in the noisy condition of the channel.	Requires large training datasets.
[110]	Q-learning	RL	Cooperative	Improves spectrum detection in CR network.	The impact of changes in the network environment on the performance of RL algorithm is not addressed.
[111]	RF	Supervised ML	Cooperative	RF exhibits better accuracy than SVM, KNN, GMM, and NB.	The work is limited to the case of a single PU and three SUs.
[112]	Actor–critic	RL	Cooperative	Reduces the communication overhead required for SS.	The authors do not evaluate the performance of the learning model in terms of $P_D$ and $P_{FA}$. The study overlooks real-world challenges and imperfections like small- and large-scale fading as well as noise interference, which may impede practical deployment.
[113]	CNN (AlexNet, LeNet, and VGG-16)	Supervised DL	Cooperative	Improves the accuracy of spectrum detection in the CRIoT network. The CNN models exhibit better sensing performance than traditional AND, OR, and voting-based SS schemes.	Large quantity of datasets is needed for its implementation.
[114]	ANN	Supervised ML	Cooperative	Utilizes an ANN to improve the performance of ED and MF at low SNRs. Achieves better accuracy in spectrum detection. Furthermore, the proposed ANN+ED and ANN+MF reduce the false alarm rate and BER. It is shown that ANN+MF outperforms ANN+ED, ED, CFD, and SVM,	High computational complexity and large training datasets are needed. The proposed models are susceptible to overfitting and require a vast amount of fine tuning to suit a real network environment.
[115]	SVM, KNN, and RL	Supervised ML and RL	Cooperative	Demonstrates the effectiveness of RL for SS.	The study lacks detailed analysis of practical deployment and challenges in the real world.
[116]	SVM	Supervised ML	Cooperative	Reduces computational time and improves the accuracy of spectrum detection. Furthermore, the work theoretically demonstrates how to reduce potential harm of redundant and abnormal SUs in an IoT network.	The paper overlooks imperfections like fading and noise interference in the real world.
[117]	Q-learning	RL	Cooperative	Reduces computational resources and delay in accessing the vacant PU band.	Noise interference in real-time implementation is not accounted for in the study.
[118]	ANN, SVM, decision tree, and KNN	Supervised ML	Experimental	Detects spectral holes of the primary communication in real time application.	Huge number of real signals are needed for training the ML models. The study overlooks the noise effect, shadowing, and multipath fading, which degrade performance in real-time applications.
[119]	Hybrid CNN-LSTM	Supervised DL	Cooperative	Improves the accuracy of spectrum detection in low SNR regimes of the wireless channel.	High computational complexity and dependence on large training datasets for implementation.
[120]	Deep Autoencoder	Unsupervised DL	__	Utilizes a deep autoencoder to learn the features of the PUs and determine whether they are active or not.	The research overlooks imperfections associated with the wireless channel. This may derail the adoption of the model for practical applications.
[121]	Deep, LSTM, and variational Autoencoders	Unsupervised DL	__	Demonstrates the superiority of LSTM and deep autoencoders in sensing and distinguishing LTE and Wi-Fi signals.	The research overlooks the impact of wireless channels on signal transmission. The effect of fading, shadowing, and noise are not considered, limiting the implementation of the study in the real world.
[122]	Denoising Autoencoder	Unsupervised DL	Cooperative	Proposes a denoising autoencoder for detecting the vacant spectrum in integrated sensors and communication networks.	The study overlooks the mobility or non-stationarity of the wireless channel, critical in real-life applications.

. In [66], the authors consider RF, SVM, decision tree, NB, KNN, LR, and ANN for SS, where it is shown that RF outperforms SVM, decision tree, NB, KNN, LR, and ANN in terms of $P_D$, $P_{FA}$, $P_{MD}$, and accuracy. All ML techniques considered as candidates for SS, exhibit high $P_{FA}$, which is undesirable in radio networks. The authors in [103] investigate MLP, SVM, and NB for SS in a cooperative radio network. By using receiver operating characteristics and area under the curve as metrics, it is demonstrated that MLP exhibits better performance than SVM and NB. The authors do not assess the models in non-stationary condition of wireless channels, where SUs and PUs are mobile, and which typifies the practical real-world application. The research is also limited to the case of a single PU and three SUs. In [104], the authors present a hybrid CNN-RNN for SS and further incorporate transfer learning to improve the performance of the hybrid model at a low SNR. The proposed technique achieves high accuracy with large training overheads. The high computational complexity of the proposed model may limit its implementation in large-scale networks.

The authors in [105] demonstrate the effectiveness of RNNs and CNNs to expand wireless capacity by identifying unoccupied bands that may be used by secondary devices. Simulations illustrating $P_D$ and $P_{FA}$ suggest that the proposed DL models surpass MF and CFD. However, the learning models require large datasets for training, limiting deployment in computer systems with few resources. The contribution of [106] introduces LSTM for SS in non-cooperative radio networks by using modulated received signals as datasets. It is seen that LSTM outperforms CFD and ED. The limitations of the study are reliance on large training datasets and utilization of high spectrum detection time that may not be feasible in real-time applications. In [107], ResNet50 is proposed to show that the PUs are detected in CRIoT networks if the congestion rate is low. The authors do not optimize the model to reduce computational time at high SNR values. In [108], the authors propose an ANN for SS in cognitive radio networks. The ANN architecture consists of one hidden layer and output layers. The learning model is trained with ED and a likelihood ratio test scheme. The study is deployed to detect vacant spectral bands of the PU and it achieves 63% gain over ED and improved ED. An assumption of the communication system with a single PU and SU is made by the authors, which limits the adoption of the study in real-time applications, where many PUs and SUs interact and communicate. The contribution of the authors in [109] utilizes K-means clustering to sense the vacant spectrum in a cooperative network, characterized by Rayleigh, Rician, and Nakagami fading channels. It is shown that K-means clustering demonstrates better performance than ED and fusion-based schemes. The learning method requires large training datasets that are not readily available in real-time applications. The authors in [110] reduce the computational time for making decisions at the control center in a cooperative network. Q-learning is utilized to update the action–reward value of the SS algorithm. The authors do not take into account unpredictable and dynamic channel states in wireless channels, which may affect the reliability of the study for practical applications.

In [111], the authors optimize the performance of RF using Bayesian learning, which is shown to be more accurate than SVM, KNN, and GMM. The learning model achieves $P_D$ and $P_{FA}$ of 0.94 and 0.1, respectively. However, the work is limited to the case of a single PU and three SUs. This does not accurately represent practical wireless communication environments with many potential users. RL is utilized by the authors in [112] to improve SS accuracy in a cooperative IoT network, where an actor–critic learning process is adopted. Evaluation of the performance of the learning model in terms of $P_D$ and $P_{FA}$, which are critical in SS is not reported. The study overlooks real-world challenges and imperfections like small and large-scale fading as well as noise interference, which may impede practical deployment. The authors in [113] train three CNN models including AlexNet, LeNet, and VGG-16 for SS, where the covariance matrix of the sensing information at the control center is removed and updated information is used as the input into the CNN models for the final decision. The decisions obtained from the CNNs are communicated to the SUs. All the learning methods demonstrate better accuracy than AND, OR, and voting-based schemes. In addition, VGG-16 exhibits higher $P_D$ and lower $P_{FA}$ compared to other CNN models. The improved performance of VGG-16 necessitates more computer resources, which may incur a higher computational cost. The study also relies on large datasets for training, which may not be available in real-time applications.

The authors in [114] utilize an ANN to improve the performance of ED and MF at low SNRs, resulting in ANN-based ED and MF techniques. The proposed models improve the detection accuracy, reduce the false alarm, and bit error rate (BER). The study demonstrates the superiority of ANN-based MF to ANN-based ED, conventional ED, conventional CFD, and SVM. The drawbacks of the proposed techniques are high computational complexity and reliance on large labeled datasets as well as inability to implement in devices with limited computer resources due to vast memory and high computational power requirements. The proposed models need to be practically validated for real-world applications. The authors in [115] present RL for SS in a cooperative network. The study effectively shows that RL performs better than SVM, KNN, and the wavelet transform. The paper lacks detailed analysis concerning practical deployment and challenges in real-time applications. In [116], the authors introduce an SVM for reducing the computational overhead required for detecting vacant spectrums and improving sensing performance in a cooperative IoT network. The proposed ML technique groups SUs into various classes for easy identification of unoccupied PU bands. The first-class groups SUs into abnormal and normal users, the second classifies SUs into redundant and non-redundant users, the third-class groups SUs into optimized cooperation class, and lastly, the fourth classifies the SUs into superior and inferior users. Simulation results portray the superiority of the proposed technique to conventional techniques. The study overlooks imperfections such as fading, shadowing, terrestrial effects, and noise interference in real-world applications. The implementation of the proposed technique in a multi-band cooperative scenario is also not considered in the study.

The authors in [117] propose RL for reducing the amount of computational resources required for scanning the PU and reducing the delay in sensing unused spectrums in a cooperative users’ network. The study considers the mobility of the PUs and the SUs as well as the dynamic nature of the wireless channel in its analysis, which is critical in real-world applications. The performance of the proposed method is evaluated over average detection probability, average number of times that SU successfully accesses the PU, and call block rate. Analytical results demonstrate the effectiveness of the proposed method to achieve a high probability of detection and low call block rate. The study fails to account for noise interference in real-time applications. In the experimental study of [118], the authors consider ANN, SVM, decision tree, and KNN to sense real signals generated by a low-cost smart embedded device that operates at 433 MHz and a wireless transmitter with amplitude shift keying (ASK) and frequency shift keying (FSK) modulation schemes. Simulation results reveal that the ANN and SVM are more accurate for the sensing task than decision tree and KNN. The drawbacks of the study include dependence on a huge number of real signals for training the algorithms and failure to take into the account in the analysis, shadowing and multipath fading, which degrade performance in real-time implementation. The authors in [119] present an innovative solution to SS problems within the noisy regime of the wireless channel by proposing a hybrid ML technique that combines a CNN and LSTM. The authors refer to the proposed technique as SenseNet. The study shows that the proposed solution achieves a 3.3% gain in sensing accuracy at an SNR of −20 dB, when compared to conventional CNNs. The limitations of the study include high computational complexity and reliance on a high quantity of datasets for training.

The study in [120] proposes a deep autoencoder to learn the features of the primary communication node, which is further classified by SVM into active and passive nodes. It is shown that a deep autoencoder with a radial basis kernel exhibits the highest accuracy in spectrogram signal while a deep autoencoder with a linear kernel demonstrates the best accuracy in amplitude phase signal. The research overlooks the imperfections associated with the wireless channel, limiting the adoption of the model for practical implementation. The authors in [121] consider deep, variational, and LSTM autoencoders for sensing and for distinguishing between LTE and Wi-Fi signals. The research effectively shows that deep and LSTM autoencoders have better values of recall and F1-score. The variational autoencoder exhibits difficulty in recognizing LTE and Wi-Fi signals. It is also demonstrated in the study that the deep autoencoder converges faster than the LSTM autoencoder, but produces lesser values of recall, compared to LSTM, while the values of precision for LSTM and deep autoencoders are the same. The research overlooks the impact of wireless channels on signal transmission. The effect of fading, shadowing, and noise are not considered, which may derail the implementation of the study.

Similarly, the research in [122] proposes a denoising autoencoder for detecting the vacant spectrum in integrated sensor and communication networks. The authors demonstrate the effectiveness of the proposed approach in detecting the vacant spectrum and its robustness to shadowing. Mobility or non-stationarity of the wireless channel, critical in real-life demonstration, is overlooked in the study. To the best of authors’ knowledge, no work has been reported in the literature on MADRL for SS in CRIoT networks and it is proposed in this work for enhancing the accuracy of spectrum detection in dynamic wireless channels, and for addressing research concerns, as observed in the foregoing.

The illustration of the trend in the adoption of ML for SS by the researchers, in the years covering 2018 to 2025, as captured in Table 2, is presented in Figure 12. It is observed in the illustration that supervised ML receives the most attention, closely followed by supervised DL, while unsupervised ML techniques receive the least attention. The preference of supervised ML by most researchers may be connected to the use of few computer resources and less rigor in its implementation as well as scalability across diverse datasets.

6. Importance of SS in 5G and Beyond IoT Networks

The rising data volume and the introduction of data-intensive applications necessitate more than ever the need to manage spectrum resources in the present cellular network. Hence, the importance of SS in the current network infrastructure cannot be overemphasized. The advantages of SS in 5G and beyond IoT networks include the following:

(i)

Management of existing data spectrum: SS encourages efficient usage of spectrum resources by identifying vacant portions of primary channels that may be deployed for communication [123,124]. This preserves the cost of developing a new bandwidth for accommodating an upsurge in mobile connectivity and upcoming data-intensive applications.

(ii)

Enhancement of capacity and spectral efficiency of the wireless network: SS in 5G and beyond IoT networks has the potential to free up space for communication, increasing throughput and spectral efficiency [15]. With SS, new IoT devices will not be short of spectral bands and the development of IoT will not be endangered in any way.

(iii)

Avoidance of interference with primary communication: SUs are opportunistic users and they become active when PU bands are free or inactive. Thus, IoT devices will only occupy unused spectrums of inactive primary channels. This process reduces interference [125], prevents network delays, and reduces potential attacks on the operation of the PUs.

(iv)

Optimization of spectrum usage: SS supports efficient optimization of usage of the spectrum in CRIoT networks [126]. This importance prevents spectrum wastage and guarantees availability of spectral bands for communication in 5G and beyond IoT networks.

7. Lessons Learnt

The takeaways from comprehensive survey carried out in this work are listed as follows:

(i)

Robustness of SS models: The wireless channel is characterized by fading, shadowing, and terrestrial disturbances. As a result, SS models must be able to withstand variation or changes in communication channels. The application of RL in such an environment suffices. The agent is trained to understand and interact perfectly with the environment. The decision is based on the maximization of the reward, which guides the training process.

(ii)

Training data: The performance of ML algorithms for SS depends on the number of datasets that is used to train the models. Supervised ML exhibits high accuracy and converges faster if the training data are high.

(iii)

Difficulty in the optimization of the models: Because of large computational resources required for training RL and DRL models, the training process can be slow, boring, unstable, and may be difficult to optimize, leading to inaccurate and inconsistent spectrum detection results.

(iv)

Generalization: Model-free RL algorithms may struggle to generalize beyond their training environment and may be unable to adapt to new conditions.

(v)

Real world applications: The trial-and-error approach of RL and DRL may create problems in real-world implementation.

8. Challenges of SS in CRIoT Network

An appraisal of the previous studies reveals quite a number of challenges that may potentially derail existing SS techniques from achieving accurate results. These challenges include the following:

(i)

Requirement for large training data: Supervised ML algorithms require a huge amount of training data for accurate SS results. It may be impossible to obtain huge training data that will capture the true characteristics of the ever-dynamic network environment in real-life scenarios. Current efforts in this regard utilize computer-generated data, which are fashioned based on simplified assumptions of the network environment and may not model the nuanced and complicated features of wireless channels in the real world [127].

(ii)

Huge computational resources: Supervised DL techniques like CNN and RNN require significant amounts of computational resources. This requirement can be a challenge in real time processing, especially in computer systems with limited computational facilities.

(iii)

Overfitting: DL techniques are susceptible to overfitting due to high levels of modeling. This may result in poor performance.

(iv)

Poor understanding of ML models: ML techniques are black boxes that may be hard to understand. In some cases, there may be the need to know why certain SS decisions are taken.

(v)

Hardware consideration: RL and DRL require fast computing power and specialized hardware like graphics processing units (GPUs) for smooth operation and implementation. Their computational complexity in real-time applications may cause difficulty when deployed in large-scale networks. This factor also limits their deployment in computer devices with constrained hardware resources.

(vi)

Dynamic propagation channel: The network environment changes from time to time, ensuring that SS techniques adapt to the nature of the network environment is still a concern in real-world applications. This may necessitate training, retraining, and fine tuning of the existing models.

(vii)

ML architecture: The nature of the architecture of the ML algorithm plays a great role in its performance. A poor choice of architecture may lead to a bad result.

(viii)

Powering IoT devices: IoT devices are run on batteries with a limited lifecycle, amid these, are the challenges of greenhouse effects due to improper disposal of the batteries, and large operational cost, owing to constant replacement of the batteries. Alas, this is not sustainable. Other ways of energizing IoT devices as proposed in the literature are to equip them with energy harvesters. The authors in [128] have theoretically described various energy sources that can be used, which include ambient environments, mechanical sources, thermoelectric generators, and tribo-electric nanogenerators. The authors in [129,130] have shown that small amount of D.C voltage (order of milli-volt) is obtained from an ambient environment by deploying an energy harvester, consisting of a rectifier circuit and antenna. This voltage is not enough to power a large-scale IoT network. Other sources theoretically proposed apart from ambient sources are in the budding stage of practical deployment.

9. Future Prospects

The following future prospects and open issues are observed and are highlighted for further exploration by the research community in mitigating the challenges in SS and improving the accuracy of spectrum detection in 5G and beyond IoT networks.

(i)

Interference measurement: Because the PUs are passive devices, SUs may not be aware of their precise location in the communication network. This phenomenon may cause SUs to interfere with the operation of the PUs. To the best of our knowledge, a device that allows SUs to estimate the interference at nearby PUs is currently non-existent. This device is essential in real-time implementation. Purposeful research efforts should be devoted to address this concern.

(ii)

Huge data transfer: The transfer of overall sensing data to the control center in CRIoT networks with many IoT devices may necessitate the need for huge amount of data in real-time applications. Wireless transfer of information without interfering with the operation of the PU may be impossible to achieve. Efforts should be geared towards addressing this concern without affecting primary communication.

(iii)

SS in multi-user environments: Most works in the literature have considered a single PU and multiple SUs as design model for investigation. IoT devices operate in multi-user network environments, where multiple PUs and SUs are present. There is a need to explore the spatial diversity inherent in multi-dimensional network environments for SS, where multiple PUs and SUs are seen to be present.

(iv)

Reduced time for SS: Critical attention should be paid to the reduction in time taken to sense an unused spectrum of the PUs. Future work should focus on further reducing sensing time, where less samples are utilized to detect the PU without compromising accuracy and without increasing $P_{MD}$ and $P_{FA}$.

(v)

Fallacious result: Malicious attacks from the SUs reduce the accuracy of SS results. The attack may be in the form of false sensing results that are sent by the SUs to mislead the control center. Though the authors in [114,131] consider this scenario in their investigations, there is a need for more robust SS techniques that will notice and demystify this conundrum.

(vi)

Emulation of the PU: A mischievous SU may emulate and exhibit the characteristics of the PU in order to prevent other SUs from accessing the primary channel [132]. This challenge necessitates the need for novel SS techniques that will give all SUs the same leverage and ensure fairness among the SUs [133].

(vii)

Cost of implementation: The requirement for specialized hardware for implementing DRL and RL models may impact on the cost of the hardware resources for SS in 5G and beyond networks. Energy-efficient hardware design, consisting of miniaturized elements and less power consumption will be vital for detecting vacant spectrums in multi-dimensional network environments. This is necessary for reducing the overall cost of implementation and execution.

(viii)

Role of transfer learning: Transfer learning can be used to accelerate the process of detecting unoccupied spectrums in CRIoT networks, where pre-trained CNNs (or other deep learning models) are utilized for extracting the features of the input data. The learned characteristics in the hidden layers of pre-trained CNNs or MLPs are further used as features for fresh and related tasks, leading to improved performance [134]. Though the authors in [104] have cascaded hybrid CNNs and RNNs with transfer learning to improve spectrum detection at low SNRs, there is a need for more research that will leverage on transfer learning to reduce spectrum detection time.

10. Conclusions

CR is essential for enhancing the growth of IoT communication in stochastic network environments, characterized by shadowing, slow and fast fading, as well as terrestrial disturbances. One of the core functions of CR is SS, where the IoT devices sense and transmit in the unused portion of the licensed spectrum assigned to the PUs. This process guarantees maximum spectrum utilization. Numerous methods are available in the literature for SS, each with its pros and cons. This paper presents a survey of works on SS to bring them up to date with the latest efforts in this domain. We focus specifically on the description of the techniques widely adopted for SS, including ED, CFD, MF, Pietra–Ricci detection, and ML. We highlight various articles where the application of the aforementioned methods is demonstrated. In addition, we identify the drawbacks and highlight challenges that affect the implementation of various techniques. Finally, prospects or open areas that may be explored as future research are observed and highlighted. It is our belief that the article will provide guidance to communication system designers and researchers willing to embark on finding missions in this field.

As a perception, the importance of model-free RL algorithms for SS in 5G and beyond networks cannot be ignored. RL is robust to extraneous network challenges, which is essential in obtaining accurate spectrum detection results. This gain may be further consolidated by multi-agent RL, which utilizes two or more agents to interact simultaneously with primary sources. Multi-agent RL can recognize non-stationarity in the network environment and yield scalable and optimal solutions that converge faster than single-agent RL. Moreover, MLP is capable of handling a high amount of sensing information generated by many IoT devices in dense networks. Thus, MADRL is proposed for addressing SS problems in 5G and beyond networks, with many IoT devices and multiple primary channels. The authors are currently investigating the application of MADRL for SS in CRIoT network and the outcome of the investigation shall be reported in our forthcoming paper.