Big Data Analysis and Prediction of Electromagnetic Spectrum Resources: A Graph Approach

Abstract

In the field of wireless communication, the increasing number of devices makes limited spectrum resources more scarce and accelerates the complexity of the electromagnetic environment, posing a serious threat to the sustainability of the industry’s development. Therefore, new effective technical methods are needed to mine and analyze the activity rules of spectrum resources to reduce the risk of frequency conflict. This paper introduces the idea of graphs and proposes a spectrum resource analysis and prediction architecture based on big data. In this architecture, a spatial correlation model of spectrum activities is constructed through feature extraction. In addition, based on this correlation model, a depth learning network based on graph convolution is designed, which uses the prior information of spatial activity to achieve the efficient prediction of spectrum resources. Numerical experiments were carried out on two datasets with different spatial scales. Compared with the best baseline model, the prediction error is reduced by 8.3% on the small-scale dataset and 11.7% on the large-scale dataset. This shows that the proposed method is applicable to different spatial scales and has more obvious advantages in complex scenes with large spatial scales. It can effectively use the results of spatial domain analysis to improve the prediction accuracy of spectrum resources.

1. Introduction

In the past 10 years, the communication industry has developed rapidly [1,2,3]. Limited spectrum resources are unable to meet the rapidly growing application needs [4], which seriously hinders the long-term development of many industries based on communication technologies [5]. On the other hand, the use of a large amount of equipment has raised the noise level of the signal and also caused the deterioration of the electromagnetic environment [6]. The complexity of the electromagnetic environment, in turn, causes interference with various communication equipment, forming a vicious circle [7]. Therefore, it is urgent to grasp the activity rules of the electromagnetic spectrum and ensure the healthy and sustainable development of the industry through the analysis and prediction of the electromagnetic spectrum situation [8]. With the maturity of big data and machine learning technology, many research results of a spectrum situation analysis and prediction technology based on big data have emerged [9]. By fitting historical big data, this technology effectively overcomes the problem of complex and changeable spectrum activities, characterizes the spectrum law, and predicts the use of spectrum resources in the future [10].

Spectrum activity analysis is a method of describing the characteristics of the electromagnetic spectrum in the time, frequency, and space domains through data mining [11]. In the existing results, researchers usually use correlations to characterize the characteristics of spectra or channels. Spectrum prediction, also known as spectrum inference [10], has seen a certain number of research results in the past few years. In earlier studies, the spectrum forecasting problem was generally defined as a time-series forecasting problem. Statistical methods or traditional machine learning methods are applied to construct activity models to predict spectrum holes. Among them, the autoregressive model (ARM) [12] and hidden Markov model (HMM) [13] are the classic and most recognized models. Later, with the rapid development of deep learning, various neural network models became the preferred solution to the problem. At the same time, researchers began to pay more attention to multi-domain joint prediction methods. The most common ones are joint predictions in the time-frequency domain. In Section 2, we briefly review the important literature on spectrum activity analysis and spectrum prediction in recent years and summarize the problems of the insufficient utilization of information on spectrum analysis results and the existing methods performing poorly on nonlinear spatial data.

Graphs and graph convolution networks (GCN) have become a new research hotspot in recent years. Because of their good nonlinear data analysis ability, they have been widely used in many fields. Based on the above problems, this paper introduces the idea of graphs and proposes a graph-based big data joint analysis and prediction method. Specifically, the contributions of this paper are as follows:

1.

To solve the problem of the application of insufficient prior information in spectrum resource prediction, a graph-based multi-domain joint analysis and prediction architecture is designed. The prior information obtained from big data analysis is used to guide the network model to improve the accuracy of analysis and prediction;

2.

A method for analyzing large data spectrum activity based on feature extraction is proposed. Through the correlation analysis of spatial characteristics, the spatial correlation model of spectrum activities is constructed;

3.

A time–frequency–space joint prediction network combining graph convolution and dilation convolution is designed, which learns the spatial and spectral characteristics of the irregular distribution of space through graph convolution and uses dilation convolution to learn the time-frequency laws. The model can use the information from the analysis results. The experimental results on two real datasets show that our model is superior to the baseline model.

The rest of this paper is organized as follows. Section 2 presents the literature review of spectrum analysis and prediction. In Section 3, we introduce the methods proposed in this paper. Specifically, our architecture is introduced in Section 3.1. In Section 3.2, we introduce the big data analysis method of spatial correlation. In Section 3.3, we introduce the prediction network model based on graph convolution. In Section 4, the numerical experiments and results are presented. The significance and conclusions of our work are discussed in Section 5.

2. Literature Review

In terms of spectrum activity analysis, researchers use parameters such as correlation to describe the characteristics of spectrum activities, including the time, frequency and space domains. In [14,15], the authors monitored the spectrum situations of London and Paris through mobile devices. A spatial semivariogram was used to describe the spatial correlation of spectrum usage. It was found that there was a strong spatial correlation between different operators, communication systems, and links. In [16], a visual abstract method was proposed to encode and represent the signal state. In [17], two clustering algorithms were proposed, which comprehensively characterized the evolution characteristics of frequency points through clustering analysis of the spectrum state evolution. The correlation between the spectrum situation evolution of different frequency points was mined by clustering. In [18], the concept of an electromagnetic environment portrait was proposed, and the multi-dimensional characteristics and correlation of a spectrum were expressed through the mining of big data.

In terms of spectrum prediction, a large number of results based on machine learning have emerged in recent years. Most of them belong to the category of supervised learning and a small number of results are based on reinforcement learning. In [19], based on reinforcement learning and Bayesian fusion, the authors proposed an idle channel prediction algorithm to realize the efficient utilization of a spectrum. Double thresholds were used to improve the perceived probability of the free frequency band and Bayesian fusion was set to obtain the final selection strategy of the free frequency band.

Among the results based on supervised learning, this paper divides them into time-series predictions and multi-domain joint predictions. The former only relies on time-domain observation data, whereas the latter uses time-domain data and other observations such as frequency-domain and air-domain data. In [20], a functional link artificial neural network was proposed. The algorithm used historical spectrum occupancy statistics to predict future spectrum usage. The authors in [21] optimized the neural network through the genetic algorithm to improve prediction accuracy. With the development and maturity of recurrent neural network (RNN) technology, models based on RNNs and their variants (such as long short-term memory(LSTM) and gate recurrent unit (GRU)) are widely used in time-domain predictions of spectra. In [22], the authors used the LSTM and GRU models to predict based on duty-cycle data and their own characteristics. In [23], a method to initialize LSTM based on prior knowledge was proposed.

Later, scholars realized that accuracy could be improved and the prediction target expanded by combining multiple domains. In [24], a collaborative spectrum prediction scheme was proposed to reduce the sensor energy consumption of CNNs. A long short-term memory network was used to predict a local spectrum and then parallel fusion technology was used to optimize the error of the local prediction model. In [25], the authors proposed a long-term prediction scheme based on tensor completion. In [26], transfer learning was applied to cross-band prediction tasks. As the superior performance of convolution neural networks (CNNs) was gradually demonstrated, the model based on a combination of CNNs and RNNs has become the first choice of researchers. In [27], a network connected in series by CNNs and bi-directional LSTM was used for satellite spectrum prediction. In [28], a new network combining CNNs and GRUs was proposed to achieve accurate prediction. In terms of joint spatial prediction, the position and spectrum state of unmanned aerial vehicles were predicted simultaneously based on homotopy theory and the HMM model [29]. In [30], a new network, ConvLSTM, which combines CNNs and LSTM, was used to learn time–frequency–space multi-dimensional correlations and long-term predictions. In addition, a network combining a CNN and ResNet was used to jointly predict in the time and spatial domains [31].

By summarizing the existing research results, we find that in the multi-domain joint prediction, the results combined with the space domain are mainly still using the CNN-based model. However, spatial data are usually distributed nonlinearly so convolution-based methods cannot guarantee stable and good results. In addition, the correlation results of spectral analysis can be used as auxiliary information for prediction. However, due to the black-box characteristics of neural networks, there is currently a lack of results combining spectrum analysis and prediction.

3. Methods

Faced with the sustainable development of electromagnetic fields, we focus on the relationships between spectrum activities in different spatial locations. According to the research results in [14], the use of a spectrum at different locations has complex correlations. These kinds of correlations can be regarded as abstract descriptions of spectrum spatial law, which has important guiding significance for spectrum resource prediction and effective management. Therefore, based on graph theory, this paper proposes a prediction method combined with correlation analysis to discover the potential spectrum use risks of using big data technology.

3.1. Graph-Based Joint Analysis and Prediction Architecture

In the joint analysis and prediction architecture proposed in this paper, we consider the problem of the synchronous prediction of spectrum states at different spatial locations. A key problem to pay attention to is that the spatial distribution of data is different from that in the frequency domain and is not linearly distributed. It is usually difficult to achieve good results using the convolution-based prediction models commonly used in the frequency domain. Therefore, the advanced graph-based method in the field of spatiotemporal prediction is introduced.

A graph is formulated as $G=(V,E)$, where V is the set of nodes and E is the set of edges. In this paper, we define the monitoring results of secondary users at different locations as the nodes of a graph, and the associations between different nodes (considered a potential spatial association) are defined as the edges. Let $u,v\in V$ denote a node and $e=(u,v)\in E$ denote an edge from u to v. In addition, we define the neighborhood of node v as $N(v)=\{u\in V\mid (u,v)\in E\}$. The neighborhood can be understood as a set of nodes associated with a node. We build the historical monitoring data into graph-structured data based on the above definition and characterize the relationships between graph nodes based on an adjacency matrix. The adjacency matrix is the mathematical representation of a graph, which is generally expressed as $A\in R^{N\times N}$.

Our graph-based prediction architecture is shown in Figure 1. Firstly, the spatial correlation of historical data is mined based on the feature extraction method to obtain the correlations between nodes. Then, the graph adjacency matrix is constructed based on the correlation conclusion, and the observed data are constructed as graph-structured data. Finally, the time–frequency–space results are predicted by the model based on graph convolutions.

3.2. Spectrum Activity Association Analysis Based on Big Data

Thanks to the development of the distributed spectrum monitoring system, the analysis method of big data can effectively mine the activity rule information hidden in the spectrum data, providing support for the prediction and management of spectrum resources.

3.2.1. Spatial Feature Extraction

In our previous work, we calculated the frequency domain correlation based on the correlation coefficient of time series between frequency points. However, in the spatial correlation calculation, the monitoring results package of different spatial locations is a matrix with two dimensions of time and frequency. The direct use of two-dimensional correlation coefficient calculations is not only highly complex but also prone to producing large errors [32]. Constructing a feature matrix is an effective method [33]. We extract features from the time series data of each frequency point and then calculate the correlation coefficient of the feature matrix to define the spatial-domain correlation. This method avoids the problems mentioned above. In this paper, we use the duty cycle and multiple temporal and statistical features to build the feature matrix. Specifically, this paper uses the following features:

(1) Duty Cycle

In general, the channel occupancy status can be characterized by the following formula:

$$\left\{\begin{array}{cc}\mathsf{\Omega }=0& y(t)=n(t)\\ \mathsf{\Omega }=1& y(t)=n(t)+s(t)\end{array}\right.,$$

(1)

where $\mathsf{\Omega }$ represents the channel status. When $\mathsf{\Omega }=1$, the channel is occupied; when $\mathsf{\Omega }=0$, it is not occupied. t is the time, $y(t)$ is the reception result in the spectrum measurement, $n(t)$ is the background noise, and $s(t)$ is the signal. The duty cycle is a statistical result of the channel occupancy status and an important characteristic index reflecting the spectrum utilization efficiency. Generally, there are two definitions: the duty cycle of a channel is defined as the ratio of the timeslot occupied to the total timeslot, and the duty cycle of a timeslot is defined as the ratio of the occupied channels to the total channels. In this paper, we use the first definition, which can be expressed as:

$$DC=\frac{1}{T}\sum _{t=1}^T\mathsf{\Omega }(t),$$

(2)

where T is the number of timeslots and $\mathsf{\Omega }(t)$ is the occupation status of the channel in the timeslot t.

(2) Mean Value

The mean value of a series is one of the most commonly used and important measures in data statistics, which reflects the average level of the series. The mean value of a frequency point over a long period of time can be defined as

$$\overline{y}=\frac{1}{T}\sum _{t=1}^{T}y(t).$$

(3)

(3) Standard Deviation

The standard deviation is a feature describing the volatility of a series, reflecting the deviation degree between the series and the mean. The larger the value, the stronger the series fluctuation. The standard deviation is defined as follows:

$$\sigma =\sqrt{\frac{1}{T}\sum _{i=1}^T{(y(t)-\overline{y})}^2}.$$

(4)

(4) Quartile Distance

A quartile is a kind of quantile in statistics, that is, all values are arranged from small to large and divided into four equal parts. The values at the three dividing points are called quartiles. The quartile distance is the difference between the third quartile and the first quartile, which is a characteristic of sequence dispersion.

$$IQR=Q3-Q1,$$

(5)

where $IQR$ is the quartile distance and $Q3$ and $Q1$ are the third and the first quartiles.

(5) Autocorrelation Coefficient

The autocorrelation coefficient measures the correlation of the same sequence between two different times, reflecting the impact of the historical results on the future. The l-order autocorrelation coefficient is defined as follows:

$$AC=\frac{{\sum }_{t=1}^{T-l}(y(t)-\overline{y})(y(t+l)-\overline{y})}{{\sum }_{t=1}^T{(y(t)-\overline{y})}^2}.$$

(6)

(6) Entropy

The entropy used in this paper is the permutation entropy, which is a parameter to describe the randomness of time series and can represent the random noise level of the series. The definition formula is as follows:

$$ENT=-{\sum }_{x\in S}P(x){log}_{2}P(x),$$

(7)

where S is all possible permutations of the subsequence. From the perspective of informatics, the above formula calculates the information entropy of the permutation probability of the subsequence, which is an expression of the degree of sequence confusion.

(7) Skewness

Skewness is a measure of the asymmetry of a distribution. A distribution is asymmetrical when its left and right sides are not mirror images. From a mathematical point of view, skewness is the third-order normalized moment of a sequence. Its definition and expression are as follows:

$$SKE=E[{(\frac{y_t-\overline{y}}{\sigma })}^3],$$

(8)

where $y_t$ is the temporal sequence.

(8) Kurtosis

Kurtosis and skewness are similar statistics used to describe the skewness of a sequence. Specifically, kurtosis is a feature representing the steep degree of the probability distribution of a time series. From the mathematical definition, kurtosis is the fourth-order normalized moment of the sequence, which is defined as follows:

$$KUR=E[{(\frac{y_t-\overline{y}}{\sigma })}^4].$$

(9)

3.2.2. Spatial Correlation Model of a Spectrum Based on Graphs

Based on the 8 features described above, we construct feature vectors for each frequency point’s sequence data in different spatial locations:

$$V_f=\{D{C}_f,{\overline{y}}_f,{\sigma }_f,IQ{R}_f,A{C}_f,EN{T}_f,SK{E}_f,KU{R}_f\},$$

(10)

where f presents a frequency point. Based on the above feature vector, we build all frequency point data of a spatial location into a feature matrix M.

$$M_i=\left[V_{f_1},V_{f_2},\cdots ,V_{f_n}\right],$$

(11)

where $M_i$ represents the feature matrix of spatial location i and n represents the number of frequency points studied. Since the above features have different value ranges, all data need to be normalized before calculating the spatial correlation. In this paper, $M^{{}^{\prime }}$ is used to represent the normalized spatial feature matrix. Then, we use the correlation coefficient of the matrix to measure the correlation between different locations. The correlation coefficient is calculated as follows:

$${\gamma }_{ij}=\frac{\sum ({M^{\prime }}_i-{{\overline{M}}^{\prime }}_i)({M^{\prime }}_j-{{\overline{M}}^{\prime }}_j)}{\sqrt{\sum {({M^{\prime }}_i-{{\overline{M}}^{\prime }}_i)}^2\sum {({M^{\prime }}_j-{{\overline{M}}^{\prime }}_j)}^2}},$$

(12)

where $\overline{M}$ is the mean value of the normalized feature matrix. Based on the correlation results, we construct the adjacency matrix A to represent the graph structural relationship using the following formula:

$$\left\{\begin{array}{cc}{A}_{ij}={\gamma }_{ij}& {\gamma }_{ij}\ge Thr\\ A_{ij}=0& {\gamma }_{ij}<Thr\end{array}\right.,$$

(13)

where $A_{i}j$ is the value of the i-th row and the j-th column of the adjacency matrix, representing the association between spatial locations i and j. $Thr$ is the threshold set by experience. For the part below the threshold value, we believe that there is no similarity. For the part above the threshold value, we think that there is a certain degree of similarity in the spectral evolution between two spatial locations, and the value of the correlation coefficient is used to characterize the level of this similarity.

By constructing the adjacency matrix, we obtain the graph model of the spatial correlation of a spectrum, which is abstract knowledge. In the following, we input these results into the network to improve the spectrum resource prediction.

3.3. Time–Frequency–Space Prediction Network Based on a GCN

The biggest difference between GCNs and CNNs is the use of an adjacency matrix. Based on the prior information hidden in the adjacency matrix, a GCN has a good effect on complex spatial prediction problems, which helps us to predict spatial spectrum resources and improve the resistance to spectrum use risks.

3.3.1. Model Architecture

The network model architecture proposed in this paper is shown in Figure 2. It consists of stacked time–frequency–space blocks (TFS blocks) and an output layer. Each TFS block contains a dilated convolution unit for learning time-frequency features and a graph convolution layer for learning spatial features. The output layer consists of an active layer and a 2D convolution layer, which is used to set the output step and obtain the expected prediction size. Each TFS block is connected to the output layer through a skip connection layer. The skip connection layer is composed of a 2D convolution with a convolution kernel size of $1\times 1$. By stacking TFS blocks, the network can learn the spatial dependence of different time scales. Specifically, at the bottom, the GCN learns short-term information, whereas at the top, the GCN learns long-term information. The input of the network is graph-structured data, which contain time–frequency–space spectrum observation data and the adjacency matrix of the spatial relationship obtained from the previous mining. The network finally outputs the joint prediction results of multiple time steps at the same time rather than generating them recursively.

3.3.2. Graph Convolution Layer

In the previous section, we built graph data based on spatial correlations. Obviously, the traditional convolution is no longer applicable so a graph convolution is introduced to learn the spatial dependence. The traditional convolution network is essentially a process of the weighted summation of pixels in a certain spatial area through a filter to obtain a new feature representation. The weighting coefficient is a parameter of the convolution kernel in a CNN that is applicable to regular multidimensional matrix data. However, it cannot be applied to non-Euclidean-structured data, that is, graph-structured data. The purpose of graph convolution is to extract the spatial features of topological graphs instead of a CNN. The convolution is generally calculated in the Fourier domain (time-domain convolution is equal to frequency-domain multiplication). In order to meet the requirements of a Fourier transform, it is necessary to find a continuous orthogonal basis corresponding to the basis of a Fourier transform. In the existing study of graph convolutions, the eigenvector of the Laplace matrix is generally used as the basis of the Fourier transform. The most commonly used symmetric normalized Laplacian formula is as follows:

$$\tilde{A}=I+D^{-1/2}A{D}^{-1/2},$$

(14)

where A is the adjacency matrix.

Let f represent the N-dimensional vector on the graph, where $f(i)$ represents the feature vector of the nodes on the topology. Let the eigenvector matrix of the Laplacian matrix L of the topological graph be U, then, the eigenvector $u_l$ corresponding to the eigenvalue ${\lambda }_l$ can be used as an orthogonal basis in the Fourier transform. The Fourier transform of the N-dimensional vector f is calculated as follows:

$$F({\lambda }_l)=\widehat{f}({\lambda }_l)=\sum _{i=1}^{N}f(i)u_l(i),$$

(15)

where $\widehat{f}=U^{T}f$. The convolution of f and convolution kernel h is the inverse transformation of the product of their Fourier transforms so the graph convolution formula is as follows:

$${(f\ast h)}_G=U((U^{T}h)\odot (U^{T}f)),$$

(16)

where ⊙ is the Hadamard product, that is, the inner product of these two vectors. Because in a graph convolution the convolution kernel is designed, the Fourier transform of h can be written in a diagonal form:

$${(f\ast h)}_G=U{U}^{T}h{U}^{T}f=U\left(\begin{array}{ccc}\widehat{h}({\lambda }_1)& & \\ & \ddots & \\ & & \widehat{h}({\lambda }_n)\end{array}\right)U^{T}f,$$

(17)

where $\widehat{h}({\lambda }_l)={\sum }_{i=1}^{N}h(i)u_l^{*}(i)$ represents the learnable parameters. In the most popular method, h is fitted by Chebyshev polynomials and the convolution kernel is fitted by Chebyshev polynomials to reduce the complexity. The final graph convolution output of graph data X is as follows:

$$Z=\sigma ((I+D^{-1/2}A{D}^{-1/2})XW)=\sigma (\tilde{A}XW),$$

(18)

where $\tilde{A}$ is an adjacency symmetric normalized Laplacian and W is a learnable parameter matrix. Using the input adjacency matrix as the prior information, we can more effectively learn the spatial dependence of the data and improve prediction accuracy by comparing convolutions.

3.3.3. Dilated Convolution Layer

For time-frequency feature learning, we refer to the existing research results and use convolutional networks. In order to improve the receptive field in long-term historical monitoring data, a dilated convolution is used here.

In the image field, a traditional convolution network usually uses a pooling layer and a convolution layer to increase the receptive field and reduce the resolution of the feature map, and then upsampling to restore the image size. The process of reducing and enlarging the feature map causes a loss of accuracy.

Unlike normal convolutions, dilated convolutions introduce a super parameter called a “dilation rate”, which defines the interval between values when a convolution kernel processes data. The advantage of dilated convolution is that it increases the receptive field without pooling information so that each convolution output contains a large range of information. It can be well applied to the problem when the image needs global information or the voice text needs long sequence information.

Spectrum prediction can be regarded as an image reasoning task. Moreover, in the time domain, the dependent learning of long-term historical data is an effective method to improve the accuracy of the model. Therefore, we believe that the convolution of expansion is more conducive to the learning of the time-frequency domain. Specifically, we use dilated convolution, batch normalization, and activation to form a dilated convolution unit, as shown in Figure 3.

4. Experiments and Results

In this section, numerical experiments are provided to verify the effectiveness of the proposed method.

4.1. Dataset Introduction

The experiment data come from ElectroSense [34], an open source spectrum monitoring project. In order to verify that the method proposed in this paper can play a good role in different scenarios, we selected two different types of spatial-scale monitoring data as the datasets of this experiment. Specifically, we selected monitoring data from multiple sensors in Madrid, Spain, and its surrounding areas from the ElectroSense platform to build a dataset with a small spatial scale and only a few towns. In this paper, we call it the Madrid dataset.

The frequency range of the dataset is 790–820 MHz, which is a part of the ultra-high frequency (UHF) band and is called the digital dividend. It was first assigned to analog and digital TV broadcasting and was approved for mobile broadband and other services after 2010. In fact, due to the coexistence of multiple services, signal interference problems often occur. In addition, due to the inconsistent policies of the management departments and the differences in the services provided by the operators, the uses of this frequency band in different cities and regions are also different. The analysis and prediction of this frequency band will help government departments to grasp the current status of frequency use in different regions and rationally re-plan spectrum resources. The specific measurement parameters of the dataset are shown in

Table 1. Measurement parameters of the Madrid dataset.

Dataset	Madrid Dataset
Location	Madrid, Spain
Number of sensors	5
Space span	about 15 km × 10 km
Time span	14 days (2 May 2021–15 May 2021)
Frequency span	790–820 MHz
Time resolution	15 min
Frequency resolution	1 MHz

.

In addition, we also selected data from multiple sensors in Switzerland to build a large-spatial-scale dataset. This dataset includes several cities in Switzerland such as Zurich, Bern, Lausanne, and St. Gallen, which in this paper, we call the Switzerland dataset. The specific measurement parameters are shown in

Table 2. Measurement parameters of the Switzerland dataset.

Dataset	Switzerland Dataset
Location	Madrid, Spain
Number of sensors	8
Space span	about 120 km × 210 km
Time span	14 days (2 May 2021–15 May 2021)
Frequency span	790–820 MHz
Time resolution	15 min
Frequency resolution	1 MHz

.

There are a few missing values in the original data we obtained, mainly due to the lack of monitoring results of a small number of timeslots (possibly due to the monitoring equipment). Therefore, we interpolated the data before the experiment. Specifically, we use the linear method to fit the frequency point sequence and then complete the observations of the missing positions according to the fitting curves.

4.2. Training and Hyperparameter Set

In this experiment, we train and evaluate the energy value (power spectral density) of the observed data. We set the input time step as 36 and the output time step as 12. Because the time resolution of both datasets is 15 min, the experiment uses the historical 9 h observation data to predict the spectrum energy in the next 3 h. For the model training, the min-max normalization method is used to scale the data values into the range [$-1$, 1]. In the evaluation, we rescale the predicted values back to the normal values, then compare them with the ground truth. For the two datasets, we divide the training set, verification set, and test set according to a ratio of 7:1:2. The mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage error (MAPE) are taken as the evaluation indexes. Their expressions are as follows:

$$\begin{array}{cc}\hfill \mathrm{MAE}& =\frac{1}{n}\sum _{i=1}^n\left|{\widehat{y}}_i-y_i\right|\hfill \\ \hfill \mathrm{RMSE}& =\sqrt{\frac{1}{n}\sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}\hfill \\ \hfill \mathrm{MAPE}& =\frac{100\%}{n}\sum _{i=1}^n\left|\frac{{\widehat{y}}_i-y_i}{y_i}\right|\hfill \end{array},$$

(19)

In order to minimize the overall loss, maintain the stability of the model, and obtain better local optimal values, a curriculum learning strategy is used [35]. At the beginning of training, only the first time step is calculated as the training loss. Then, we set a time step training parameter. Each time the number of training iterations reaches this parameter, a time step is added for calculating the loss. With this method, the loss calculation is gradually extended to multiple time steps with the increase in the number of iterations. By controlling the parameters, all time steps are included in the loss calculation before the end of training.

The hyperparameter settings of the time–frequency–space joint prediction model proposed in this paper are shown in

Table 3. The hyperparameter settings of the time–frequency–space joint prediction network.

Hyperparameter	Value
Number of TFS blocks	3
Number of GCNs	3
Dilation rate	(2, 2)
Kernel size of dilated convolution	3 × 3
Number of filters in dilated convolution	(128, 256, 512)
Number of filters in skip connection	(128, 256, 512)
Activation function	ReLU
Number of epochs	100
Learning rate	0.0005
Weight decay	0.0001
Optimizer	Adam
Loss function	Mean Absolute Error

. The number of GCNs in the table represents the number of graph convolution layers in each TFS. According to the existing theory, the number of GCN layers should not be too large so it was set to three in the experiment.

4.3. Baselines

We selected five methods as the baseline models, including single time prediction and multi-domain joint prediction. The following is a detailed introduction to the baseline models:

• ARIMA: The ARIMA model is a classical time-series prediction method. Through the observation of historical data, the number of autoregressive terms, the number of different times, and the number of moving average terms of the model parameters are, respectively, set as 4, 1, and 0. In the experiment, we build models for different channels and different sensors and finally take the average value of the prediction error as the comparison result.
• 1D LSTM: This model includes two LSTM layers and one output layer. The dropout in the LSTM layer is set to 0.3. The numbers of hidden units are set as 128 and 512, respectively. This model predicts time series only, and the mean value of the prediction error is taken as the comparison result.
• 2D LSTM: This model is a time-frequency joint prediction model and includes two LSTM layers and one output layer. The dropout in the LSTM layer is set to 0.3. The numbers of hidden units of the LSTM are 128 and 512, respectively. The mean value of the prediction error of the different spatial positions is taken as the comparison result.
• CNN+LSTM: This model is the most popular time-frequency joint-prediction model at present and includes two CNN units, two LSTM layers, and one output layer. The CNN unit includes a convolution layer with a kernel size of $3\times 3$, a batch normalization layer, and an activation layer. The filters of the convolution layer are set to 64 and 128, and the hidden units of the LSTM are set to 128 and 512. The mean value of the prediction error of the different spatial positions is taken as the comparison result.
• ConvLSTM: The ConvLSTM model is an advanced time–frequency–space joint prediction model. The model includes two ConvLSTM layers and one output layer. The convolution kernel size of the ConvLSTM layer is $3\times 3$ and the numbers of hidden units are 64 and 128.

4.4. Results

We used the two datasets discussed in Section 3.1 to verify the performance of the proposed analysis and prediction architecture and compare it with the baseline models. First, we analyzed the spatial activity correlation of the entire 14-day historical observation data in the datasets. Specifically, we constructed 5 × 5 and 8 × 8 adjacency matrices for the different sampling locations in the Madrid and Switzerland datasets to characterize the correlation of the spectral activities. In the actual operation, we selected the threshold in Equation (13) as $Thr=0.3$ based on experience. Based on the adjacency matrix, the original observation data can be regarded as the graph-structure data in the spatial domain. Then, we took the observation data and adjacency matrix together as the input of the prediction network model. According to Equation (14), the Laplace operator required for the graph convolution is constructed using the adjacency matrix, and the spatial and time-frequency characteristics of the data are learned through the graph convolution and expansion convolution, respectively. Finally, we used the parameters and training strategies described in Section 3.2 to train the proposed model and the baseline models.

Table 4. The average prediction errors of the proposed and baseline models.

Dataset	Methods	MAE	RMSE	MAPE
Madrid dataset	ARIMA	1.3646	2.6571	5.7903%
	1D LSTM	1.1974	2.4479	5.2381%
	2D LSTM	1.1333	2.3663	4.9895%
	CNN+LSMT	1.0428	2.2234	4.7639%
	ConvLSTM	0.9915	2.0481	4.4344%
	Proposed	0.9094	1.7384	3.9719%
Switzerland dataset	ARIMA	1.3017	2.0680	6.2210%
	1D LSTM	1.2278	1.9050	5.9658%
	2D LSTM	1.1381	1.8181	5.4785%
	CNN+LSTM	1.0162	1.6283	4.8030%
	ConvLSTM	1.0909	1.7569	5.2469%
	Proposed	0.8969	1.4534	4.2126%

shows the average prediction errors of the proposed model and the baseline models, where the smallest error results are indicated in bold.

The results in Table 4 show that the proposed method was superior to the baseline models on two different datasets. On the Madrid dataset, the MAE of the proposed method was reduced by 8.3% compared to the second-ranking ConvLSTM model. On the Switzerland dataset, it was 11.7% lower than the CNN+LSTM model. These results show that our method has a good performance improvement compared to the baseline models. In addition, we found a pattern that showed that the multi-domain joint prediction was usually better than the time-series prediction. In the Madrid dataset, due to the small spatial distances and strong correlation between the different monitoring locations, the time–frequency–space joint predictions (ConvLSTM and the proposed) were better than the time-frequency joint predictions (2D LSTM and CNN+LSTM). In the Switzerland dataset, the larger spatial distance made the correlation between different monitoring locations more complex. At this time, the performance of the convolution-based ConvLSTM model declined and was even lower than that of the CNN+LSTM model, although the proposed method still maintained good performance. This shows that our method can achieve good results at different spatial scales. Figure 4 describes the prediction errors of the models at each time step. One time step in the experimental data represents 15 min so the following figure reflects the prediction error curve for every 15 min in the subsequent 3 h.

In Figure 4, it can be seen that almost all models had similar prediction accuracies at the first time step. The performance difference among the models was mainly reflected in the follow-up time steps. In addition, in the multi-step prediction, the proposed model demonstrated a more gradual increase in errors, that is, the proposed model showed more obvious performance advantages with increasing time steps.

5. Conclusions and Discussion

5.1. Conclusions

In the context of the current shortage of spectrum resources and the gradual deterioration of the electromagnetic environment, the analysis and prediction ability of spectrum resource activity rules play a vital role in improving spectrum utilization efficiency, reducing spectrum conflict risk, and ensuring the sustainable development of the industry. Aiming at the complexity of the spatial distribution of spectrum data, this paper constructs a graph-based joint analysis and prediction architecture by combining the analysis technology of big data and the powerful fitting ability of deep networks. This architecture breaks through the problem of the insufficient use of prior information in existing spectrum resources research and has certain reference significance for future research in the field of electromagnetic spectra.

In the numerical experiments in Madrid and Switzerland, we verified the advantages of the proposed model for joint prediction, and the graph-based method was more suitable for applications in space scenes. At the same time, we also verified the advantages of the multi-domain joint prediction model over the single time prediction model. The results showed that multi-domain integration could be a trend in future research and more information (such as weather, human activities, and geomorphic features) may be included in the scope of joint predictions in the future.

Compared with the baseline models, the proposed method effectively reduced the prediction errors of spectrum resources and showed more obvious advantages on the Switzerland dataset with a larger spatial range. This shows that the research results of this paper can be used as a reference to improve the prediction ability of spectrum resources in the spatial domain.

5.2. Limitations and Future Research

Although the method proposed in this paper achieved better results than the baseline models, in fact, there are still many details of the method in this paper that can be improved. Here, we mainly discuss two important points:

Feature selection can be optimized for spectrum activity analysis. The analysis of the spatial correlation in this paper uses eight artificially selected features, which are all given equal weight. In practice, these features are selected based on experience. Obviously, this scheme is not an optimal solution and there may be invalid or even negatively effective features. In addition, different datasets can use different feature combinations and weights. This problem can be viewed as an optimization decision problem.

In terms of spectrum prediction, the problem of the practical deployment of the model needs to be solved. In fact, the use of electromagnetic spectra is constantly changing. Therefore, offline learning is out of date and online learning strategies should be adopted. At the same time, transfer learning can be used to speed up the model deployment.

In response to the above two issues, this paper concludes with an outlook on future research. First of all, for the optimization problem of feature selection, we believe that advanced optimization algorithms (such as heuristic, meta-heuristic) are effective. Advanced optimization algorithms have been used as mature solutions in many different fields such as resource scheduling, decision optimization problems in transportation [36], medical care [37], etc. In future research, feature selection and weight setting can be optimized to further improve the effect of spectrum analysis and prediction. A potential problem is that there is no widely accepted metric for measuring the quality of the analysis results. Although the final prediction accuracy can be seen as an indirect measure, it is still not enough.

For the problem of predictive model deployment, we believe that the emerging online transfer learning technique is a feasible solution. It can learn domain-invariant features offline and adapt the predictive model to new data by updating the policy online [38]. This method can quickly adapt the model to new data through online migration, and at the same time, enable the model to have the capability for continuous learning. This study has important implications for model deployment.