Adversarial Sample Generation Method for Modulated Signals Based on Edge-Linear Combination

Abstract

In complex electromagnetic environments, wireless communication system reliability can be compromised by various types of jamming. To address the issue of jammers using deep neural network models to identify communication signal modulation method and apply targeted interference, this paper proposes a method for generating adversarial samples of modulation signals based on the Mixup linear combination approach. The method generates edge-linear combination samples with small perturbations by linearly combining the original signal samples near the decision edges, and then inputs them into the neural network model for identification test, determines the best perturbation signals for each type of signals according to the identification results, and then generates the adversarial samples by selecting the best perturbation signals for each type of modulation during the attack. Simulation results show that, compared to traditional gradient-based adversarial sample generation algorithms, the proposed method performs better under white-box attacks. Under black-box attacks, the proposed method achieves higher attack success rates and lower attack signal-to-noise ratios compared to random noise adversarial samples with the same disturbance coefficient.

1. Introduction

In recent years, the use of wireless communications has significantly increased. However, due to the openness of the wireless channel, wireless communications are facing increasing interference threats. In addition to unintentional interference from various frequency-using devices [1], malicious interference is also a major threat to wireless communication systems. In modern wireless communication systems, modulation technology, as one of the core technologies of wireless communication transmission, assumes the important responsibility of efficient and accurate information transmission. Modulation is the process of embedding an information signal into a carrier signal, and common modulation methods include amplitude modulation (AM), frequency modulation (FM), phase modulation (PM), and various digital modulation methods. In communication confrontations, if a jamming device can identify the modulation mode of a wireless communication system, it can apply targeted and efficient jamming against that specific modulation mode [2]. This type of jamming poses a greater threat to wireless communication systems than conventional jamming, as shown in Figure 1. Currently, modulation recognition methods have evolved from maximum likelihood estimation and statistical pattern recognition based on deep learning-based modulation recognition, which significantly improves the accuracy of modulation recognition [3]. Presently, deep learning algorithms for modulation recognition primarily use deep neural networks (DNN). These models automatically extract complex features from large datasets, eliminating the need for manually designed features and significantly improving recognition accuracy. As proposed in the literature [4], a multilayer deep neural network (MLDNN) is designed to enhance the modulation diversity. This is achieved by converting the signal into both a temporal in-phase/quadrature (I/Q) format and an amplitude/phase (A/P) representation, thereby improving the accuracy of signal modulation classification. The literature [5] introduces a multimodal deep learning method that integrates image, amplitude, and phase features, further improving signal recognition accuracy. Meanwhile, the literature [6] proposes a modulation recognition method based on a deep CVCNN-LSTM architecture, combining the strengths of convolutional neural networks (CNN) for feature extraction and long short-term memory networks (LSTM) for sequence learning. These methods significantly enhance both the accuracy and efficiency of modulation recognition in communication systems. This has made it a critical research issue to effectively resist the recognition of wireless communication modulation by jamming devices, thereby preventing targeted and efficient jamming.

The study by Szegedy et al. [7] reveals the limitations and non-interpretability of DNN learning mechanisms, noting that automatic modulation recognition (AMR) algorithms are susceptible to adversarial sample attacks. Adversarial samples can induce false recognition in deep neural network models by carefully adding small perturbations to the original samples. Attacking a machine learning model with adversarial samples is known as an adversarial attack. The literature [8] demonstrates that generating high-quality adversarial signal samples can effectively disrupt the jammer’s modulation mode recognition model, reducing its accuracy and causing it to incorrectly identify the modulation mode. This prevents the jammer from implementing targeted interference, thereby protecting normal communication. Therefore, generating high-quality adversarial samples has important practical significance and application value.

Currently, many adversarial sample generation methods that are widely used in the field of image recognition have been extended and applied to the generation of electromagnetic signal adversarial samples. Among them, fast gradient sign method (FGSM) [9] and its variants [10,11] are common attack methods. These methods generate adversarial samples by adding perturbations in the opposite direction of the gradient, which in turn affects the output of the model. However, such methods are global attacks and cannot implement precise perturbations. To address this problem, the Jacobian Saliency Map Attack (JSMA) [12] was developed. This method uses the Jacobian matrix of the neural network output relative to the input to identify the input features that have the greatest impact on the model prediction and accordingly generates the adversarial samples with the largest perturbations. However, these methods rely on model gradient information and may be more limited in practical applications, particularly in models where confidence and loss functions are unavailable, potentially causing failure.

Various explanations have been given by researchers regarding the reason for the existence of adversarial samples [12]. Shamir et al. [13] explored the existence of adversarial samples in neural networks and pointed out that deep neural networks usually work in high-dimensional spaces, and the decision boundaries of these models are very complex and nonlinear. On such complex decision boundaries, the adversarial samples and the original samples have only a small Hamming distance.

In our experiments, we observed an interesting phenomenon: when different samples are fed into the neural network by linearly weighted combinations, even though these combinations are very close to the original samples (e.g., linear weighting coefficients are less than 0.06), they may still lead to classification errors in the neural network. We refer to such linear combinations that are very close to the original samples as edge-linear combinations. It has been shown that the decision boundary of a neural network may be very close to the training samples [14], which implies that edge-linear combinations may contain adversarial samples.

This paper examines whether adding perturbations near the decision boundaries of the original samples can influence the recognition model’s decision. It also proposes a method for generating adversarial samples using edge-linear combinations. The proposed method is based on the property that the adversarial samples may be located near the edges of the original samples, and the original samples are first processed by the edge-linear combination. Then, their recognition results in the model are counted in order to determine the optimal perturbation signals for each class of samples that are most susceptible to the method. When performing the attack test, a specific best perturbation signal is selected based on the original signal class for edge-linear combination to generate the adversarial samples.

The contributions of the paper are mainly the following:

• The edge-linear combination adversarial sample generation method can quickly generate effective adversarial samples. It does not depend on the intrinsic parameters of the deep neural network model, and the direct generation of the adversarial samples can be achieved only by the input and output results.
• The proposed method reduces the neural network recognition accuracy by 30% on average, compared with the traditional gradient attack method under the white-box targetless attack; under the black-box attack, the method described in this paper has a higher attack success rate and a lower attack signal-to-noise ratio compared with the random noise adversarial samples with the same perturbation coefficients.

The remainder of this paper is organized as follows:

Section 2 provides a comprehensive overview of traditional gradient-based adversarial sample generation methods. In Section 3, the principles of the Mixup data linear combination method are introduced. Section 4 presents the algorithmic implementation of the edge-linear combination method, including detailed steps and a flowchart. Section 5 reports simulation experiments conducted to evaluate the performance of the proposed method and compares its success rate with other adversarial sample generation methods under both white-box and black-box attacks. Section 6 summarizes the main contributions of this study. Section 7 discusses the directions and prospects for future work.

2. Gradient-Based Adversarial Sample Generation Method

The adversarial sample generation process can be represented as (1):

$$\left\{\begin{array}{l}{x}^{adv}=x+\arg \min {‖\delta ‖}_P\\ F\left(x^{adv}\right)\ne F\left(x\right)\end{array}\right.$$

(1)

where x is the original sample, $F\left(·\right)$ is the DNN model operation, $x^{adv}$ is the adversarial sample, and $\arg \min {‖·‖}_p$ is used to describe the search for the $\delta$ minimum by reducing the paradigm. P is a paradigm constraint on the perturbation. 0-parameter constrains the number of non-zero elements in the perturbation, 2-parameter constrains the modulus of the perturbation, and ∞-parameter constrains the maximum value of the perturbation.

When choosing an adversarial sample generation method, the decision needs to be made based on the specific attack target, the limitation of computational resources, and other factors. Several commonly used gradient-based adversarial attack techniques are described below. Fast Gradient Sign Method (FGSM) is a classical gradient attack algorithm proposed by Goodfellow et al. [9]. By calculating the gradient of the loss function of the input sample to the input data, and then adding the same perturbation as the gradient method, adversarial samples are generated to increase the model’s prediction error. The advantages of the FGSM are simple to implement and fast to generate, but due to a single-step attack, the generated samples are poor for attacking complex DNN models. Considering the low success rate of adversarial samples generated by single-step computation, Kurakin et al. [10] designed Iterative Fast Gradient Sign Method (I-FGSM) to gradually enhance the perturbation by applying FGSM through multiple iterations. In each iteration, the gradient of the loss function is calculated based on the current sample to update the sample, thus generating a better performance of the adversarial samples. I-FGSM is able to generate better adversarial samples, but with each update, the excessive movement of the adversarial samples along the direction of the gradient will cause the loss function to fall into the non-optimal local extreme point, resulting in the generation of the adversarial samples of poor relocatability. To solve this problem, Dong et al. [11] proposed the Momentum Iterative Fast Gradient Sign Method (MI-FGSM) to accelerate convergence and enhance the cumulative effect of perturbations by integrating the direction of the cumulative gradient in multiple iteration steps into a single momentum term. Although FGSM is a single-step adversarial generation method with low computational overhead, methods such as I-FGSM and MI-FGSM require multiple iterations of gradient computation, which significantly increases the computational cost especially when dealing with complex deep neural networks. For some application scenarios that require real-time processing or limited resources, gradient-based adversarial sample attacks may be less effective.

3. Mixup Data Linear Combination Method

Mixup [15] is a popular data augmentation technique based on the convex combination of sample pairs and their labels approach that has been shown to improve the robustness and generalisation of trained models [16]. The core idea is to generate new random convex combinations of data points and labels from the training set using linear interpolation, and the method allows decision boundaries to transition linearly from one class to another providing smoother uncertainty estimates [17]. The concept of decision boundaries here applied to the adversarial sample attack modulation recognition model can be understood as the minimum value of the perturbation coefficient that adds a perturbation to the samples to make the model recognition wrong. This method linearly combines two different samples $x_i$ and $x_j$ to generate a new sample $x_{\alpha }$. Label $y_{\alpha }$ is also obtained by linearly interpolating $y_i$ with $y_j$ as shown in (2):

$$\left\{\begin{array}{l}{x}_{\alpha }=\alpha x_i+(1-\alpha )x_j\\ y_{\alpha }=\alpha y_i+(1-\alpha )y_j\end{array}\right.$$

(2)

In the process of introducing data augmentation methods into the signal samples to generate the adversarial samples, we seek to add as little perturbation as possible while ensuring a high success rate of the attack on the recognition model. Therefore, when introducing the Mixup method into the generation of adversarial samples, we can flexibly adjust the parameter settings so that the two items in the combination have different weights, with one of them having a larger weight as the active term and the other having a smaller weight as the perturbation term. However, it is worth noting that there are certain requirements on the data itself when using the Mixup method for data enhancement. Especially when dealing with sample sets with large differences or imbalances, such as the common I/Q signal modulation identification datasets RML2016 [18] and RML2018 [19], the direct application of the method may result in the newly generated samples being biased towards the modulation categories with a larger percentage of amplitude, thus affecting the balance and representativeness of the data.

In label processing, the Mixup linear combination method performs a weighted sum of the one-hot encodings of labels x and y, generating z, which represents a weighted combination of class x and class y. This operation allows the model to learn the relationships between classes. However, given that the goal of this paper is to generate adversarial samples that can deceive the recognition model, we still use the labels of their original samples for the combined samples. This edge-linear combination method, which is improved based on the Mixup data enhancement method, not only maintains the constraints of the original labels and avoids the potential interference that may be caused by label inconsistency, but also compensates for the shortcomings of the original method in processing signal samples and retains the ability to quickly generate adversarial samples. By carefully designing and adjusting the parameters, we can ensure that efficient and deceptive antagonistic samples are generated while keeping the data balanced and representative.

4. Adversarial Sample Generation Method Based on Edge-Linear Combination

Traditional gradient-based adversarial sample generation methods require multiple iterations of updating the adversarial samples through the gradient parameters. In contrast, the research in this paper focuses on the adversarial attack in the case where the modulation recognition model parameters and gradient cannot be directly accessed. Under this attack condition setting, an adversarial sample generation method based on edge-linear combination is proposed, and the overall description is as follows:

The dataset after min-max normalisation is first divided into different sets of original signal samples X according to modulation and signal-to-noise ratio, and the modulation of each original signal sample x is recorded as the true label $y_x$. For two given different original signal samples $x_i$,$x_j$$(x_i,x_j\in X)$, define the edge-linear combination of samples for $x_i$ original signal samples as (3):

$$prox(x_{ij})=(1-\alpha )\cdot x_i+\alpha \cdot x_j$$

(3)

where α is the added perturbation coefficient, in order to ensure that the generated $prox(x_{ij})$ is near the neighbourhood of the original signal sample $x_i$, it is necessary to be a smaller number of α. In this paper, we determine $0.05\le \alpha \le 0.07$, so that the edge-linear combination of samples $x_{ij}$ is very close to the original signal sample $x_i$. Then, the modulation of the signal recognition neural network model is $f(x;\theta )$. Here, $\theta$ is the neural network training parameter, which is not directly accessible in the attack scenario given in this paper, and we can only query the output of the neural network for classification of x. The edge-linear combination sample $x_{ij}$ generated for $x_i$ is put into the trained neural network model for classification to obtain the classification prediction label $y_{x_{ij}}$; if $y_{x_{ij}}$ is different from $y_{x_i}$ then the attack is successful. Finally, according to the success rate of different combinations of attacks, the best perturbation sample for the original signal sample $x_i$ is determined, and the edge-linear combination is used to generate the adversarial sample. The adversarial sample generation method is shown in Figure 2, and the steps are described in detail below.

4.1. Edge-Linear Combination of Original Samples

The number of samples is reduced by randomly sampling all n original signal samples $x_i$$(x_i\in X,i=1,2,\cdot \cdot \cdot ,n)$ in the original sample dataset. The number of signals within each sample is reduced by random sampling to k. The sampled data greatly reduces the time consumed by the calculation, while the random sampling method remains representative. Thus, a randomly sampled dataset $X^{RS}$ is obtained, in which each sample is selected as the original sample $x_i^{RS}\left(x_i^{RS}\in X^{RS},i=1,2,\cdot \cdot \cdot ,k\right)$, and then all target samples $x_j^{RS}\left(x_j^{RS}\in X^{RS},j=1,2,\cdot \cdot \cdot ,k,j\ne i\right)$ in $X^{RS}$ that are different from itself are iterated to perform the edge-linear combination. The edge-linear combination of all the randomly sampled data of the original signal samples is obtained as in (4):

$$X_{ij}^{RS}=\left\{(1-\alpha )\cdot x_i^{RS}+\alpha x_j^{RS},(i,j=1,2,\cdot \cdot \cdot ,k,i\ne j)\right\}$$

(4)

where $\alpha$ is set to $0.05\le \alpha \le 0.07$, the entire set of linear combinations satisfies the edge combination condition. Next, the generated edge-linear combinations are subjected to neural network model identification to explore whether changes in the vicinity of the original samples can affect the neural network classification changes, and in the next step, the success rate of each combination attack is counted to determine the best perturbation samples.

4.2. Determine the Optimal Perturbation Sample

For the original samples $x_j^{RS}$ and target samples $x_{ij}^{RS}$ with different modulation categories and signal-to-noise ratios in the randomly sampled dataset $X^{RS}$, the $x_{ij}^{RS}$ generated after the edge-linear combination operation is input into the neural network $f(x_{ij}^{RS};\theta )$ for querying. The corresponding query results are obtained $y_{x_{ij}^{RS}}$, and the number of all edge-linear combination samples that are out of order is counted, which is denoted as $N\left(x_{ij}^{RS}\right)$ shown in (5):

$$N\left(x_{ij}^{RS}\right)=\left|\left\{x_{ij}^{RS}|y_{x_{ij}^{RS}}\ne y_{x_i}\right\}\right|$$

(5)

where $\left|\left\{·\right\}\right|$ is the power of the set and represents the number of elements in the set. For sample $x_i^{RS}$, check all its edge-linear combinations of samples $x_{ij}^{RS}$ to identify the number of errors, and define the target sample $x_j^{RS}$ with which it has the most combinations of errors as the best perturbation signal $x_{OP}^{RS}$ for sample $x_i^{RS}$ under perturbation α. The target sample c is the one with the most errors.

4.3. Generate Adversarial Sample

After determining the optimal perturbation signal for a signal sample under a specific modulation mode and signal-to-noise ratio, when it is necessary to generate an adversarial sample for a certain signal, the optimal perturbation sample corresponding to the signal can be selected based on a priori experience and edge-linear combination can be carried out to quickly generate an effective adversarial sample. The algorithm flowchart is shown below (Algorithm 1).

Algorithm 1 Steps

Input: $x_i^{RS}\left(x_i^{RS}\in X^{RS},i=1,2,\cdot \cdot \cdot ,k\right)$, $x_j^{RS}\left(x_j^{RS}\in X^{RS},j=1,2,\cdot \cdot \cdot ,k,j\ne i\right)$, $\alpha$$\left(0.05\le \alpha \le 0.07\right)$, CLDNN model $f(x;\theta )$ Output: the optimal perturbation sample $x_{OP}^{RS}$

1. An edge-linear combination of $x_i^{RS}\left(x_i^{RS}\in X^{RS},i=1,2,\cdot \cdot \cdot ,k\right)$ and $x_j^{RS}\left(x_j^{RS}\in X^{RS},j=1,2,\cdot \cdot \cdot ,k,j\ne i\right)$ using (4) yields $X_{ij}^{RS}$;

2. Input $X_{ij}^{RS}$ into neural network $f(x_{ij}^{RS};\theta )$ for query, find the corresponding query result $y_{x_{ij}^{RS}}$;

3. Use (5) to count the number of errors for all samples of edge-linear combinations, denoted as $N\left(x_{ij}^{RS}\right)$;

4. Combination of the most error-prone target sample locus $x_{OP}^{RS}$;

5. Return $x_{OP}^{RS}$.

In the process of generating adversarial samples, based on the framework of the edge-linear combination method, we can accurately set the coefficients α of the edge-linear combination with the help of the maximum infinite-paradigm perturbation that the signal can withstand. The selection of this coefficient directly determines the degree of fusion between the original signal samples and the optimal perturbation signals, which significantly affects the generation effect of the adversarial samples. Specifically, by adjusting the value of α, the strength of the perturbation and the corresponding attack effect can be effectively controlled; when the value of α increases, the attack effect is obvious, while at smaller values of α, the perturbation is more subtle and difficult to detect. Figure 3 depicts the relationship between the attack effect of the generated edge-linear combination samples on the modulation recognition neural network and the α value.

5. Simulation Setup and Performance Analysis

5.1. Simulation Setup

The experiments in this paper use the I/Q signal modulation identification dataset RADIOML2016.10A, which contains 220,000 signal samples with 11 modulation modes, including three types of analog signals (WBFM, AM-DSB, AM-SSB) and eight types of digital signals (QAM16, QAM64, 8PSK, BPSK, CPFSK, GFSK, PAM4, QPSK). There are 20 signal-to-noise ratios ranging from −20 dB to 18 dB for each class of signal, and each signal sample contains both in-phase and quadrature data of length 128. Considering that in the case of too low signal-to-noise ratio, the noise occupies a larger proportion in the signal, the signal quality is low, and the probability of error of the neural network for recognition without adversarial sample modulation is greatly increased. So, this experiment will focus on high signal-to-noise ratio to ensure the stability and reliability of the experimental results.

Before training the modulation recognition neural network, the original dataset needs to be preprocessed for the operation. The signals of the original dataset have significant differences in the range of values between different samples. For example, AM-DSB analogue signals can have a maximum amplitude of up to 20, whereas digitally modulated signals such as 8PSK have a global amplitude of only between 0 and 1. Considering that the difference in signal amplitude between different modulation modes is not so large in the actual communication situation, and that if the high amplitude signal is added to the low amplitude signal as the target signal, even though the perturbation coefficients are set to be small, the low amplitude signal will still be changed a lot, which is not in accordance with the requirement of the edge-linear combination; therefore, min-max normalisation is performed on all signals before the training.

Considering the characteristics of the signal samples, model parameters and recognition effect, two modulation recognition models, CLDNN (Convolutional Long Short-Term Memory-Deep Neural Network) and ResNet (Residual Neural Network), are selected. The CLDNN model consists of an input layer, six convolutional and max-pooling layers, two LSTM layers with 128 units each, and two fully connected layers. The input layer has a size of 2 × 1024, and the convolutional layers use 3 × 3 kernels. The ResNet model, on the other hand, includes an input layer, six residual blocks, two fully connected layers, and an output layer. The input size is also 2 × 1024. Each residual block contains a 1 × 1 linear convolutional layer, two residual units, and a max-pooling layer. The convolutional layers in the residual blocks have 32 filters, with a kernel size of 3 × 1. Each model is trained for 180 rounds with an initial learning rate of 0.001, and during training, the loss function of the validation set is halved if it does not decrease for five times in a row. The hidden layer is uniformly selected as the last fully connected layer in front of the output layer. Then, 300 samples were randomly selected as a subset from each modulation category and signal-to-noise ratio, i.e., a total of 56, 100 samples, or 30% of the total samples in the original dataset.

In the following experiments, the perturbation factor of the edge-linear combination does not exceed 0.07 and here the x perturbation factor is set to three different values of 0.05, 0.06, and 0.07. Similarly, for the white-box and black-box attacks, the perturbation values for the gradient attack on the original signal and the addition of random noise are the same as the x-perturbation factor set for the edge-linear combination.

The hyperparameters are set consistently, ensuring that the values of the perturbation coefficients remain unchanged when performing hyperparametric analyses, and only the number of random samples is changed. The aim is to find an optimal number of samples, which not only ensures the speed advantage of generating adversarial samples, but also takes into account the quality of the adversarial samples.

5.2. White-Box Experimental Analysis of Edge-Linear Combination Samples

In order to determine the effect of individual samples under different modulation methods and signal-to-noise ratios on the classification of the recognition model, the following edge-linear combination of samples $x_{ij}^{RS}\left(i,j=1,2,\dots ,k,i\ne j\right)$ generated in the subset is fed into the CLDNN recognition model; the number of $x_{ij}^{RS}\left(i=1,2,\dots ,k,i\ne j\right)$ errors corresponding to each $x_i^{RS}\left(i=1,2,\dots ,k\right)$ is counted $N\left(x_{ij}^{RS}\right)$, and the success rate of the attack is calculated. The attack success rate is compared to determine the optimal perturbation samples for this modulation and SNR, and applied to the original dataset to obtain the adversarial samples for each signal in the original dataset. Figure 4 illustrates the heatmap of the success rate of the attack of each sample of the edge-linear combination when the value of the perturbation coefficient is 0.06 and the signal-to-noise ratio is 0. The horizontal axis is the original signal, the vertical axis is the added target signal, and the diagonal data of the picture is 0. Because the signal of the experimental setup is not combined with itself for the edge-linear combination, it can be seen that the edge-linear combination of adversarial samples works best for modulations 8PSK and GFSK, while AM-SSB is the best for generating adversarial samples for all modulations, as target signals for this perturbation value and signal-to-noise ratio and its generated adversarial samples result in an average of 45.45 percent recognition error rate for different signal-to-noise ratios.

Then, the white-box targetless attack is performed under the CLDNN model using the above methods and the four gradient-based adversarial sample generation methods: the FGSM [9], the I-FGSM [10], and the MI-FGSM [11]. The model recognition error rate is shown in Figure 5.

Comparing the four kinds of adversarial samples on the model recognition error rate, the following can be observed: (1) The effect of different adversarial samples on the same model is different—the attack effect of FGSM is the worst because this method is a single-step attack. In the nonlinear depth model, its gradient direction may be incorrect, resulting in the generation of adversarial samples with poor effect, and in case the signal-to-noise ratio is greater than 2 dB, it will only increase the recognition error rate of the model by five percentage points or so. (2) Both I-FGSM and MI-FGSM gradually increase the adversarial perturbation by updating the gradient, and the effectiveness of the adversarial samples generated by both methods is better than that of the single-step attack method FGSM. MI-FGSM also introduces a momentum term to increase the stability and effectiveness of the adversarial samples, so the adversarial samples generated by the MI-FGSM method with a signal-to-noise ratio lower than 0 dB will increase the model recognition. Therefore, the MI-FGSM method increases the model recognition error rate by 20% at a SNR lower than 0 dB, and by 15% at a SNR higher than 0 dB. (3) The edge-linear combination adversarial sample generation method has been used in this paper. Although its attack effect is slightly inferior to the gradient-based adversarial sample generation method in the case of low signal-to-noise ratio, the edge-linear combination attack effect is optimal when the signal-to-noise ratio is higher than −4. Especially, it improves the model recognition error rate by 30 percentage points near the 0 dB.

Table 1. White-box attack runtime on CLDNN.

Method	Time/h
FGSM	0.12
I-FGSM	0.81
MI-FGSM	0.9
edge-linear combination	0.08

shows the time required for different algorithms to complete the attack with all samples under the CLDNN recognition model in the white-box scenario. Because I-FGSM and MI-FGSM need to constantly update the gradient direction and iteratively generate the adversarial samples, they take longer time. FGSM is a single-step attack, which requires only a single-step calculation, and therefore takes a shorter time. However, the method described in this paper requires some time to determine the optimal perturbation sample, but once that has been completed, generating adversarial samples is quick and easy in terms of both time and computational complexity. This is one of the advantages of this adversarial sample generation method.

5.3. Black-Box Experimental Analysis of Edge-Linear Combination Samples

In order to verify the migration ability of the adversarial samples generated through the edge-linear combination method on other neural networks, another neural network model, ResNet, which has a similar structure to the CLDNN network, was selected for the experiment. The recognition error rate for clean samples in the ResNet model is 9.86%. In the experiment, adversarial samples identified by the CLDNN recognition model are input into the ResNet model. The recognition error rate of the neural network is then analyzed under different signal-to-noise ratios to examine the migratory behavior of the adversarial samples generated by this method. The experimental results are shown in Figure 6. We compare the edge-linear combination of adversarial samples with three different values of perturbation coefficients added with the adversarial samples generated by adding random noise with the same perturbation coefficients to the original signals. It can be seen that at a perturbation coefficient of 0.05, the adversarial samples with the addition of random noise have a very small effect on the recognition rate of the model, whereas at the same perturbation coefficient, the edge-linear combination of samples leads to a thirty-percentage point increase in the model recognition error rate, and the effect of the attack is more obvious as the value of the perturbation coefficient increases.

Next, the size of the perturbation was statistically analysed. The generated antagonistic sample $x_{ij}$ is subtracted from the original clean sample $x_i$ to obtain the antagonistic perturbation ${\delta }_i$, and the power of the antagonistic perturbation is calculated by computing its 2-paradigm mean as in (6):

$$P_{\delta }={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\delta }_i^2}$$

(6)

where N is the number of samples of the signal and ${\delta }_i$ is the perturbation value at i-th sample point. Then, the obtained adversarial perturbation power is divided by the original signal power to obtain the ratio of the adversarial perturbation to the original signal power, here denoted as ASNR. A lower ASNR, while maintaining the same model recognition rate, indicates a smaller perturbation amplitude of the adversarial sample, thus demonstrating better performance. The ASNR values for the same model recognition rate with each perturbation coefficient are shown in Figure 7. It can be seen that the adversarial samples generated by adding perturbations using the edge-linear combination have less perturbation power.

In addition, the experiment also observed whether the change in the waveform of the adversarial samples generated by individual signal samples after edge-linear combination was significant. Taking = 0.06 as an example, Figure 8a shows a signal with modulation class BPSK and signal-to-noise ratio value of six. After edge-linear combination with its best perturbed sample (b), the generated adversarial sample (c) is misidentified as 64-QAM in the modulation in the ResNet model. It can be seen that the waveforms do not change significantly, which proves that the method has a better concealment property.

5.4. Hyperparametric Analysis

The study of hyperparameters needs to follow the control variable method, and the rest of the parameters in this section remain unchanged except for the variation of the sampling number N of a subset of the original data samples. If the number of random samples is lower than 5% of the original data samples, the generated sampling dataset is too one-sided and unrepresentative; if it is higher than 40%, the calculation volume becomes too large, negating the purpose of using random sampling method to reduce the calculation volume. Therefore, the value of the sampling number N is incremented from 50 to 400 in steps of 50, and the adversarial samples generated by different random sampling numbers of the original samples are input into the CLDNN model. Then, the generation time of the adversarial samples and the impact on the recognition error rate of the model are statistically evaluated. The results of the experiments are shown in Figure 9. It can be seen that with the increase of the number of samples the attack success rate rises slightly, but when the number of samples exceeds 300, the model recognition error rate does not rise significantly. And a larger number of samples N requires more computational overhead. Considering the effect of the attack and the cost of time, the experiment set N = 300.

6. Conclusions

In this paper, we propose an adversarial sample generation method based on edge-linear combination, which addresses the potential loopholes in automatic modulation recognition (AMR) algorithms based on deep neural network models, and puts forward a new idea to generate adversarial samples without accessing the internal parameters of the models. Through edge-linear combination operation, this paper successfully generates effective adversarial samples without relying on the model gradient information and verifies its advantages under both white-box and black-box attacks in several experimental scenarios. In the experiments, the adversarial sample generation method based on edge-linear combination shows a higher attack success rate than the traditional gradient attack methods (e.g., FGSM, I-FGSM, MI-FGSM), and the generated adversarial samples are able to significantly degrade the accuracy of the modulation recognition model, especially in the case of a high signal-to-noise ratio. In addition, the proposed method in this paper has a stronger control of signal perturbation during the attack process, which makes the generated adversarial samples finer in terms of perturbation amplitude, providing a higher level of concealment and practicality for the adversarial attack.

7. Discussion

The adversarial sample generation method based on edge-linear combinations proposed in this paper can be applied not only in adversarial attacks, but also in adversarial defense, specifically in adversarial training. By assigning the correct labels to the generated adversarial samples and training them alongside normal samples, the model’s robustness can be improved, enhancing its defensive capability. There are various adversarial defense techniques, such as network distillation, adversarial detection, and network verification. Due to space limitations in this paper, a detailed discussion of these methods is not provided. The research team to which the author belongs has already conducted related studies on model defense methods. In the future, we will summarize and refine the findings based on the progress of the research and experimental analysis.