Design of a Test for Detecting the Presence of Impulsive Noise

This paper proposes a new test method of detecting the presence of impulsive noise based on a complementary cumulative density function (CCDF). Impulsive noise severely degrades performance of communication systems and the conventional Kolmogorov–Smirnov (K–S) test may not perform well, because the test does not consider the characteristics of impulsive noise. In order to detect the presence of impulsive noise reliably, the CCDF of measurement samples is analyzed and compared with the CCDF of additive white Gaussian noise to find the difference between those CCDFs. Due to the nature of heavy-tails in impulsive noise, only the maximum difference may not be sufficient for the accurate detection of impulsive noise. Therefore, the proposed method applies the test hypothesis using the weighted sum of all the differences between those CCDFs. Simulation results justify that the proposed test is more robust and provides lower miss detection probability than the K–S test in the presence of impulsive noise.


Introduction
Most existing digital communication systems are designed to perform optimally in an additive white Gaussian noise (AWGN) environment, but there are some environments where impulsive noise is dominant [1,2]. For example, the performance of the vehicular network in the presence of a cellular network is limited by impulsive noise [3,4]. Since the performance of conventional communication systems is severely degraded in such an environment, detecting the presence of impulsive noise is an important task in most communication systems [5]. Upon the detection of impulsive noise, a receiver can execute a special filter, such as a non-linearity pre-processor [6,7] which is designed based on the characteristics of impulsive noise, to circumvent the impulsiveness of the noise and further achieve the optimal performance in an impulsive noise environment [8,9].
Various studies [10][11][12][13][14][15][16][17] that considered impulsive noise, including power line communications [10] and environments such as a underwater environment [11] in their conclusion. A noise detection method was researched [12][13][14][15][16][17] in an environment with impulsive noise. A previous study [12] reported the actual disturbances encountered by devices operating in one of the bands of interest during normal use. Time-domain measurement procedures are proposed to measure radiated transient interference, which is a source of distortion that offers performance degradation of wireless digital communication systems [13][14][15]. Because transient electromagnetic interference has an enormous impact on the performance of mobile communications on railways, the analysis tool of electromagnetic interference is presented based on a computation of minima so that the power ratio between transmitted signal and interference is smaller than a certain threshold [16,17], but it is difficult to adapt this scheme to a practical system since the threshold is numerically calculated. noise and their distribution is either Gaussian noise or impulsive noise. Among various impulsive noise models, Middleton class A noise and SαS noise models are widely accepted in the literature because their models are derived from the real physical mechanisms that generate disturbance in communication systems and agree well with physical phenomena [1,2].
The probability density function (PDF) of the instantaneous amplitude of Middleton class A noise is given as [1] where the variance σ 2 m is expressed as where σ 2 is the average power of the noise. The parameter A is denoted as the impulsive index, which is the average number of m concerning the probability e −A A m m! [23]. Γ is called the Gaussian factor which is the ratio of the intensity of the independent Gaussian components and that of non-Gaussian components.
The other noise model is SαS noise model. A characteristic function of SαS noise is defined as where λ is the scale parameter larger than 0, δ is the location parameter with −∞ < δ < ∞, and α is the shape parameter with 0 < α < 2. Note that the PDF of the SαS noise is calculated by inverse Fourier transform, since the PDF does not have a closed form.

Motivation for Noise Classification
As mentioned earlier, noise classification (or an identification of noise statistics) of the surrounding environment is of prime importance for communication systems, since the optimal receiver design is based on the noise statistics to achieve optimal error performance. Note that most existing communication systems are designed to achieve optimal performance in an AWGN environment and those existing systems suffer significant performance loss in another noise environment. Figure 1 shows a bit error rate (BER) performances of the conventional receiver with a binary phase-shift keying (BPSK) in impulsive noise and AWGN. In the case of an AWGN environment, the BER graph of BPSK is exponentially decreased as an SNR increases. When impulsive noise is present, however, the BER graph of BPSK shows a plateau due to the impulsive component. This clearly indicates that there is a performance loss for the conventional receiver when impulsive noise is present. Thus, a receiver needs to first classify the noise model in the given environment as accurately as possible in order to avoid or mitigate such performance loss. For this reason, it is essential to classify the noise model to decide a valid strategy since the communication method depends on the noise environment.

Conventional Kolmogorov-Smirnov Test
The K-S test is a non-parametric test based on the maximum difference between an empirical and a hypothetical cumulative distribution [24]. Failing to reject the null hypothesis shows that two data populations are drawn from the same distribution to a certain required level of significance. On the other hand, accepting the alternative hypothesis that they are from different distributions can be used. The hypotheses are stated as where F 0 is a hypothesized CDF and F 1 is an empirical CDF of the amplitude of received samples. Here, the hypotheses H 0 and H 1 are stated as a set of Gaussian amplitudes and a set of non-Gaussian amplitudes, respectively. The theoretical CDF of AWGN amplitude for the K-S test is computed as where σ 2 z = N 0 2 and N 0 is a complex noise power. In the K-S test, the largest absolute difference between the empirical and the hypothesized CDFs is used as the goodness-of-fit statistic, which is given by where F 1 (x) is the value of the empirical CDF evaluated at x. For practical implementation with N measurement samples, where t 1 , . . . , t N are the amplitudes of those samples in ascending order, the difference between the empirical and the hypothesized CDFs for practical environments is computed asD In addition, an empirical CDF from the measurement samples is computed as where I(·) is an indicator function that gives one if the condition is true for a sample, and zero otherwise and t i is the amplitude of the received sample in ascending order. The hypothesis H 0 is rejected at a significance level γ once P(D >D) < γ, where P(D >D) is given as [18] where [24] has demonstrated that Q(d) is convergent with thousands of m terms that may have large complexity for small error.
For detecting the presence of impulsive noise, the hypotheses in (5) are stated as

Proposed Test
Although the K-S test is a common method to measure the goodness of a fit, [25] mentions that a test using K-S statistic to impulsive noise samples have poor performance since K-S statistic does not consider the characteristics of non-normal samples. This section proposes a test to detect the presence of impulsive noise considering the characteristics of impulsive noise. First, the idea behind the design of the proposed test is discussed. Then, an approximation of the threshold for the test hypothesis to satisfy target false alarm probability is presented.

Design of the Proposed Test
The discriminating feature for detecting the presence of impulsive noise is CCDF of the received signals are normalized amplitude since the bell-curve density of impulsive noise has a heavy-tails. The CCDF of the normalized amplitude of the received signal is compared with the reference CCDF of AWGN. Let us consider a measurement campaign that observes N noise measurement samples, where t 1 , · · · , t N are the amplitudes of those samples in ascending order, i.e., t i is the i-th smallest amplitude of the measurement samples. Since the value of t i , the i-th smallest amplitude of the measurement samples, may not be the same for every measurement campaign, t i can be considered as a discrete random variable. Lett i denote the actual value of the i-th smallest sample from a measurement campaign. Figure 2 illustrates the theoretical CCDF of complex Gaussian distribution and the empirical CCDF, which consists of those measurement samples with unknown distribution. The empirical CCDF is a decreasing step function that has the same number of steps as the number of samples because the number of measurement samples is finite. D i is the absolute difference between the two CCDFs for t i and is can be represented as which is a random variable since t i is a random variable. When measurement samples are available, D i is defined as the absolute value of the difference att i and is computed aŝ where F 0 (t i ) is computed by substituting t intot i in (6) and F 1 (t i ) is given as i N . That is, all the values ofD i are obtained by (13) wheret 1 , · · · ,t N−1 in ascending order. Since the empirical CCDF ont N equals to zero,t N is not considered. Comparisons are carried out between the received signals CCDF and reference CCDF as shown in Figure 2. Unlike the conventional K-S test, the proposed test method considers all the values ofD i between the CCDFs in order to detect the presence of impulsive noise reliably. In addition, we employ a weighted sum of the probabilities of differences between the CCDFs for the proposed test since they are not equally weighted due to the characteristic of heavy-tails in impulsive noise. Figure 2. Plotting empirical CCDF with unknown distribution, theoretical CCDFs of complex Gaussian distribution and the differencesD 1 ,D 2 , · · · ,D N−1 between them ont 1 ,t 2 , · · · ,t N−1 for the proposed test. N is the number of the received samples. F 0 (t i ) is the theoretical CCDF ont i under the hypothesis. D i is the differences between empirical and theoretical CCDFs ont i .
To sum up, the proposed test is defined as where γ is the significance level, P i and w i will be discussed below. Finally, an algorithm based on (14) will be depicted. Firstly, P i in (14) is computed as where P i,k is the discrete probability density function of D i and k is the index of the discrete values of D i . K is the set of those indices that satisfy the following condition: P i,k can be obtained by k samples with the probability F 0 (t i ), and N − k samples with the probability 1 − F 0 (t i ), and the number of selecting k samples with the probability F 0 (t i ) among N samples is ( N k ). Therefore, the probability P i,k is computed as Figure 3 shows the empirical and theoretical probabilities of P i,k wheret i = 0.45 and ten samples are considered for the simulation above. It is clearly observed that the probabilities are well-matched.

Computation of Weight
As mentioned in the first section, the probabilities of the differences between the two CCDFs are not equally weighted to determine the presence of impulsive noise. Unlike the conventional K-S test, it is very important to give different weights for all of the received samples according to the difference in CCDF in our proposed algorithm. Therefore, it is significant to capture the CCDF difference more clearly between noise models. Impulsive noise is characterized by heavy-tails, which shows non-linearity in log(log(x)) and log scales. Whereas, AWGN is shown as linearity in log(log(x)) scales. Thus, we use a CCDF of log(log(x)) scale and log scale to calculate the weights using these differences.
The weight for the proposed test is computed based on distributions of the noise. There are two noise models in terms of the distributions: Middleton class A noise and SαS noise models. Thus, we introduce the two weights for the impulsive noise models, respectively. First, Middleton class A noise model is considered. Let us assume that the parameters of noise model are not estimated, because the parameter estimation is carried out after detecting the presence of impulsive noise. Without the values of the parameters, it is impossible to do an exact calculation of the weight. Thus, a graphical approach of Middleton class A noise distribution is used based on the characteristics. Figure 4a illustrates CCDFs for a Middleton class A noise model and AWGN. The scale of the y-axis in Figure 4a shows an obvious difference between CCDFs of Middleton class A noise model and AWGN since the background component of Middleton class A noise and AWGN are illustrated as a straight line in the scale of y-axis in Figure 4a. The straight line can be used in order to compute a weight. Thus, the weight w i,log−log for P i becomes w i,log−log = log 10 ( log 10 (F 1 (t i )) log 10 (F 0 (t i )) )  Second, let us consider SαS noise model. Figure 4b shows CCDFs for SαS noise and AWGN. The CCDFs of SαS noise are plotted on log scale due to the fact that a slope of the CCDF is constant whereas the CCDF of AWGN is sharply decreasing in Figure 4b. Once the impulsiveness gets stronger, the linear curve with a constant slope of the CCDF gets higher. Owing to the linear curve in large amplitude, the logarithm function is suitable for giving more weight P i on a large amplitude due to the extremely low CCDF of AWGN. Since the difference of CCDFs between AWGN and SαS noise is remarkable on log-scale, for SαS noise, the weight of P i is By substituting (18) and (19) into w i in (14), weight sums S log−log and S log respectively corresponding to w i,log−log and w i,log are computed. Then, the hypothesis in (11) is determined by computing with 1 − γ. For detecting the presence of impulsive noise, two weight sums are simultaneously used.
Finally, based on (14), (15), (18), and (19), the proposed algorithm for the test is stated in Algorithm 1. In Algorithm 1, D is a set ofD i . The procedure is briefly explained along with the line numbers corresponding to each step as follows:

2.
P i can be computed by summing the probability P i,k in (15) based on (17) (line 2-4).

3.
S log and S log−log are computed by multiplying P i in lines 2-4 and the weights w i,log and w i,log−log , respectively (line 5-6).

4.
Then, S log and S log−log are compared with 1 − γ that is a threshold to determine if hypothesises are accepted or rejected (lines 7-11).

Approximation of Threshold for the Proposed Test
The test to detect the presence of impulsive noise is proposed in the above subsection. However, a threshold to satisfy a target P FA is not defined. Thus, the threshold to satisfy P FA is derived. The threshold of the proposed test is approximated due to its complexity. To approximate the threshold, the asymptotic performance of the proposed test is analyzed. Figure 5a shows P FA with N = 50, 300 and 1000 to show asymptotic performance of the test. It is clear that the false alarm probability P FA of the proposed test is improved with a higher N but there is no considerable improvement when the number of samples N exceeds 1000. Therefore, we can obtain the approximation of the threshold on the P FA in N = 1000. Value of the threshold in the range between 0.05 to 0.2 is considered in Figure 5a, because the curve in Figure 5a can be modelled as a linear curve. The x-axis and y-axis of the curve are log-scaled in Figure 5a and the relation between log-scaled x-axis and y-axis becomes log 10 (y) = a log 10 ( where a is a slope of the linear function and b is a y-intercept. Thus, the relation between P FA and threshold in Figure 5a is given as log 10 (P FA ) = a log 10 (γ) + b. To determine a and b, two pairs of (log 10 (P FA ), log 10 (γ)) are required and Table 1 shows the pairs obtained from simulation results. Once the two pairs of (log 10 (P FA ), log 10 (γ)) are considered, a is computed as a = log 10 (P FA,γ=0.2 ) − log 10 (P FA,γ=0.05 ) log 10 (0.2) − log 10 (0.05) Also, b is obtained as b = log 10 (P FA,γ=0.05 ) − a · log 10 (0.05) By substituting a in (22) and b in (23) into a and b in (21), the relation is decided as log 10 (P FA ) = 3.0612 log 10 (γ) + 1.4141.
Therefore, the threshold γ is determined as Figure 5b shows the simulation results and proposed threshold in (25) to satisfy target P FA . As shown in the figure, (25) and simulated thresholds are well matched. Table 1. Two pairs of (log 10 (γ), log 10 (P FA )) in N = 1000.

Results and Discussion
In this section, simulation results are provided to evaluate the performance of the proposed test methodology, especially the miss detection probability performance for classification an impulsive noise model under various false alarm probability. For the extensive simulation, the false alarm probability is given as and the miss detection probability is shown as where P D is the detection probability. For simulations, complex impulsive noise samples are generated based on the Middleton class A model in (2) and SαS model in (4), respectively, where the real and the imaginary components are dependent and uncorrelated. For the Middleton class A, the model is characterized by the parameters A and Γ such that the impulsiveness of the noise can be represented with A and Γ. When A is increasing, Middleton class A noise becomes more likely to be Gaussian noise and, conversely, a low value of A represents highly structured impulsive noise especially when A 1 and ΓA 1 [26,27]. Therefore, in order to effectively compare the performance of the proposed test method, we set A and Γ to 0.01 and 5, respectively, for the impulsive noise model. The SαS model is characterized by the parameter α, and if the value of α is small that implies the stronger the impulsive property. In general, the value of α falls into between 1 and 2 in a practical environment [28]. Thus, we consider the SαS model with α equals to 1.8 in this paper.
Receiver operating characteristic (ROC) analysis is widely used in signal processing and communications systems to evaluate the detection performance, where it exploits two-dimensional curves plotted by detection probability against false alarm probability [29]. Note that we adopt the P MD as a performance metric instead of P D , thus the ROC curve is represented with P MD and P FA , in this paper. Figure 6 shows a comparison of P MD of the conventional K-S test and the proposed test in Middleton class A noise with respect to false alarm probability where the number of observed samples are 500. Here, it was observed that the proposed test has a lower miss detection probability than the K-S test in the various impulsive noise environments. In particular, it can be easily observed that the proposed test outperforms the K-S test by providing approximately 30% lower miss detection probability at a P FA = 0.01 for the Middleton class A noise environment. Furthermore, in the case of SαS noise model, the CCDF of the impulsive noise model is similar to the Gaussian noise. It means that the difference of CCDF of those is not considered at anyt i . Thus, the detection performance of the conventional K-S test to SαS is poor. Note that the K-S test has a higher miss detection probability under various false alarm probability because the K-S test does not enough capture the characteristics of impulsive noise compared with the proposed test. On the other hand, compared with the K-S test in the SαS noise model, the miss detection probability gain of the proposed test over the K-S test is significant even at low false alarm probability.
In summary, the conventional K-S test uses the maximum difference between CCDFs of received samples and the assumed distribution when the distribution of received samples follows the distribution of impulsive noise. However, the maximum difference may correspond to Gaussian component that is not the characteristic of impulsive noise when an impulsive property is stronger. Unlike the conventional K-S test, the proposed test considers all amplitudes of the received samples and computes differences of CCDFs between the assumed distribution and the received samples. Besides, the proposed test improves the classification performance by utilizing the characteristics of impulsive noise that it gives more weight to amplitudes corresponding to the characteristics of impulsive noise than weight for the other amplitudes corresponding to the Gaussian component. Through the simulation results, it was confirmed that the P MD performance of our proposed method was improved by at least 28% and 87% for Middleton class A and SαS models, respectively, compared to the existing conventional K-S test performance.

Conclusions
This paper proposes a test to detect the presence of impulsive noise. The proposed method is designed based on a weighted sum of all differences between CCDFs using thresholds. To improve detection performance, different weights are proposed according to the Middleton class A and SαS model, respectively. Finally, a threshold to satisfy target false alarm probability is obtained by analyzing an asymptotic false alarm probability. In particular, a significant highlight of our proposed test method is that it operates on a high detection probability with low false alarm probability; it can be used in various cellular networks, for example, a vehicular network where an impulsive noise exists.