Assessment of Techniques for Detection of Transient Radio-Frequency Interference (RFI) Signals: A Case Study of a Transient in Radar Test Data

Abstract

The authors present a case study of the investigation of a transient signal that appeared in the testing of a radar receiver. The characteristics of the test conditions and data are first discussed. The authors then proceed to outline the methods for detecting and analyzing transients in the data. For this, they consider several methods based on modern signal processing and evaluate their utility. The initial method used for identifying transients is based on computer vision techniques, specifically, thresholding spectrograms into binary images, morphological processing, and object boundary extraction. The authors also consider deep learning methods and methods related to optimal statistical detection. For the latter approach, since the transient in this case was chirp-like, the method of maximum likelihood is used to estimate its parameters. Each approach is evaluated, followed by a discussion of how the results could be extended to analysis and detection of other types of transient radio-frequency interference (RFI). The authors find that computer vision, deep learning, and statistical detection methods are all useful. However, each is best used at different stages of the investigation when a transient appears in data. Computer vision is particularly useful when little is known about the transient, while traditional statistically optimal detection can be quite accurate once the structure of the transient is known and its parameters estimated.

1. Introduction

This work describes an investigation of transient interference observed in radar testing. In this introductory section, we provide a general description of the investigation. We then follow with a summary of the different state-of-the-art for approaches and applications of transient signal identification.

1.1. Investigation Overview

When operating radar or radio receivers, it is not unusual to see interfering signals in addition to expected signals and ubiquitous thermal noise. For such interfering signals, typical questions center around the rate of occurrence, signal characteristics, the effects on the desired operation of the system, and whether interference can be reliably detected and possibly removed. When faced with the problem of detecting and characterizing unwanted transients in data, the analyst finds that a number of methods are available; however, picking which one or ones to use can be challenging. The next subsection provides a high-level summary and review of some of the available methods. To review and compare methods, we present a case study of a particular radio-frequency interference (RFI) signal occasionally seen in the noise-only testing of a radar receiver. This receiver is part of a spaceborne radar in development, with a launch planned in the near future. After surveying previous work on transient detection, we provide details about the radar receiver, test conditions, and initial observations of the transient in Section 2. Additionally, we provide the transient’s characteristics, estimated manually using a standard spectrogram. In Section 3, several techniques for automatically detecting the presence of the transients in test datasets are discussed in some detail. Section 4 describes the results of applying these techniques to our data, with conclusions provided in Section 5.

1.2. Overview of State-of-the-Art

Applications of transient signal detection include radar, microwave radiometry, radio astronomy, underwater acoustics, and RF fingerprinting [1,2,3,4,5,6]. Methods have included traditional signal processing and, more recently, the application of machine learning techniques [6]. We summarize a number of methods here, grouping them by method rather than by application. The methods with perhaps the longest history have their foundations in statistical decision theory. A standard reference within the electrical engineering community is that of Van Trees [7]. This volume presents the classical detection theory of communications or radar signals in noise, including matched filtering for the detection of known signals. The paper by Friedlander and Porat [8] applies statistical analyses to linear transformations of signals, including common time-frequency representations. Such representations have also been used in some of the more recent machine learning procedures described below. A relatively recent review of statistical detection methods applied to transient problems is found in [9]. Following this review, the reference describes a new method based on sequential probability ratio tests. Another relatively recent work on statistical detection is described in [10]. The application used here is radar, and the approach is based on a generalized likelihood ratio test (GLRT) for detecting a Gaussian signal in Gaussian noise. As described in Section 3, GLRT denotes a test in which unknown parameters of the signal are estimated prior to applying the likelihood ratio test. A related work [11] addresses this problem using statistical tests other than GLRT. The overview of RF fingerprinting in [5] primarily covers statistical techniques that are often used in this application; however, the problems there tend to be more related to a change in the received signal than a finite signal in noise. This distinction is further described in [9]. As such, the methods in [5] are related to but are not necessarily directly applicable to RFI transient detection.

Many of the recent methods for transient detection use machine-learning approaches and are applied to areas like drone signature recognition and radio astronomy. The latter also uses statistical detection theory but is often generalized for typical radio astronomy measurements, such as using sensor arrays. The technique described in [6] examines signals emitted by drones. Such signals can be used to identify intruding or otherwise unwanted drones. In [6], received data are first transformed, e.g., wavelet, and then used as an input to a hierarchical classifier. In this application, as also noted for the RF fingerprinting applications in [5], the transient is usually a change in signal rather than one of finite duration (temporary) interference. Nevertheless, the problem here is related to RFI detection, especially when viewed using machine learning. This is also true of [12], which is again applied to drones and RF fingerprinting; it uses a neural network that is applied directly to the received complex data. The problem described in [13] is similar to that being investigated here, namely RFI in radar data. The approach there uses time-frequency images of the data as input to a neural network classifier, which can detect the RFI. An alternative machine learning approach is a support vector machine, which is applied in [14] to recognize transients from features also extracted from a time-frequency representation. Reference [15] describes the application of two types of neural networks for RFI detection. There, RFI is not only detected but classified as to its likely source. Both [14] and [15] describe astronomical applications, as does [16], which uses a neural network to directly classify time-frequency images.

This brief overview of methods, especially those discussed in recent publications, provides a background for the specific techniques that we use and compare. These are described in Section 3, following the discussion of the radar data and transient overview in the next section.

2. Data

The data used here were recorded by the radar. The radar receiver has a bandwidth of 80 MHz and operates with a carrier near 1.257 GHz (L-band). The received L-band data are filtered and sampled at 240 MHz and then are digitally down-converted to a 96 MHz complex sample rate. The Nyquist sampling criterion is satisfied because the complex sampling results in a single-sideband signal that fits unambiguously inside the 0–96 MHz spectrum. The data recorded within each interpulse period are referred to as a range line; each range line has 105,152 complex samples at a 96 MHz sample rate (roughly 1 ms in length). Each dataset used here contains up to about one hundred thousand range lines, corresponding to nearly a minute of the radar receiver operation, based on 0.5 ms between range lines. All eight datasets were acquired indoors in the noise-only mode with no transmitted waveform. The overall noise level depended primarily on the environment (noisy laboratory versus test chamber); the inherent gain and noise level of the receivers were stable.

The left panel in Figure 1 shows an example of a typical, noise-only range line. The very small amplitudes near 300 and 850 μs are due to receiver blanking during radar pulsing, although pulsing was not happening in this case. The statistics of these data showed approximate Gaussian behavior of the in-phase and quadrature (I/Q, or real and imaginary) parts of the data. This indicated relatively pure thermal noise, as would be expected for a high-quality receiver with no signal inputs. Additionally, as shown in Figure 1 (right panel) a spectrogram for this noise range line is presented. Although some harmonics or spurious tones (vertical lines) are present, these are low and not are expected to affect the radar measurements. Figure 2, left, shows a different range line that appears similar to that in Figure 1 but also contains an unexpected transient signal starting around time 900 μs. However, this level iss nearly the same as the noise; therefore, the transient is almost invisible in the time domain. The corresponding spectrogram for this range line, on the right, shows the transient much more clearly, highlighted by an ellipse. The transient appears to have a roughly linear frequency increase with time, making it approximately a linear, frequency-modulated (LFM) chirp, and increasing in frequency from 83 MHz to about 85 MHz as time increases from roughly 900 to 1000 μs.

3. Data Analysis Methodology

3.1. Computer Vision

A priority when investigating a newly noticed transient is the determination of how often it occurs in the data (i.e., rate of occurrence), followed by the determination of the characteristics of the transients. In light of a lack of visibility in the transient in the time domain (Figure 2, left), spectrograms were used (see right panels in Figure 1 and Figure 2). These were computed using the Short-Time Fourier Transform, which computed the Discrete Fourier Transform of time segments in the data. As a first step, a simple video, or movie, of all the spectrograms in a dataset allowed the manual detection of some of the transients. For automated detection, several approaches were possible, as summarized in Section 1.2. One approach used early in our investigation included computer (or machine) vision [17,18], which could be applied to spectrograms. This method was considered because it can find arbitrary objects in images without being sensitive to the exact time or frequency within the spectrogram. Since computer vision algorithms extract features from images and use these features for decisions, we consider this a relative of the machine learning techniques mentioned in Section 1.2. Although the feature extraction approaches used in [6,14] might not be classical computer vision, the overall approach of feature extraction and decision-making is analogous to that considered here. While we are not aware of a specific application of computer vision to RFI transient detection, we note in [19], for example, the application of computer vision to the detection of anomalous signals or events for network security.

Because the noise is visible even in normal spectrograms (Figure 1, right), an estimated thermal noise level is used to set a threshold. All pixels below the threshold are set to zero, while all above are set to one, yielding a binary image. In some cases, there are linear features above the threshold, as shown in the binary images. These are found by summing the binary image both horizontally and vertically; the presence of a line results in a larger-than-average sum for a problematic row or column, allowing it to be removed. Following this, we use morphological image processing functions [17,20] to remove isolated pixels above the threshold. This processing results in images that are relatively clean, except for the transients, when present. Figure 3 shows the application of these steps to the spectrogram in Figure 2 on the right. Once we have a cleaned binary spectrogram, we apply a computer vision routine (called bwboundaries), which locates objects in a binary scene [20]. It returns information on the size, shape, and location of each object found. From the object dimensions and the spectrogram pixel sizes, we perform a somewhat crude estimate of the starting time and frequency, the bandwidth, and the duration of each transient. For the example transient in Figure 2 and Figure 3, the function returned a set of boundary indices that allowed the code to calculate a duration of 88 μs and a bandwidth of 1.87 MHz. The code saves this information, along with the object’s peak power, range line number, and range sample number for each detection. This allows an examination of the statistics of the transient characteristics over the dataset, which is discussed in Section 4. In the remainder of this paper, we denote this approach as a computer vision algorithm or CVA.

3.2. Convolutional Neural Network

Another method considered was the convolutional neural network (CNN) [21]. This approach is already noted in Section 1.2 as being used for RFI detection in radar data [13]. To avoid conducting pre-processing, we let the CNN operate directly on the spectrograms [13,16]. This was reasonable since CNNs excel at extracting features from images and using them for classification. While one could train deep CNNs from scratch, such training typically requires hundreds of thousands of images, which were not available in this case. Instead, we used the approach of transfer learning. As described in [16,21,22,23], the general idea of transfer training is to start with a pre-trained deep network and adapt it for a new task by replacing the last hidden layer and the output layer and re-training on a much smaller, task-specific dataset. The studies described in [16,22] demonstrate RFI detection using CNNs and transfer learning, with the input data in [22] based on continuous wavelet transform, which is analogous to the spectrograms used here. For the problem of transient RFI detection in our radar noise data, we also started with a pre-trained CNN, in this case, SqueezeNet [21,23]. SqueezeNet is a CNN with 18 convolutional layers that are designed to classify color images of size 227 × 227 × 3 into 1000 different categories. It was trained on over 1 million input images; image categories included common scenes and objects, such as various animals, pencils, keyboards, and coffee mugs. While these categories are very different from spectrograms with RFI, the method can work because much of a deep network, starting with the input layers, is good at extracting features. The retraining of output layers changed the way the network used the extracted features for the new problem. For the Matlab implementation of SqueezeNet, we replaced the last convolutional layer with a new convolutional layer, setting the number of filters in the layer to match the number of classes, which was two in this case, corresponding to either RFI or noise. The output classification layer was also replaced. Prior to training, the learning rate for the new convolutional layer was set to a much larger number than the rate for the original SqueezeNet layers. When training was started, these different rates caused only the new layer to be trained. This transfer learning technique allowed the deep CNN to be re-trained in only a few minutes on a typical laptop.

3.3. Statistical Decision Theory

Lastly, we considered methods that are traditionally considered statistical detection methods within the signal-processing community. The approach here is similar to the statistical approaches surveyed in Section 1.2. The basic strategy is to make the choice that optimized some error criterion. As described in [7,24,25,26], the general strategy computes a likelihood ratio and compares this with a threshold. Using the notation of [26], the likelihood ratio is the ratio of the conditional probability of the data with the signal present $f_1(\tilde{x})$ compared to the conditional probability of the data when the signal is not present $f_0(\tilde{x})$, where $\tilde{x}$ is the observed data, in our case, a noisy range line that either has or does not have a transient signal in it. In the simplest model, the signal to be detected is fully deterministic. The optimal detection method is the matched filter (MF), in which a deterministic signal is cross-correlated with the data [7,24,25,26]. The optimal detection approach for the case of a set of unknown parameters $\mathsf{\psi }$ is also discussed in [7,24,25,26]. This problem becomes a computation of a so-called generalized likelihood ratio, which is the ratio of the maximum of $f_1(\tilde{x};\mathsf{\psi })$ over the possible $\mathsf{\psi }$ when the chirp is present to the likelihood function $f_0\left(\tilde{x}\right)$ with no chirp. The GLRT is discussed in Section 1.2 in relation to [10] and was needed in our case since we did not have a priori knowledge of the chirp starting frequency and rate. We implemented this case by correlating with a chirp whose parameters were estimated with the maximum likelihood estimation (MLE). In [27], it is shown that the MLE of chirp parameters could be found by maximizing the peak of the cross-correlation of a model chirp with the data. The search was carried out over the set $\mathsf{\psi }$, here consisting of the starting frequency $f_0$ and the chirp rate α; these parameters were chosen to maximize:

$$X(f,\alpha )=|\sum x_{i }{e}^{j2\pi \left(f_0{t}_i+\frac{\alpha }{2}{t}_i^2\right)}|$$

(1)

where j is the square root of −1, $t_i=(i-1)\mathsf{\Delta }T$, $\mathsf{\Delta }T$ is the sampling interval, and the complex samples $x_i$ represent the observed data. The exponential was complex conjugated to correspond to the usual definition of cross-correlation. While matched filtering was also used in the power domain (e.g., in image processing), the use of complex data was preferred so that the output SNR could be boosted by the effect of coherent integration. Hence, the above approach was preferred on theoretical grounds to the matched filtering of the spectrogram by a time-frequency representation of the chirp power. Details of MLE for chirp parameters are provided in Appendix A.

4. Results

4.1. Generation of Training and Validation Data

Following the initial notice of our transient in a few spectrograms, our investigation turned to the methods described above in Section 3. To provide quantitative results for these methods, we created subsets for training and validation by arbitrarily selecting range lines within the various full datasets noted in Section 2. The subsets needed to be large enough that sufficient data could be used but small enough that the classifications could be verified by a human viewing of their contents. Our procedure used the results of the CVA to create 3000 spectrograms, half with transients and half without. All 3000 spectrograms were manually inspected to check for correct identification (requiring several seconds per image). Incorrect classifications were noted for CVA accuracy estimates and then replaced with new spectrograms; hence, the final set of 3000 contained the desired half-and-half mix. This set was used for the training and validation of the CNN, as described in Section 4.3 below. In this case, 1020 of the 3000 spectrograms were used for independent validation. In the generation of the spectrograms, the transients were shifted to other, randomly selected frequencies to keep the CNN from memorizing cases with one frequency. The validation set had 510 spectrograms with transients and 510 without, which were verified manually. The CVA output was also used to identify a set of complex data (range lines) for the testing of the matched filtering method (described in Section 4.4). As with the CNN training data, the range lines were manually inspected to verify the assessment and the presence/absence of transients. Hence, the complex validation set had 510 range lines with transients and 510 without, matching the CNN validation set.

4.2. CVA Results

As noted in Section 3.1, for automated detection and characterization, CVA is a good first choice because it can detect fairly arbitrary objects, requiring minimal assumptions about the data. Additionally, it provides not only detections but the basic characteristics of the objects it finds. Figure 4 shows example histograms from a selected dataset with transients. There is no averaging in the values shown; each value of a characteristic corresponds to one transient. Duration (time from transient start to stop) tends to be near 100 μs, while bandwidths are mostly less than a few MHz. The transients seemed to occur at several different frequencies, while the starting time within a range line (denoted as fast time) could be anywhere from 0 to nearly 1 ms. Figure 4c shows the occurrence of transients detected using CVA in slow time (proportional to the pulse number); they tended to occur in clusters separated by roughly 12 s. As all timing on the radar is much faster than 12 s, this result suggests a source outside the radar. Zooming into this plot, which is not shown, the fast time looks mostly random, but we could see evidence of systematic changes in fast time location between the adjacent pulses. This varying and generally random fast-time behavior (Figure 4d) suggested that the transients were not synchronized with radar pulsing and were likely external. The hypothesis that the transients are external is further supported by their non-occurrence in subsequent testing in a different environment.

To quantitatively evaluate the accuracy of the CVA, we refer back to Section 4.1 and the generation of the spectrograms. For the 3000 spectrograms initially selected by CVA, a manual inspection found 10 false alarms, corresponding to about 4 false alarms in the test, or validation, data set. This result is reflected in the value of 1016 out of 1020 spectrograms that were correctly identified by CVA (99.6%), as shown in

Table 1. Comparison of RFI detection methods.

Method	Est. Run Time (s)	Validation Data	Correct
Computer Vision Algorithm (CVA)	40	1020 spectrograms	1016
Convolutional Neural Net (CNN)	45	1020 spectrograms	1007
Matched filter (MF)	25	1020 range lines	990

. These results for CVA support our claim that it can successfully detect transient events and can provide useful characteristics; however, it relies, to some extent, on knowing how to set the threshold to separate transients from noise. This and some additional parameters of the algorithm, including the size of the erosion and dilatation filters, are best set by experimenting with CVA performance.

4.3. CNN Results

The next method evaluated was the deep learning of spectrograms with a CNN. Since CNNs operate directly on the spectrogram of each range line, there were no explicit thresholds to be set. However, unlike CVA, this method requires accurately trained data and so is typically not a good first choice for the analysis of a newly discovered transient signal. Indeed, as noted in Section 4.1, both the CNN and the statistical detection algorithms rely on the prior use of CVA for the preliminary characterization of the transients and to detect a sufficient number of them so that training data that can be developed. Figure 5 shows the results of re-training with the SqueezeNet network, which was conducted with the transfer learning approach. The final accuracy after 1320 training iterations was 98.7% on the independent validation data, where accuracy is the percentage of correct classifications for both signals present and absent. Hence, CNN detects about the same number of transients as that detected by CVA (Table 1). CNN is also relatively robust since the validation data consist of transients at differing chirp rates and time-frequency locations. Indeed, it is highly likely that the validation data contained transients with frequencies not represented in the training data, indicating generalization by the CNN.

4.4. MF Results

This section reports on the statistical detection of transients, implemented here as matched filtering. We note that this matched filter, or MF, processing is identical to that in pulse compression radars, in which the transmitted waveform is correlated with the received signal. MFs using observed data and an ideal chirp are shown in Figure 6. The observed chirp was extracted from noisy range line data (e.g., Figure 2, left) using a filtering technique that we developed, retaining the chirp while removing most of the noise. The observed chirp has significant amplitude modulation, unlike the ideal chirp. Furthermore, the amplitude modulation could vary between observations. We tested both filters by applying them to the 1020 lines of complex validation data. To use the observed chirp, its frequency was shifted to match transients in the validation data set. We found that with a well-selected threshold, the detection rate was 97%, with either the ideal or observed chirps used as an MF (Table 1). However, estimating chirp parameters with MLE and then generating a corresponding ideal chirp was somewhat simpler than trying to extract and modify an observed pulse from another dataset. Figure 7 illustrates the performance of a matched filter on example range lines.

An inspection of the incorrectly classified range lines verified that all had transients that were missed. To determine the reason for each miss, we looked at the power spectrum of the corresponding range line. We found that all apparent misses did, in fact, contain a transient but with different start frequencies or chirp rates than those assumed in the matched filter. While it would be feasible to apply a combination of MLE and MF on each range line, the time required to search the entire range line would be significant, likely resulting in slower run times than CVA. If the frequencies and rates of all the transients were somehow known, a simpler algorithm could apply a set of matched filters to the data, with the set being based on all the known transient signals. The test statistic, in this case, would be the maximum of all the outputs on a given range line; based on the above tests, such an algorithm should detect all the transients. The missed detections here highlight MF’s lack of ability to generalize.

5. Conclusions

We have reported on a case study of a specific type of transient observed in noise-only radar testing. The purpose of this work was to compare several techniques that could be used in such a situation. While our particular transient was approximately a linear frequency-modulated chirp, we believe that the approach here could be generalized to many kinds of radio-frequency interference occurring in test data. Once the transients were noted in our data, we began an investigation with computer vision techniques to detect unknown “objects” within spectrograms or their equivalent. While we did need to estimate the level of background noise in the spectrograms for thresholding, the algorithms were able to clean up the thresholded spectrograms and found objects with relatively little adjustments and a priori knowledge. An exception is morphological processing, which did require some changes based on the experiment to better keep the objects of interest while removing noise. We found the computer vision approach to be extremely useful, providing most of the information needed for characterizing the transients in our data, as well as allowing the creation of training and validation data for other methods. Because it does not depend on the transient being a linear frequency-modulated chirp, we believe that computer vision can be a useful first step for investigating a variety of transients.

The convolutional neural network, trained via transfer learning, proved to be accurate and had the advantage of working directly from spectrograms without the need for thresholds or other adjustable parameters. However, without first using the computer vision algorithms, identifying sufficient training data for the neural network was very difficult. Furthermore, the training data had to be rather carefully generated, making sure to cover the basic ways in which the transient could occur in a spectrogram. As with the computer vision approach, the neural network can be applied to many types of transients. The last method evaluated was statistical detection using matched filtering. The matched filter applied to the complex range lines was quite sensitive (Figure 7). However, this approach required a reference function that matched the transient interference signal of interest. Consequently, one needed to estimate the transient parameters and then generate the matched filter. This process can be applied to many types of transient signals. Once the transient has been characterized, more general models, such as in [28,29], could be used, with the maximum likelihood estimation of model parameters. However, the MF approach is best used only if it is already known that the transients are all identical or have only a few sets of parameters. The experience here confirms that all the methods can be very accurate but differ significantly in their need for a priori information and their ability to generalize. While computer vision recognizes fairly arbitrary objects, CNNs are likely more specific but can generalize. Matched filters only use their given parameters and do not generalize well to transients with significantly different parameters. To summarize, the following approach for investigating arbitrary transients is suggested: (1) A computer vision for obtaining general characteristics of objects in spectrograms or other time-frequency images. Once training data can be identified, a neural network approach (2) using transfer learning could potentially achieve high accuracy with no assumptions beyond the training data. Finally, classical matched filtering methods (3) could potentially provide a very high detection accuracy; however, the transient properties must be well-known to allow for the construction of an accurate matched filter or filters.