RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets
Abstract
:1. Introduction
- We present a novel radar data augmentation technique (RADIO) based on the measured properties of the radar signal. This models signal attenuation and resolution change over range, speckle noise and background shift for radar image generation.
- We demonstrate that such data augmentation can boost the accuracy and generalizability of deep models for object classification and detection, trained only with a small amount of source radar data. This verifies the effectiveness of the RADIO approach.
2. 300 GHz FMCW Radar
3. RADIO: Radar Data Augmentation
3.1. Attenuation
3.2. Change of Resolution
3.3. Speckle Noise and Background Shift
4. Neural Network Architecture
5. Experimental Results
5.1. Classification
5.2. Detection and Classification
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
RADIO | Radar data augmentation |
LIDAR | Light detection and ranging |
RADAR | Radio detection and ranging |
CFAR | Constant false alarm rate |
CA-CFAR | Cell averaging constant false alarm Rrate |
OS-CFAR | Order statistics constant false alarm rate |
DBSCAN | Density-based spatial clustering of applications with noise |
SDA | Standard data augmentation |
RDA | Range data augmentation |
BS | Background Shift |
DCNN | Deep convolutional neural networks |
MIMO | Multiple input multiple output |
SAR | Synthetic aperture radar |
FMCW | Frequency modulated continuous wave |
dB | Decibel |
FFT | Fast Fourier transform |
ReLU | Rectified linear unity |
SGD | Stochastic gradient descent |
RCS | Radar cross section |
AP | Average precision |
mAP | Mean average precision |
References
- Premebida, C.; Carreira, J.; Batista, J.; Nunes, U. Pedestrian Detection Combining RGB and Dense Lidar Data. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014; pp. 4112–4117. [Google Scholar]
- Du, X.; Ang, M.H.; Karaman, S.; Rus, D. A general Pipeline for 3D Detection of Vehicles. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018; pp. 3194–3200. [Google Scholar]
- Yang, F.; Choi, W.; Lin, Y. Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2129–2137. [Google Scholar]
- Liang, M.; Yang, B.; Chen, Y.; Hu, R.; Urtasun, R. Multi-Task Multi-Sensor Fusion for 3D Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 7345–7353. [Google Scholar]
- Daniel, L.; Phippen, D.; Hoare, E.; Stove, A.; Cherniakov, M.; Gashinova, M. Low-THz Radar, Lidar and Optical Imaging Through Artificially Generated Fog. In Proceedings of the International Conference on Radar Systems (Radar 2017), Belfast, UK, 23–26 October 2017; pp. 1–4. [Google Scholar]
- Texas Instruments. Short Range Radar Reference Design Using AWR1642; Technical Report; Texas Instruments: Dallas, TX, USA, 2017. [Google Scholar]
- Marchetti, E.; Du, R.; Willetts, B.; Norouzian, F.; Hoare, E.G.; Tran, T.Y.; Clarke, N.; Cherniakov, M.; Gashinova, M. Radar cross-section of pedestrians in the low-THz band. IET Radar Sonar Navig. 2018, 12, 1104–1113. [Google Scholar] [CrossRef]
- Sheeny, M.; Wallace, A.; Wang, S. 300 GHz Radar Object Recognition based on Deep Neural Networks and Transfer Learning. IET Radar Sonar Navig. 2020. [Google Scholar] [CrossRef]
- Rohling, H.; Heuel, S.; Ritter, H. Pedestrian detection procedure integrated into an 24 GHz automotive radar. In Proceedings of the 2010 IEEE Radar Conference, Washington, DC, USA, 10–14 May 2010; pp. 1229–1232. [Google Scholar]
- Bartsch, A.; Fitzek, F.; Rasshofer, R. Pedestrian Recognition Using Automotive Radar Sensors. Adv. Radio Sci. 2012, 10. [Google Scholar] [CrossRef]
- Angelov, A.; Robertson, A.; Murray-Smith, R.; Fioranelli, F. Practical Classification of Different Moving Targets Using Automotive Radar and Deep Neural Networks. IET Radar Sonar Navig. 2019, 12, 1082–1089. [Google Scholar] [CrossRef] [Green Version]
- Lombacher, J.; Laudt, K.; Hahn, M.; Dickmann, J.; Wöhler, C. Semantic Radar Grids. In Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA, 11–14 June 2017; pp. 1170–1175. [Google Scholar]
- Phippen, D.; Daniel, L.; Gashinova, M.; Stove, A. Trilateralisation of Targets Using a 300 GHz Radar System. In Proceedings of the International Conference on Radar Systems, Belfast, UK, 23–26 October 2017. [Google Scholar]
- Marchetti, E.; Daniel, L.; Hoare, E.; Norouzian, F.; Cherniakov, M.; Gashinova, M. Radar Reflectivity of a Passenger Car at 300 GHz. In Proceedings of the 19th IEEE International Radar Symposium (IRS), Bonn, Germany, 20–22 June 2018; pp. 1–7. [Google Scholar] [CrossRef]
- Radar Tutorial. 2019. Available online: https://www.radartutorial.eu/index.en.html (accessed on 14 October 2019).
- Knott, E.F.; Schaeffer, J.F.; Tulley, M.T. Radar Cross Section; SciTech Publishing: Raleigh, NC, USA, 2004. [Google Scholar]
- Schuler, K.; Becker, D.; Wiesback, W. Extraction of Virtual Scattering Centers of Vehicles by Ray-tracing Simulations. IEEE Trans. Antennas Propag. 2008, 56, 3543–3551. [Google Scholar] [CrossRef] [Green Version]
- Ding, J.; Chen, B.; Liu, H.; Huang, M. Convolutional Neural Network with Data Augmentation for SAR Target Recognition. IEEE Geosci. Remote Sens. Lett. 2016, 13, 364–368. [Google Scholar] [CrossRef]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; The MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Chen, S.; Wang, H.; Xu, F.; Jin, Y.Q. Target Classification Using the Deep Convolutional Networks for SAR Images. IEEE Trans. Geosci. Remote Sens. 2016, 54, 4806–4817. [Google Scholar] [CrossRef]
- Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A. Automatic Differentiation in PyTorch. In Proceedings of the NIPS Autodiff Workshop, Long Beach, CA, USA, 9 December 2017. [Google Scholar]
- Jung, A.B.; Wada, K.; Crall, J.; Tanaka, S.; Graving, J.; Yadav, S.; Banerjee, J.; Vecsei, G.; Kraft, A.; Borovec, J.; et al. Imgaug. 2019. Available online: https://github.com/aleju/imgaug (accessed on 25 September 2019).
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Richards, M. Fundamentals of Radar Signal Processing; McGraw-Hill Education (India) Pvt Limited: Noida, India, 2005. [Google Scholar]
- Ester, M.; Kriegel, H.P.; Sander, J.; Xu, X. A Density-based Algorithm for Discovering Clusters a Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, Portland, OR, USA, 2–4 August 1996; pp. 226–231. [Google Scholar]
- Everingham, M.; Eslami, S.A.; Van Gool, L.; Williams, C.K.; Winn, J.; Zisserman, A. The pascal visual object classes challenge: A retrospective. Int. J. Comput. Vis. 2015, 111, 98–136. [Google Scholar] [CrossRef]
Train 3.8 m | Test 6.3 m | |
---|---|---|
Bike | 90 | 90 |
Trolley | 90 | 90 |
Mannequin | 90 | 90 |
Cone | 25 | 25 |
Traffic Sign | 90 | 90 |
Stuffed Dog | 90 | 90 |
Total | 475 | 475 |
Trolley | Bike | Cone | Mannequin | Sign | Dog | |
---|---|---|---|---|---|---|
MSAD (3.8 m) | 2.593 | 1.444 | 0.073 | 0.551 | 0.426 | 0.964 |
MSAD (6.3 m) | 2.374 | 1.298 | 0.056 | 0.506 | 0.359 | 0.881 |
Input | Conv 16 () [ReLU] | MaxPool | Conv 32 () [ReLU] | MaxPool | Conv 64 () [ReLU] | MaxPool | Dropout 0.5 | Conv 128 () [ReLU] | Conv 6 () [Softmax] | Label Distribution |
Learning rate () | 0.001 |
Momentum () | 0.9 |
Epochs | 100 |
Batch size | 100 |
Samples | Precision (%) | Recall (%) | F1-Score (%) | Accuracy (%) | |
---|---|---|---|---|---|
Original Data | 475 | 33.36 | 39.15 | 31.39 | 39.15 |
Original Data + SDA | 19,950 | 80.71 | 82.13 | 79.49 | 82.13 |
Original Data + SDA + SN + BS (Ours) | 19,950 | 88.51 | 83.83 | 82.42 | 83.83 |
Original Data + SDA + RDA + BS (Ours) | 19,950 | 95.28 | 94.89 | 94.45 | 94.89 |
Original Data + SDA + RDA + SN + BS (Ours) | 19,950 | 99.79 | 99.79 | 99.79 | 99.79 |
AP | Overall | Short | Mid | Long | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Overall | Short | Mid | Long | Overall | Short | Mid | Long | Overall | Short | Mid | Long | |||||
bike | 65.78 | 81.7 | 50.0 | 88.89 | 75.0 | 53.82 | 0.0 | 59.92 | 66.67 | 66.36 | N/A | 67.2 | N/A | 25.0 | 69.31 | 71.43 |
bike (ours) | 71.94 | 90.98 | 50.0 | 94.44 | 100.0 | 64.55 | 100.0 | 59.99 | 100.0 | 53.43 | N/A | 48.92 | N/A | 75.0 | 67.17 | 100.0 |
cone | 42.29 | 51.25 | 50.0 | 61.11 | 50.0 | 66.44 | 65.08 | 83.33 | 28.57 | 34.35 | 62.5 | 42.5 | 23.19 | 60.44 | 52.71 | 24.22 |
cone (ours) | 49.96 | 66.67 | 50.0 | 66.67 | 100.0 | 69.23 | 66.67 | 66.67 | 83.33 | 35.07 | 62.5 | 13.33 | 0.0 | 62.07 | 37.04 | 45.31 |
dog | 26.72 | 45.07 | 55.56 | 47.32 | 30.95 | 25.22 | 33.33 | 36.67 | 16.67 | 13.68 | 50.0 | N/A | 7.5 | 48.0 | 38.33 | 9.44 |
dog (ours) | 67.26 | 86.43 | 88.89 | 99.05 | 11.11 | 64.74 | 83.33 | 60.42 | 50.0 | 30.91 | 40.0 | N/A | 0.0 | 68.0 | 85.02 | 6.67 |
mannequin | 42.05 | 71.28 | 53.33 | 87.02 | 55.56 | 28.3 | 42.42 | 34.18 | 7.5 | 44.47 | 14.29 | 58.82 | 19.25 | 34.72 | 54.16 | 14.94 |
mannequin (ours) | 41.15 | 82.61 | 83.33 | 92.86 | 33.33 | 28.56 | 45.45 | 26.67 | 14.55 | 36.28 | 14.29 | 52.94 | 10.92 | 45.83 | 49.94 | 10.62 |
sign | 49.36 | 41.3 | 0.0 | 44.71 | 40.0 | 59.77 | N/A | 62.5 | 57.89 | 40.35 | N/A | 41.94 | 38.46 | 0.0 | 50.57 | 49.28 |
sign (ours) | 77.26 | 87.01 | 100.0 | 90.09 | 66.67 | 74.49 | N/A | 80.19 | 70.27 | 76.92 | N/A | 77.78 | 76.19 | 100.0 | 82.52 | 72.13 |
trolley | 84.61 | 90.23 | 84.72 | 99.92 | 76.47 | 87.35 | 100.0 | 91.66 | 68.81 | 78.51 | 95.99 | 85.78 | 26.67 | 96.29 | 91.69 | 61.94 |
trolley (ours) | 90.08 | 97.08 | 97.62 | 99.44 | 94.12 | 94.16 | 100.0 | 96.42 | 81.36 | 83.82 | 100.0 | 87.79 | 44.37 | 99.01 | 92.91 | 75.11 |
ine mAP | 51.80 | 63.47 | 48.94 | 71.49 | 54.66 | 53.48 | 48.17 | 61.38 | 41.02 | 46.29 | 55.69 | 59.25 | 23.01 | 44.07 | 59.46 | 38.54 |
mAP (ours) | 66.28 | 85.13 | 78.31 | 90.43 | 67.54 | 65.96 | 79.09 | 65.06 | 66.58 | 52.74 | 54.2 | 56.15 | 26.3 | 74.99 | 69.1 | 51.64 |
AP | Overall | Short | Mid | Long | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Overall | Short | Mid | Long | Overall | Short | Mid | Long | Overall | Short | Mid | Long | |||||
bike | 54.97 | 81.96 | 50.0 | 71.83 | 100.0 | 57.49 | 50.0 | 59.69 | 33.33 | 29.82 | N/A | 32.47 | N/A | 50.0 | 52.07 | 71.43 |
bike (ours) | 70.60 | 88.55 | 70.0 | 74.16 | 100.0 | 63.75 | 100.0 | 57.3 | 66.67 | 61.13 | N/A | 58.92 | N/A | 78.85 | 61.66 | 83.67 |
cone | 12.95 | 45.45 | 62.5 | 25.0 | 0.0 | 41.72 | 55.56 | 66.67 | 0.0 | 1.07 | 0.0 | 1.39 | 0.0 | 25.17 | 13.06 | 0.0 |
cone (ours) | 18.41 | 27.27 | 25.0 | 16.67 | 0.0 | 45.1 | 55.56 | 50.0 | 0.0 | 2.94 | 0.0 | 6.67 | 0.0 | 18.72 | 17.9 | 0.0 |
dog | 18.76 | 36.83 | 66.67 | 43.17 | 4.17 | 17.86 | 33.33 | 21.43 | 0.0 | 8.68 | 40.0 | N/A | 1.56 | 48.0 | 27.29 | 1.28 |
dog (ours) | 58.74 | 77.88 | 77.78 | 98.08 | 5.56 | 50.51 | 66.67 | 44.44 | 0.0 | 33.75 | 48.33 | N/A | 0.0 | 63.53 | 74.81 | 1.54 |
mannequin | 41.14 | 72.6 | 64.0 | 85.87 | 44.44 | 21.64 | 29.09 | 28.39 | 9.41 | 48.54 | 0.0 | 61.76 | 34.81 | 27.83 | 52.93 | 18.67 |
mannequin (ours) | 39.49 | 87.18 | 100.0 | 92.86 | 44.44 | 26.0 | 36.36 | 23.33 | 19.23 | 33.65 | 0.0 | 50.0 | 13.33 | 39.13 | 47.27 | 15.3 |
sign | 46.26 | 43.43 | 0.0 | 44.71 | 50.0 | 50.87 | N/A | 54.11 | 45.6 | 43.32 | N/A | 37.72 | 50.0 | 0.0 | 44.78 | 47.81 |
sign (ours) | 72.67 | 88.22 | 100.0 | 91.93 | 66.67 | 71.94 | N/A | 78.05 | 65.25 | 69.51 | N/A | 65.24 | 78.36 | 50.0 | 76.82 | 68.43 |
trolley | 80.03 | 83.94 | 68.33 | 94.36 | 73.12 | 84.18 | 97.44 | 88.26 | 64.48 | 74.47 | 79.56 | 75.54 | 34.0 | 82.82 | 83.87 | 59.2 |
trolley (ours) | 82.75 | 81.06 | 84.72 | 88.37 | 64.46 | 88.66 | 95.6 | 92.89 | 68.35 | 82.1 | 87.41 | 80.15 | 45.24 | 88.19 | 84.48 | 58.0 |
ine mAP | 42.35 | 60.7 | 51.92 | 60.82 | 45.29 | 45.63 | 53.08 | 53.09 | 25.47 | 34.32 | 29.89 | 41.78 | 24.08 | 38.97 | 45.67 | 33.06 |
mAP (ours) | 57.11 | 75.02 | 76.25 | 77.01 | 46.85 | 57.66 | 59.03 | 57.66 | 36.58 | 47.18 | 22.62 | 43.49 | 22.82 | 56.40 | 60.49 | 37.82 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sheeny, M.; Wallace, A.; Wang, S. RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets. Appl. Sci. 2020, 10, 3861. https://doi.org/10.3390/app10113861
Sheeny M, Wallace A, Wang S. RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets. Applied Sciences. 2020; 10(11):3861. https://doi.org/10.3390/app10113861
Chicago/Turabian StyleSheeny, Marcel, Andrew Wallace, and Sen Wang. 2020. "RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets" Applied Sciences 10, no. 11: 3861. https://doi.org/10.3390/app10113861
APA StyleSheeny, M., Wallace, A., & Wang, S. (2020). RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets. Applied Sciences, 10(11), 3861. https://doi.org/10.3390/app10113861