# Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Typical NN-NUC Method

## 3. TV-Regularized NN-NUC Algorithm

#### 3.1. Scheme Description

#### 3.2. Gated Adaptive Learning Rate

## 4. Experiments and Results Analysis

#### 4.1. Simulation with Artificially Corrupted Data

#### 4.1.1. Comparison of Ghost Suppression Performance

#### 4.1.2. Comparison of Correction Precision and Convergence Rate

#### 4.2. Applications to Real Infrared Image Sequences

#### 4.3. Comparison of Real-Time Performance

## 5. Conclusions

## Author Contributions

## Acknowledgments

## Conflicts of Interest

## References

- Scribner, D.A.; Sarkady, K.A.; Kruer, M.R.; Caulfield, J.T.; Hunt, J.; Colbert, M.; Descour, M. Adaptive Retina-like Preprocessing for Imaging Detector Arrays. In Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA, 28 March–1 April 1993; Volume 3, pp. 1955–1960. [Google Scholar]
- Harris, J.G.; Chiang, Y.-M. Minimizing the Ghosting Artifact in Scene-Based Nonuniformity Correction. In Proceedings of the SPIE Conference on Infrared Imaging Systems: Design Analysis, Modeling, and Testing IX, Orlando, FL, USA, 15–16 April 1998; Volume 3377, pp. 106–113. [Google Scholar]
- Harris, J.; Chiang, Y. Nonuniformity correction of infrared image sequences using the constant-statistics constraint. IEEE Trans. Image Process.
**1999**, 8, 1148–1151. [Google Scholar] [CrossRef] [PubMed] - Qian, W.; Chen, Q.; Gu, G. Space low-pass and temporal high-pass nonuniformity correction algorithm. Opt. Rev.
**2010**, 17, 24–29. [Google Scholar] [CrossRef] - Bai, J.; Chen, Q.; Qian, W.; Wang, X. Ghosting reduction in scene-based nonuniformity correction of infrared image sequences. Chin. Opt. Lett.
**2010**, 8, 1113–1116. [Google Scholar] - Zuo, C.; Chen, Q.; Gu, G.; Qian, W. New temporal high-pass filter nonuniformity correction based on bilateral filter. Opt. Rev.
**2011**, 18, 197–202. [Google Scholar] [CrossRef] - Vera, E.; Torres, S. Fast adaptive nonuniformity correction for infrared focal plane array detectors. EURASIP J. Appl. Signal Process.
**2005**, 13, 1994–2004. [Google Scholar] [CrossRef] - Hardie, R.C.; Baxley, F.; Brys, B.; Hytla, P. Scene-Based Nonuniformity Correction with Reduced Ghosting Using a Gated LMS Algorithm. Opt. Express
**2009**, 17, 14918–14933. [Google Scholar] [CrossRef] [PubMed] - Rui, L.; Yin-Tang, Y.; Duan, Z.; Yue-Jin, L. Improved neural network based scene-adaptive nonuniformity correction method for infrared focal plane arrays. Appl. Opt.
**2008**, 47, 4331–4335. [Google Scholar] [CrossRef] - Zhang, T.; Shi, Y. Edge-directed adaptive nonuniformity correction for staring infrared focal plane arrays. Opt. Eng.
**2006**, 45, 016402-1-11. [Google Scholar] [CrossRef] - Rossi, A.; Diani, M. Bilateral filter-based adaptive nonuniformity correction for infrared focal-plane array systems. Opt. Eng.
**2010**, 49, 057003-1-13. [Google Scholar] [CrossRef] - Sheng-Hui, R.; Hui-Xin, Z.; Han-Lin, Q.; Rui, L.; Kun, Q. Guided filter and adaptive learning rate based non-uniformity correction algorithm for infrared focal plane array. Infrared Phys. Technol.
**2016**, 76, 691–697. [Google Scholar] [CrossRef] - Zhaolong, L.; Tongsheng, S.; Shuli, L. Scene-based nonuniformity correction based on bilateral filter with reduced ghosting. Infrared Phys. Technol.
**2016**, 77, 360–365. [Google Scholar] - Yu, H.; Zhang, Z.; Wang, C. An improved retina-like nonuniformity correction for infrared focal-plane array. Infrared Phys. Technol.
**2015**, 73, 62–72. [Google Scholar] [CrossRef] - Zhang, B.H.; Zhang, J.J.; Xu, H. A Nonuniformity Correction Enhancement Method Based on Temporal Statistical for Infrared System. In Proceedings of the IEEE International Symposium on Photonics and Optoelectronics, Shanghai, China, 21–23 May 2012; pp. 1–4. [Google Scholar]
- Boutemedjet, A.; Deng, C.; Zhao, B. Robust Approach for Nonuniformity Correction in Infrared Focal Plane Array. Sensors.
**2016**, 16, 1890. [Google Scholar] [CrossRef] [PubMed] - Tang, L.; Fang, Z. Edge and contrast preserving in total variation image denoising. EURASIP J. Adv. Signal Process.
**2016**, 13, 2–21. [Google Scholar] [CrossRef] - David, S.; Chan, T. Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl.
**2003**, 19, 165–187. [Google Scholar] - Kwong, R.; Johnston, E.W. A Variable Step Size LMS Algorithm. IEEE Trans. Signal Process.
**1992**, 40, 1633–1642. [Google Scholar] [CrossRef] - Vera, E.; Meza, P.; Torres, S. Total variation approach for adaptive nonuniformity correction in focal-plane arrays. Opt. Lett.
**2011**, 36, 172–174. [Google Scholar] [CrossRef] [PubMed] - Hayat, M.M.; Torres, S.N.; Armstrong, E.; Cain, S.C.; Yasuda, B. Statistical algorithm for nonuniformity correction in focal-plane arrays. Appl. Opt.
**1999**, 38, 772–780. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Scheme of the proposed TV-regularized neural-network nonuniformity correction (TVRNN-NUC) method.

**Figure 2.**Comparison of the mean square error (MSE) gain estimation obtained by operating various methods upon sequence 1. GTV: gated total variation; NN: neural network; TV: total variation.

**Figure 3.**Comparison of correction results for artificially corrupted data in sequence 1. (

**a**) Corrupted 1210th frame; (

**b**) Corrected with NN-NUC; (

**c**) Corrected with TV-NUC; (

**d**) Corrected with GTV-NUC ($K=10$); (

**e**) Corrected with TVRNN-NUC ($K=1$); (

**f**) Corrected with TVRNN-NUC ($K=10$).

**Figure 4.**NUC performance of various methods for artificially corrupted sequence 1. (

**a**) Peak signal-to-noise ratio (PSNR, dB) and (

**b**) Roughness index (ρ).

**Figure 5.**NUC performance of various methods for artificially-corrupted sequence 2. (

**a**) PSNR (dB); (

**b**) Roughness index (ρ).

**Figure 7.**Correction results of different NUC methods for the 1155th frame in sequence 3. (

**a**) Real scene with nonuniformity; (

**b**) Corrected with NN-NUC; (

**c**) Corrected with TV-NUC; (

**d**) Corrected with TVRNN-NUC. The red boxes mark out the partial obvious ghosting artifacts in the images.

**Figure 9.**Correction results of different NUC methods for the 148th frame in sequence 4. (

**a**) Real scene with nonuniformity; (

**b**) Corrected with NN-NUC; (

**c**) Corrected with TV-NUC; (

**d**) Corrected with TVRNN-NUC. The red box marks out the partial obvious ghosting artifacts.

**Table 1.**Mean PSNR (dB) and roughness index (ρ) for artificially-corrupted sequence 1. FPN: fixed pattern noise.

Performance Metrics | FPN-Corrupted Image | Corrected Images | ||||
---|---|---|---|---|---|---|

NN-NUC | TV-NUC | GTV-NUC | TVRNN-NUC (k = 1) | TVRNN-NUC (k = 10) | ||

PSNR (dB) | 20.29 | 28.10 | 28.31 | 27.73 | 29.63 | 28.27 |

ρ | 0.2760 | 0.1093 | 0.1083 | 0.1191 | 0.0999 | 0.1122 |

Performance Metrics | FPN-Corrupted Image | Corrected Images | ||||
---|---|---|---|---|---|---|

NN-NUC | TV-NUC | GTV-NUC | TVRNN-NUC (k = 1) | TVRNN-NUC (k = 5) | ||

PSNR (dB) | 22.17 | 32.55 | 33.34 | 29.84 | 36.65 | 33.34 |

ρ | 0.3708 | 0.0927 | 0.1041 | 0.1354 | 0.0719 | 0.1029 |

Index | NN-NUC | TV-NUC | GTV-NUC | TVRNN-NUC |
---|---|---|---|---|

Sequence 1 (471 × 358) | 272 | 210 | 46 | 43 |

Sequence 2 (532 × 478) | 199 | 115 | 31 | 28 |

Sequence 3 (384 × 288) | 580 | 442 | 86 | 81 |

Sequence 4 (320 × 256) | 628 | 549 | 112 | 99 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lai, R.; Yue, G.; Zhang, G.
Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images. *Symmetry* **2018**, *10*, 157.
https://doi.org/10.3390/sym10050157

**AMA Style**

Lai R, Yue G, Zhang G.
Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images. *Symmetry*. 2018; 10(5):157.
https://doi.org/10.3390/sym10050157

**Chicago/Turabian Style**

Lai, Rui, Gaoyu Yue, and Gangxuan Zhang.
2018. "Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images" *Symmetry* 10, no. 5: 157.
https://doi.org/10.3390/sym10050157