# An Improved DINEOF Algorithm Based on Optimized Validation Points Selection Method

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The Improved Validation-Point DINEOF Algorithm

## 3. Data Source and Dataset Construction

#### 3.1. Ideal Dataset (Dataset1)

#### 3.2. Constructed Dataset Based on Satellite Remote-Sensing Data (Dataset2)

## 4. Performance of IV−DINEOF Algorithm

#### 4.1. Verification Metrics

#### 4.2. Reconstruction Results of Dataset1

#### 4.3. Comparison of the Reconstruction Performance for Different Rates of Missing Data

#### 4.4. Reconstruction Results of Dataset2

## 5. Discussion

#### 5.1. Optimal EOFs and SSIM

#### 5.2. r and SNR

#### 5.3. RMSE and MAD

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Jiang, X.W.; Lin, M.S.; Zhang, Y.G.; Ma, Y. Development history and trend prospect of ocean remote sensing satellite and its application. Satell. Appl.
**2018**, 5, 10–18. [Google Scholar] - Zhao, H.; Qi, Y.Q.; Wang, D.X.; Wang, W. Study on the features of chlorophyll-a derived from SeaWiFS in the South China Sea. Acta Oceanol. Sin.
**2005**, 27, 45–52. [Google Scholar] - Ping, B.; Su, F.Z.; Meng, Y.S. Reconstruction of Satellite-Derived Sea Surface Temperature Data Based on an Improved DINEOF Algorithm. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2015**, 8, 4181–4188. [Google Scholar] [CrossRef] - He, R.Y.; Robert, H.; Weisberg, Z.H.Y.; Muller-Karger, P.E.; Helber, R.W. A cloud-free, satellite-derived, sea surface temperature analysis for the West Florida Shelf. Geophys. Res. Lett.
**2013**, 30, 15. [Google Scholar] [CrossRef][Green Version] - Sun, W.F.; Wang, J.; Zhang, J.; Ma, Y.; Meng, J.; Yang, L.; Miao, J. A new global gridded sea surface temperature product constructed from infrared and microwave radiometer data using the optimum interpolation method. Acta Oceanol. Sin.
**2018**, 37, 41–49. [Google Scholar] [CrossRef] - Beckers, J.M.; Rixen, M. EOF calculations and data filling from incomplete oceanographic datasets. J. Atmos. Ocean. Technol.
**2003**, 20, 1839–1856. [Google Scholar] [CrossRef] - Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum likelihood from incomplete data via the EM algorithm. Proc. R. Stat. Soc.
**1977**, 39, 1–22. [Google Scholar] - Schoellhamer, D.H. Singular spectrum analysis for time series with missing data. Geophys. Res. Lett.
**2001**, 28, 3187–3190. [Google Scholar] [CrossRef][Green Version] - Shen, Y.; Peng, F.; Li, B. Improved singular spectrum analysis for time series with missing data. Nonlin. Proc. Geophys. Discuss.
**2015**, 1, 371–376. [Google Scholar] [CrossRef][Green Version] - Jiang, X.B.; Hu, Y.M.; Liu, Z.H.; Tan, Z.X.; Liao, Q. Reconstructing NDVI Time-series Data Using a Linear-Interpolation with Extended Kalman Filter. Bull. Sci. Technol.
**2017**, 33, 137–142. [Google Scholar] - Gong, J.D.; Qiu, C.J.; Wang, Q.; Chen, W.M. The numerical experiment inarea four dimensional variational data assimilation. Acta Meteorol. Sin.
**1999**, 57, 4–15. [Google Scholar] - Moctar, D.; Oriani, F.; Tumbulto, J.; Mariethoz, G.; Schaefli, B. Gap-filling of daily streamflow time series using Direct Sampling in various hydroclimatic settings. J. Hydrol.
**2018**, 569, 573–586. [Google Scholar] - Fu, M.L.; Fan, T.C.; Ding, Z.A.; Salih, S.Q.; Al-Ansari, N.; Mundher Yaseen, Z. Deep Learning Data-Intelligence Model Based on Adjusted Forecasting Widow Scale: Application in Daily Streamflow Simulation. IEEE Access
**2020**, 8, 32632–32651. [Google Scholar] [CrossRef] - Oriani, F.; Stisen, S.; Demirel, M.C.; Mariethoz, G. Missing Missing Data Imputation for Multisite Rainfall Networks: A Comparison between Geostatistical Interpolation and Pattern-Based Estimation on Different Terrain Types. J. Hydrometeorol.
**2020**, 10, 2325–2341. [Google Scholar] [CrossRef] - Guo, H.X. Data Reconstruction of Satellite Remote Sensing Chlorophyll-a in Offshore China and the related Characteristic of Spatial-Temporal Variations. Master’s Thesis, State Oceanic Administration, Fujian, China, 2016. [Google Scholar]
- Mosavi, A.; Ozturk, P.; Chau, K. Flood Prediction Using Machine Learning Models: Literature Review. Water
**2018**, 10, 1536. [Google Scholar] [CrossRef][Green Version] - Ping, B.; Su, F.Z.; Meng, Y.S. An Improved DINEOF Algorithm for Filling Missing Values in Spatio-Temporal Sea Surface Temperature Data. PLoS ONE
**2016**, 11, e0155928. [Google Scholar] [CrossRef] [PubMed] - Zhang, Z.H.; Zhang, C.; Meng, L.; Tang, K.; Zhu, H. Application of improved DINEOF algorithm in the reconstruction of missing remote sensing data of Chlorophyll-a in the Bohai Sea, China. J. Geo-Inf. Sci.
**2021**, 23, 737–748. [Google Scholar] - Beckers, J.M.; Barth, A.; Alvera-Azcárate, A. DINEOF reconstruction of clouded images including error maps application to the Sea-Surface Temperature around Corsican Island. Ocean Sci.
**2006**, 2, 183–199. [Google Scholar] [CrossRef][Green Version] - Alvera-Azcárate, A.; Barth, A.; Rixen, M.; Beckers, J.M. Reconstruction of incomplete oceanographic datasets using empirical orthogonal functions: Application to the Adriatic Sea surface temperature. Ocean. Model.
**2005**, 9, 325–346. [Google Scholar] [CrossRef]

**Figure 1.**Comparison of cross-validation points among different algorithms (Gray area represents land, white area represents missing data points, and green area represents valid data points).

**Figure 3.**Schematic diagram of the construction process of Dataset2: (

**a**) Original SST Data, (

**b**) Original chl-a Data, and (

**c**) Constructed SST Missing Data. (The gray area represents land, the white area represents missing data points, and the green area represents valid data points).

**Figure 6.**Comparison of results of every single month between DINEOF and IV−DINEOF: (

**a**) correlation coefficient, (

**b**) signal-to-noise ratio, (

**c**) root mean square error, and (

**d**) mean absolute difference.

**Figure 7.**(

**a**) Original SST missing data in Feb. 2003, (

**b**) Constructed SST missing data according to chl-a, (

**c**) Reconstructed SST data using DINEOF, and (

**d**) Reconstructed SST data using IV−DINEOF.

**Figure 8.**(

**a**) Original SST missing data in Jul. 2010, (

**b**) Constructed SST missing data according to chl-a, (

**c**) Reconstructed SST data using DINEOF, and (

**d**) Reconstructed SST data using IV−DINEOF.

**Figure 9.**(

**a**) Original SST missing data in Jun. 2013, (

**b**) Constructed SST missing data according to chl-a, (

**c**) Reconstructed SST data using DINEOF, and (

**d**) Reconstructed SST data using IV−DINEOF.

**Figure 10.**The differences of SSIM, r, and MAD (

**a**,

**d**,

**g**missing rate: 20%;

**b**,

**e**,

**h**: missing rate: 40%; and

**c**,

**f**,

**i**missing rate: 60%).

**Figure 11.**The differences of RMSE and MAD (

**a**,

**d**missing rate: 20%;

**b**,

**e**missing rate: 40%; and

**c**,

**f**missing rate: 60%).

**Table 1.**Comparison of mean values of verification metrics among DINEOF, IV−DINEOF, and VE−DINEOF for Dataset1.

Verification Metrics | r | SNR | RMSE | MAD | SSIM | Average Number of Iterations |
---|---|---|---|---|---|---|

DINEOF | 0.8919 | 2.7401 | 0.7873 | 0.3831 | 0.3289 | 7071 |

IV−DINEOF | 0.9225 | 3.4502 | 0.6609 | 0.3125 | 0.3334 | 6776 |

VE−DINEOF | 0.8713 | 1.7867 | 0.8821 | 0.4322 | 0.3237 | 2563 |

**Table 2.**Comparison of verification metrics among DINEOF, IV−DINEOF, and VE−DINEOF for different overall missing rates in Dataset1.

Overall Missing Rate | Algorithm | r | SNR | RMSE | MAD | SSIM |
---|---|---|---|---|---|---|

20% | DINEOF | 0.9818 | 25.9378 | 0.2678 | 0.0635 | 0.3335 |

IV−DINEOF | 0.9916 | 35.8065 | 0.1553 | 0.0348 | 0.3337 | |

VE−DINEOF | 0.9531 | 3.2707 | 0.5336 | 0.1691 | 0.3358 | |

40% | DINEOF | 0.9358 | 4.4649 | 0.5850 | 0.2387 | 0.3331 |

IV−DINEOF | 0.9518 | 4.9534 | 0.5048 | 0.1995 | 0.3350 | |

VE−DINEOF | 0.8900 | 1.9148 | 0.8271 | 0.3874 | 0.3300 | |

60% | DINEOF | 0.8992 | 2.2693 | 0.7654 | 0.3706 | 0.3288 |

IV−DINEOF | 0.9015 | 2.2557 | 0.7641 | 0.3667 | 0.3292 | |

VE−DINEOF | 0.8530 | 1.5783 | 0.9398 | 0.5044 | 0.3207 |

Verification Metrics | r | SNR | RMSE | MAD | Number of Iterations |
---|---|---|---|---|---|

DINEOF | 0.8833 | 1.8598 | 4.4152 | 2.4008 | 2.1919 |

IV−DINEOF | 0.9378 | 2.6805 | 3.2955 | 1.6278 | 2.1919 |

**Table 4.**Comparison of verification metrics of reconstruction results between DINEOF and IV−DINEOF with different single-month missing rates using Dataset2.

Month | Missing Rate | Verification Metrics | r | SNR | RMSE | MAD |
---|---|---|---|---|---|---|

Feb. 2003 | 19.56% | DINEOF | 0.1922 | 1.0189 | 5.9978 | 4.1773 |

IV−DINEOF | 0.5644 | 1.1512 | 3.5289 | 2.4369 | ||

Jul. 2010 | 41.11% | DINEOF | −0.1527 | 0.8846 | 8.5714 | 5.2358 |

IV−DINEOF | 0.9075 | 2.1632 | 1.1000 | 0.7468 | ||

Jun. 2013 | 57.19% | DINEOF | 0.3934 | 0.8863 | 2.4923 | 1.4579 |

IV−DINEOF | 0.7350 | 1.3311 | 1.6238 | 1.0983 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yang, Z.; Xia, X.; Teo, F.-Y.; Lim, S.-P.; Yuan, D.
An Improved DINEOF Algorithm Based on Optimized Validation Points Selection Method. *Water* **2023**, *15*, 392.
https://doi.org/10.3390/w15030392

**AMA Style**

Yang Z, Xia X, Teo F-Y, Lim S-P, Yuan D.
An Improved DINEOF Algorithm Based on Optimized Validation Points Selection Method. *Water*. 2023; 15(3):392.
https://doi.org/10.3390/w15030392

**Chicago/Turabian Style**

Yang, Zhenteng, Xinchen Xia, Fang-Yenn Teo, Sin-Poh Lim, and Dekui Yuan.
2023. "An Improved DINEOF Algorithm Based on Optimized Validation Points Selection Method" *Water* 15, no. 3: 392.
https://doi.org/10.3390/w15030392