Evaluation of Fengyun-4A Detection Accuracy: A Case Study of the Land Surface Temperature Product for Hunan Province, Central China

Abstract

Land surface temperature (LST) is an important parameter in determining surface energy balance and a fundamental variable detected by the advanced geostationary radiation imager (AGRI), the main payload of FY-4A. FY-4A is the first of a new generation of Chinese geostationary satellites, and the detection product of the satellite has not been extensively validated. Therefore, it is important to conduct a comprehensive assessment of this product. In this study, the performance of the FY-4A LST product in the Hunan Province was authenticity tested with in situ measurements, triple collocation analyzed with reanalysis products, and impact analyzed with environmental factors. The results confirm that FY-4A captures LST well (R = 0.893, Rho = 0.915), but there is a general underestimation (Bias = −0.6295 °C) and relatively high random error (RMSE = 8.588 °C, ubRMSE = 5.842 °C). In terms of accuracy, FY-4A LST is more accurate for central-eastern, northern, and south-central Hunan Province and less accurate for western and southern mountainous areas and Dongting Lake. FY-4A LST is not as accurate as Himawari-8 LST; its accuracy also varies seasonally and between day and night. The accuracy of FY-4A LST decreases as elevation, in situ measured LST, surface heterogeneity, topographic relief, slope, or NDVI increase and as soil moisture decreases. FY-4A LST is also more accurate when the land cover is cultivated land or artificial surfaces or when the landform is a platform for other land covers and landforms. The conclusions drawn from the comprehensive analysis of the large quantity of data are generalizable and provide a quantitative baseline for assessing the detection capability of the FY-4A satellite, a reference for determining improvement in the retrieval algorithm, and a foundation for the development and application of future domestic satellite products.

1. Introduction

Land surface processes are important in Earth system research, and accurate modeling of the state of the land surface is essential for predicting the weather, modeling climate, and simulating the water cycle [1,2]. Land surface temperature (LST), which characterizes the state of surface heat balance, is an important physical parameter in modeling land surface processes [3,4], whether in climate, hydrology, ecology, biogeochemistry, or agricultural research [5,6,7].

LST is usually measured by in situ observation, predicted by models, or retrieved from satellite-based remote sensing data. In situ measurements are accurate, but recording stations are sparsely distributed, and their data do not meet the needs of regional-scale research or applications [8]. Models can process long LST time series with high spatial and temporal resolutions, but several input parameters are required, and the accuracy of the product is compromised by land surface inhomogeneity, land cover diversity, and difficulties in parameterizing land surface processes [2]. Satellite-based remote sensing is the principal method of detecting LST, and it provides real-time spatial and temporal distributions of surface temperature at regional and global scales [9]. Of the many techniques used for remote sensing, geostationary satellite-based remote sensing technology uses measurements in the thermal infrared (TIR) spectral bands to obtain LST at high temporal and spatial resolutions.

Remote sensing products require continuous comprehensive assessments to improve both detection technology and product performance [10]. Such activity promotes their use. Thus, new satellite detection technology and new products must be evaluated. The Fengyun-4 series (FY-4) represents the second generation of Chinese geostationary meteorological satellites. The prototype satellite FY-4A was launched on 11 December 2016. Its successful launch and the applications it carries make it a key component of the geosynchronous satellite constellation [11]. The advanced geostationary radiation imager (AGRI), the primary instrument of satellite payload, was designed principally to image land surface, atmosphere, and cloud targets with high spatial and temporal resolution [12]. FY-4A products, from the most advanced integrated atmospheric observation satellite of its time, have not been comprehensively evaluated.

Validating remote sensing products uses many methods of testing and assessment, such as ground-based validation [13], radiance-based validation [14], and satellite product comparison [15]. Temperature and radiance measured in situ provide the most accurate data for validation [16]. Testing product accuracy by directly comparing product data with in situ data is conceptually straightforward, but land surface heterogeneity and scale mismatch can lead to errors in this type of validation [17]. Direct comparison of satellite products may not generate noise-free results and is therefore inadequate for evaluating a remote sensing product in terms of other products [18]. Triple collocation analysis (TCA) is a method of calculating differences in random error in a single geophysical variable between three collocated datasets [19] that does not require the use of a high-quality reference dataset. TCA has become an important method of assessing Earth observation data [20], but it is not widely used to assess LST products. However, the recent development of innovative remote sensing and reanalysis products has resulted in the creation of many independent LST data products, which allows the use of TCA to test the accuracy of remote sensing LST products. All metrics calculated by TCA theoretically lie between the evaluated product and the unknown ground truth value [21], which enables a comparison of the accuracy and error characteristics of different products [22,23].

Various environmental factors distort the data recorded by satellite-based LST products [24,25]. It is necessary to identify how these factors affect the data to ensure the retrieval algorithm extracts accurate data by correcting for the mechanisms that reduce data accuracy. Studies have identified the various ways in which land cover [26,27], landform [28], human activity [29], meteorological conditions [30], and geographic factors [31] reduce the accuracy of satellite-based LST products, and researchers have created corresponding algorithms to increase the accuracy of detected LST data [31]. Identification and modeling of the environmental factors that affect the accuracy of FY-4A/LST data products at a regional scale and analysis of the influencing mechanisms are conducive to the improvement of the algorithms and thus increase product utility.

This study was intended to complete a comprehensive evaluation of the FY-4A/AGRI LST product for Hunan Province, China, and proceeded as follows. We conducted an accuracy test by direct comparison of FY-4A data with in situ measured data. We then conducted TCA with FY-4A data and two reanalysis products and determined the influence of several environmental factors. We used the Himawari-8 Advanced Himawari Imager LST product for comparison; this product is extensively used and has been well-validated in remote sensing LST assessments. The sensing instrumentation is functionally similar to FY-4A/AGRI, which makes the Himawari-8 product good for comparison [32,33] that will provide information used to improve and promote the use of FY-4A LST products. In situ measurement and grid products were included in our assessment, and a variety of methods were used in our comprehensive evaluation. The use of a large quantity of data suggests that the research conclusions are universally generalizable. The results of this study provide a baseline reference for assessing the detection capability of FY-4A and the utility of FY-4A LST products. This study also provides a basic analysis of how environmental factors, such as day and night, seasons, elevation, topography, land cover, landform, soil moisture, and vegetation, affect the accuracy of retrieved LST data. The work of this study supports the adoption and improvement of Fengyun 4 satellite LST products.

2. Materials and Methods

2.1. Study Area

Hunan Province is in south-central China, between 24°38′–30°08′ N and 108°47′–114°15′ E. The province covers an area of 211,800 km² and is in the transition zone from the Yunnan-Guizhou Plateau to the Jiangnan Hills and the Nanling Mountains to the Jianghan Plain. The province is highest in the south and lowest in the north and is surrounded by mountains on three sides (Figure 1). The central region alternates between hills and river valley basins, and the Dongting Lake Plain in the north is low and flat. The province has a high forest cover and a good natural environment. The climate of Hunan is continental subtropical monsoonal humid with abundant light, heat, and water resources but large intra-annual variations and significant vertical changes.

2.2. Data

The datasets used in this study are shown in

Table 1. Datasets used in this study.

Data Category	Data Name	Time Period	Temporal Resolution	Spatial Resolution
Remote sensing products	FY-4A/AGRI LST	1 October 2019 0 h–30 September 2019 23 h	1 h	4 km
	Himawari-8/AHI LST	1 October 2019 0 h–30 September 2019 23 h	1 h	0.045°
In situ measured data	In situ measured LST from CMA stations	1 October 2019 0 h–30 September 2019 23 h	1 h	/
Reanalysis products	CLDAS LST	1 October 2019 0 h–30 September 2019 23 h	1 h	0.0625°
	ERA5-Land LST	1 October 2019 0 h–30 September 2019 23 h	1 h	0.1°
Auxiliary data	NASA ASTER GDEM v3	2021	/	30 m
	China’s GlobalLand30 v2020	2020	/	30 m
	Geomorphic type data from IGSNRR, CAS	2009	/	1:1,000,000
	Soil moisture data retrieval from SMAP	1 October 2019– 30 September 2019	1 d	36 km
	NDVI data of NASA VNP13A2	22 September 2019– 30 March 2021	16 d	1 km

.

2.2.1. Remote Sensing Products

FY-4A/AGRI LST

FY-4A is the first satellite in the FY-4 series of independently developed Chinese second-generation geostationary meteorological satellites. FY-4A was the most advanced integrated atmospheric observation satellite of its time. It was launched in December 2016 and delivered in September 2017, positioned over the equator at 99.5° E (it later drifted to 104.7° E). The three-axis stabilized FY-4 series offers full disk coverage every 15 min or better (in contrast to the 30 min of FY-2), and there is an option for more frequent regional and mesoscale observation modes. The Advanced Geostationary Radiation Imager (AGRI) on FY-4A, with 14 channels, provides data that can be used to improve applications in a wide range of oceanic, land, and atmospheric monitoring activities and in forecasting extreme weather, especially typhoons and thunderstorms [12]. FY-4A LST data retrieval is based on the split-window algorithm [34] and uses two TIR channels (10.3–12.5 µm). The sensitivity of the product is 0.2 K, the spatial resolution is 4 km, and the temporal resolution is up to 15 min. The product was projected using the normalized projection (NOM) and was downloaded from http://satellite.nsmc.org.cn/ (accessed on 2 April 2021). Data were stored in netCDF format.

Himawari-8/AHI LST

Himawari-8 is a Japanese meteorological satellite that became operational in July 2015. It is positioned over the equator at 140° E. The primary instrument aboard Himawari-8 is the advanced Himawari imager (AHI), which is a 16-channel multispectral imager that captures visible light and infrared images of the Asia–Pacific region. AHI is similar to AGRI, so its LST product can be used for comparison with the FY-4A LST product. The Eastern Asia and Oceania LST of the Copernicus LST version 2 datasets derived from Himawari-8 data was retrieved using the generalized split-window algorithm (GSW) for two adjacent TIR channels (10.8–13.1 μm). The spatial resolution is 0.045°, the temporal resolution is 1 h, and it was projected using WGS 1984 and downloaded from http://land.copernicus.eu/ (accessed on 2 April 2021). Data were stored in netCDF format.

2.2.2. In Situ Measured LST Data

The in situ LST measurements were obtained from the National Meteorological Stations of the China Meteorological Administration (CMA). Most of the stations are located in flat terrain areas with uniform land cover and so can represent the local scale land surface state and thus are suitable for verification data (

Table A1. Overview of in situ measured stations.

Station Number	Station Name	Latitude (°)	Longitude (°)	Elevation (m)	Land Cover Type	Geomorphic Type
1	Longshan	29.46	109.44	488.7	Artificial surfaces	Plain
2	Sangzhi	29.4	110.16	318.8	Artificial surfaces	Hill
3	Zhangjiajie	29.12	110.42	218.5	Artificial surfaces	Platform
4	Shimen	29.58	111.36	116.9	Artificial surfaces	Platform
5	Cili	29.43	111.09	167.7	Forest	Hill
6	Lixian	29.67	111.73	38.1	Cultivated land	Platform
7	Linli	29.47	111.67	89.3	Forest	Platform
8	Nanxian	29.35	112.43	40.3	Cultivated land	Plain
9	Huarong	29.54	112.6	49.5	Artificial surfaces	Platform
10	Anxiang	29.41	112.2	33.6	Cultivated land	Plain
11	Yueyang	29.38	113.09	53	Artificial surfaces	Plain
12	Linxiang	29.48	113.45	60.4	Artificial surfaces	Plain
13	Huayuan	28.58	109.46	341	Artificial surfaces	Platform
14	Baojing	28.68	109.65	438.1	Cultivated land	Hill
15	Yongshun	29.01	109.84	268.2	Cultivated land	Platform
16	Guzhang	28.68	109.98	294	Forest	Platform
17	Jishou	28.24	109.68	254.6	Artificial surfaces	Plain
18	Yuanling	28.46	110.4	151.6	Artificial surfaces	Platform
19	Luxi	28.23	110.21	186	Forest	Platform
20	Chenxi	28.01	110.19	152.8	Artificial surfaces	Plain
21	Taoyuan	28.91	111.48	48.7	Artificial surfaces	Plain
22	Changde	29.12	111.68	150.6	Forest	Hill
23	Hanshou	28.92	111.96	31.9	Artificial surfaces	Plain
24	Taojiang	28.51	112.17	136.9	Forest	Platform
25	Anhua	28.38	111.25	196	Artificial surfaces	Platform
26	Yuanjiang	28.85	112.37	37	Water body	Plain
27	Xiangyin	28.73	112.93	63	Forest	Platform
28	Heshan	28.57	112.38	46.3	Artificial surfaces	Platform
29	Ningxiang	28.25	112.56	74.7	Artificial surfaces	Plain
30	Huanghua	28.21	113.2	101.4	Cultivated land	Platform
31	Miluo	28.86	113.11	82.5	Forest	Platform
32	Pingjiang	28.71	113.57	106.3	Artificial surfaces	Plain
33	Changsha	28.11	112.79	119	Forest	Platform
34	Liuyang	28.16	113.63	101.1	Forest	Plain
35	Fenghuang	27.95	109.6	349.6	Artificial surfaces	Platform
36	Mayang	27.87	109.8	176.6	Artificial surfaces	Plain
37	Xinhuang	27.37	109.16	355.5	Forest	Platform
38	Zhijiang	27.45	109.68	272.2	Artificial surfaces	Plain
39	Huaihua	27.61	110.03	286.9	Cultivated land	Plain
40	Xupu	27.92	110.6	204	Forest	Plain
41	Hongjiang	27.21	109.84	252	Artificial surfaces	Hill
42	Dongkou	27.03	110.61	339.5	Cultivated land	Platform
43	Lengshuijiang	27.7	111.44	249.2	Artificial surfaces	Plain
44	Xinhua	27.75	111.29	211.9	Artificial surfaces	Plain
45	Lianyuan	27.71	111.68	249.2	Artificial surfaces	Hill
46	Loudishi	27.69	112	205.8	Forest	Platform
47	Xuefengshan	27.32	110.41	1420	Forest	High relief mountain
48	Shaoyangshi	27.18	111.45	311	Cultivated land	Platform
49	Longhui	27.13	111.01	308.4	Cultivated land	Platform
50	Xinshao	27.34	111.45	294.1	Forest	Platform
51	Shaodong	27.24	111.74	252.6	Artificial surfaces	Plain
52	Shaoshan	27.93	112.53	88.3	Cultivated land	Platform
53	Xiangxiang	27.75	112.51	86.9	Forest	Plain
54	Xiangtan	27.88	112.83	63.8	Cultivated land	Platform
55	Shuangfeng	27.45	112.17	100	Artificial surfaces	Platform
56	Nanyue	27.3	112.69	1265.9	Forest	Medium relief mountain
57	Hengshan	27.26	112.84	159.1	Forest	Hill
58	Hengdong	27.05	112.98	109.4	Forest	Platform
59	Youxian	27.06	113.35	115.2	Forest	Platform
60	Zhuzhou	27.87	113.17	74.6	Artificial surfaces	Platform
61	Lilin	27.64	113.51	72.7	Artificial surfaces	Platform
62	Jingzhou	26.56	109.67	320.3	Artificial surfaces	Plain
63	Huitong	26.88	109.72	281.4	Forest	Platform
64	Tongdao	26.17	109.78	397.5	Artificial surfaces	Plain
65	Suining	26.59	110.15	310.3	Artificial surfaces	Platform
66	Xinning	26.46	110.83	346.1	Forest	Platform
67	Wugang	26.74	110.64	341	Artificial surfaces	Plain
68	Chengbu	26.37	110.31	477.7	Artificial surfaces	Plain
69	Shaoyangxian	27	111.29	283.3	Cultivated land	Hill
70	Lengshuitan	26.5	111.62	192.8	Cultivated land	Plain
71	Yongzhoushi	26.23	111.62	172.6	Artificial surfaces	Plain
72	Dongan	26.4	111.29	169	Artificial surfaces	Plain
73	Qiyang	26.59	111.86	113.2	Artificial surfaces	Plain
74	Qidong	26.76	112.08	218.9	Cultivated land	Platform
75	Hengyangxian	26.97	112.37	90.8	Artificial surfaces	Plain
76	Hengyang	26.89	112.6	104.9	Artificial surfaces	Platform
77	Changning	26.41	112.39	116.6	Artificial surfaces	Platform
78	Hengnan	26.76	112.69	137	Grassland	Platform
79	Leiyang	26.43	112.83	135	Cultivated land	Platform
80	Anren	26.71	113.26	101.8	Artificial surfaces	Plain
81	Chaling	26.79	113.55	136.2	Grassland	Plain
82	Yanling	26.48	113.79	268.8	Cultivated land	Plain
83	Yongxing	26.13	113.11	167.6	Artificial surfaces	Platform
84	Guidong	26.08	113.94	835.9	Artificial surfaces	Hill
85	Shuangpai	26.03	111.66	205	Artificial surfaces	Platform
86	Daoxian	25.53	111.6	192.2	Artificial surfaces	Plain
87	Ningyuan	25.59	111.96	244.2	Grassland	Plain
88	Jiangyong	25.28	111.31	269	Forest	Plain
89	Xintian	25.91	112.21	224.2	Artificial surfaces	Platform
90	Chenzhou	25.74	112.98	368.6	Forest	Hill
91	Guiyang	25.75	112.72	329.1	Artificial surfaces	Hill
92	Jiahe	25.58	112.37	214.5	Grassland	Hill
93	Lanshan	25.38	112.2	277	Artificial surfaces	Plain
94	Yizhang	25.41	112.94	222.8	Forest	Hill
95	Linwu	25.27	112.55	292	Artificial surfaces	Plain
96	Zixing	25.97	113.22	139.3	Artificial surfaces	Platform
97	Rucheng	25.51	113.68	645.6	Forest	Plain
98	Jianghua	25.18	111.57	265.7	Artificial surfaces	Hill
99	Pumanxiang	25.65	112.54	291	Cultivated land	Platform

). All of the in situ data were automatically observed by platinum resistance sensors and subjected to data quality control processes. The data range was (−80, 80) °C, and the accuracy was 0.1 °C. There are 99 stations in Hunan Province (Figure 1), and the data were obtained through the China Integrated Meteorological Information Sharing System (CIMISS) with a temporal resolution of hours.

2.2.3. Reanalysis Products

CLDAS

CMA Land Data Assimilation System (CLDAS) Version 2.0 [35] is a reanalysis grid product developed by CMA that covers Asia (0–65° N, 60–160° E) and consists of model, ground, and satellite data ensembles; it is of better quality than similar international products for China. We used the more accurate near real-time type LST product, which contains 0.0625° equal latitude and longitude grid data with 1 h temporal resolution. The data agreed well with ground observations: for China, the average correlation coefficient was 0.98, root mean square error (RMSE) was 1.8 K, and Bias was 1.4 K. The product was available from http://data.cma.cn/ (accessed on 13 March 2022). The data were saved in netCDF format.

ERA5-Land

The European Centre for Medium-Range Weather Forecasts (ECMWF) ERA5-Land is a fifth-generation reanalysis land surface dataset. ERA5-Land is produced by replaying the land component of the ECMWF ERA5 climate reanalysis; it combines model data with observations from across the world into a globally complete and consistent dataset. The product is available at 1 h temporal resolution and ~9 km spatial resolution from ERA5-Land servers. The ERA5-Land LST product is a state-of-the-art global reanalysis dataset for land applications. When compared with the MODIS LST, it showed a good correlation (correlation coefficient 0.94, Bias 1.36 K, RMSE 3.78) [36]. The product was available at https://cds.climate.copernicus.eu/ (accessed on 2 April 2021). The data were saved in netCDF format.

2.2.4. Auxiliary Data

The following data for environmental factors was used in the assessment.

1. Mean elevations within the FY-4A grids were calculated using a 30 m resolution elevation dataset from the NASA Advanced Spaceborne Thermal Emission and Reflection Radiometer Global Digital Elevation Model Version 3 (https://doi.org/10.5067/ASTER/AST14DEM.003; accessed on 20 January 2021). Slopes were calculated by the arcGIS planar method using this dataset.

2. Land cover type and information entropy within the FY-4A grids were derived from the 30 m resolution land cover dataset of the Ministry of Natural Resources of the People’s Republic of China, GlobalLand30 version 2020 (http://globallandcover.com; accessed on 2 April 2021). Information entropy is a measure of system uncertainty and indicates surface heterogeneity [37,38]. The probability distribution of a random variable X with n possible outcomes is P(X = x_i) = p_i for i = 1, 2,…, n. Information entropy H(X) is calculated as:

$$H(X)=-{\displaystyle \sum _{i=1}^n{p}_i}\log p_i$$

(1)

3. The landform type for FY-4A grids was derived from the 1:1,000,000 dataset of the spatial distribution of landforms in China from the Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences (https://www.resdc.cn/data.aspx?DATAID=124; accessed on 30 March 2022).

4. The mean value of soil moisture within the FY-4A grids was calculated using the Soil Moisture Active Passive (SMAP) L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture dataset (https://nsidc.org/; accessed on 15 April 2022) from 1 October 2019 to 30 September 2021.

5. The mean value of the normalized difference vegetation index (NDVI) in the FY-4A grids was calculated using an NDVI dataset with a 16 d acquisition period at 1 km resolution from 22 September 2019 to 30 September 2021. The data were obtained from the Suomi National Polar-Orbiting Partnership NASA Visible Infrared Imaging Radiometer Suite Vegetation Indices (VNP13A2) Version 1 (https://lpdaac.usgs.gov; accessed on 4 April 2022).

2.3. Methods

2.3.1. Research Methods

The LST measurement unit of each product in the study was standardized as °C, and the time was standardized as UTC. Since the data quality flags in FY-4A LST products are identified as invalid parameters, no data filtering was performed on the two remote sensing products. The following steps were taken to conduct this research (Figure 2).

• Acquire FY-4A/AGRI LST and Himawari-8/AHI LST products from 1 October 2019 0 h to 30 September 2021 23 h and decode and extract data for Hunan Province; carry out nearest-neighbor sampling to match 1 h remote sensing products with in situ measurements. The direct authenticity test and assessment based on measured data use this matched dataset.
• Obtain the CLDAS ground surface temperature data and the ERA5-Land skin temperature data of ERA5-Land for the same time period as step 1. Resample the CLDAS and ERA5-Land products to the FY-4A and Himawari-8 grids using three points inverse distance weighting, respectively, and match them hour by hour to form the FY-4A-CLDAS-ERA5-Land and Himawari-8-CLDAS-ERA5-Land datasets. The TCA-based LST accuracy assessment was carried out only for grids for which all three datasets were significantly positively correlated [20]; there were >100 data triplets [39].
• Identify the influencing mechanisms of environmental factors on FY-4A LST products by analyzing the effects of topography, land cover, landform, soil moisture, vegetation, and other environmental factors on TCA errors.

2.3.2. Performance Indicators

Direct Authenticity Test

The performance of remote sensing LST products in Hunan Province was assessed using the following error parameters: Pearson correlation coefficient (R), Bias, relative bias (Bias_r), root mean square error (RMSE), and unbiased RMSE (ubRMSE). The equations for these indexes are:

$$R=\frac{\mathrm{cov}(RST,IST)}{{\sigma }_{RST}{\sigma }_{IST}}$$

(2)

$$Bias=\frac{1}{m}{\displaystyle \sum _{i=1}^m(RS{T}_i-IS{T}_i)}$$

(3)

$$Bia{s}_{\mathrm{r}}=\frac{1}{m}{\displaystyle \sum _{i=1}^m\frac{\left|RS{T}_i-IS{T}_i\right|}{IS{T}_i}}$$

(4)

$$RMSE=\sqrt{\frac{1}{m}{\displaystyle \sum _{i=1}^m(RS{T}_i-IS{T}_i{)}^2}}$$

(5)

$$ubRMSE=\sqrt{RMS{E}^2-Bia{s}^2}$$

(6)

where RST is the LST of each remote sensing-based product, IST is the in situ measured LST, cov() is the covariance, and σ is the standard deviation.

Triple Collocation Analysis

The following parameters were used as indicators of TCA: Sensitivity, standard error (Stderr), correlation coefficient (Rho), fractional RMSE (fRMSE), and signal-to-noise ratio in decibels (Snr_db). The equations used are:

$${\mathit{Sensitivity}}_a=\frac{\mathrm{cov}(a,b)\times \mathrm{cov}(a,c)}{\mathrm{cov}(b,c)}$$

(7)

$${\mathit{Errvar}}_a=\mathrm{cov}(a,a)-Sensitivit{y}_a$$

(8)

$$\mathrm{If} {\mathit{Errvar}}_{\mathrm{a}} \ge 0, \mathrm{then} Stder{r}_a=\sqrt{{\mathit{Errvar}}_a}$$

(9)

$$\mathit{Rho}=\left[\begin{array}{c}\sqrt{\frac{\mathrm{cov}(a,b)\times \mathrm{cov}(a,c)}{\mathrm{cov}(a,a)\times \mathrm{cov}(b,c)}}\\ sign(\mathrm{cov}(a,c)\times \mathrm{cov}(b,c))\times \sqrt{\frac{\mathrm{cov}(a,b)\times \mathrm{cov}(b,c)}{\mathrm{cov}(b,b)\times \mathrm{cov}(a,c)}}\\ sign(\mathrm{cov}(a,b)\times \mathrm{cov}(b,c))\times \sqrt{\frac{\mathrm{cov}(a,c)\times \mathrm{cov}(b,c)}{\mathrm{cov}(c,c)\times \mathrm{cov}(a,b)}}\end{array}\right]$$

(10)

$$fRMSE=\sqrt{1-Rh{o}^2}$$

(11)

$$Sn{r}_a=\frac{\mathrm{cov}(a,a)\times \mathrm{cov}(b,c)}{\mathrm{cov}(a,b)\times \mathrm{cov}(a,c)}-1$$

(12)

$$\mathrm{If} {\mathit{Snr}}_{\mathrm{a}} \ge 0, \mathrm{then} Snr\_d{b}_a=-10\times \log (Sn{r}_a)$$

(13)

where a, b and c are the triple-collocated LST products in each grid.

Of the preceding parameters, greater values of R, Sensitivity and Rho, and lesser values of Bias, Bias_r, RMSE, ubRMSE and Stderr, indicate better product performance. The value of fRMSE has a well-defined range from 0 (perfect estimates) to 1 (total noise, with no ground truth signal), with values > ∼0.7 indicating an error variance that exceeds the variance of the true time series [40]. A value of zero for Snr_db indicates that signal variance is equal to noise variance; +3 (+6) dB indicates that signal variance is twice (four times) noise variance; −3 (−6) dB indicates that signal variance is half (one-fourth) noise variance, and so forth [20].

3. Results

3.1. Direct Authenticity Test and Evaluation of FY-4A LST Using In Situ Measurement

Comparative analysis of the FY-4A LST hourly data for Hunan Province matched with in situ measurements from 1 October 2019 0h to 30 September 2021 23 h shows (

Table 2. Error parameters for comparison between FY-4A LST and Himawari-8 LST tested against in situ measurements.

Product		R	Bias	Biasr	RMSE	ubRMSE	Data Quantity
FY-4A	overall	0.893	−6.295	0.303	8.588	5.842	5.394 × 105
	daytime	0.920	−9.244	0.336	11.078	6.105	2.181 × 105
	nighttime	0.864	−4.294	0.265	6.365	4.699	3.213 × 105
Himawari-8	overall	0.926	−4.933	0.238	7.508	5.660	5.191 × 105
	daytime	0.923	−7.413	0.247	9.606	6.110	2.368 × 105
	nighttime	0.849	−2.854	0.219	5.124	4.256	2.823 × 105

Daytime is the local meteorological daytime in Hunan Province, i.e., UTC [0, 12), and nighttime is UTC [12, 24).

) that the FY-4A product captured changes in the surface temperature well (R = 0.893), but that it generally underestimated LST (Bias = −6.295 °C) and had some deviation from in situ measurement (RMSE = 8.58 °C; ubRMSE = 5.842 °C). FY-4A was better at capturing daytime surface temperature changes than nighttime changes but also had greater Bias; ubRMSE was significantly lower than RMSE, indicating that FY-4A was more affected by random errors in observations. The Himawari-8 LST product was better than FY-4A in terms of all parameters except that there was overall less data due to the coarser resolution of the product; the deviation in LST was clearly less, as shown by Bias and RMSE.

When LST was low (<25 °C), the accuracy of FY-4A LST was better and stable. The center of density of its scatterplot was around the y = x line, but as the temperature increased, the deviation from observation gradually increased, and the underestimation of LST became greater (Figure 3), which may be one reason for its larger overall error (Table 2); there were also some outliers in the product. In contrast, the Himawari-8 LST product was overall more stable, with the center of density of the scatterplot close to the y = x line. This product also showed less deviation from direct observation than FY-4A at higher values of LST (>25 °C).

Table 3. Error indicators for comparison between FY-4A LST and in situ measurement of classifications of different factors.

Factors	Classification	R	Bias	Biasr	RMSE	ubRMSE	Data Quantity
Land surface temperature (°C)	−20–3	0.333	−0.276	0.937	3.179	3.167	0.316 × 105
	3–26	0.778	−4.791	0.316	6.779	4.795	3.301 × 105
	26–49	0.673	−9.031	0.274	10.141	4.613	1.617 × 105
	49–72	0.333	−21.512	0.401	21.868	3.929	0.16 × 105
Seasons	Spring	0.849	−6.446	0.306	8.630	5.737	0.972 × 105
	Summer	0.756	−8.137	0.261	10.160	6.084	1.277 × 105
	Autumn	0.854	−6.573	0.290	8.829	5.895	1.624 × 105
	Winter	0.819	−4.354	0.444	6.637	5.009	1.521 × 105
Elevation (m)	32–379	0.895	−6.378	0.303	8.652	5.845	5.011 × 105
	379–726	0.874	−6.348	0.344	8.182	5.163	0.233 × 105
	726–1073	0.848	−5.954	0.362	8.297	5.779	0.052 × 105
	1073–1420	0.847	−2.125	0.150	6.051	5.665	0.099 × 105

Data in red are the optimal parameters of each classification, and data in yellow are the worst parameters.

shows the values of indicators when FY-4A LST was compared with in situ measurement for equivalent classification of environmental factors. It can be seen that FY-4A LST was worst (R, Bias, RMSE were all worst) when LST was highest (49–72 °C). When LST was low (−20–3 °C), FY-4A performed better according to several indicators (Bias, RMSE, ubRMSE were all optimal), but its ability to capture changes in land surface temperature was not good (R) in this temperature range; the better indicator values may be due to the lower values of LST in this temperature range, as can be seen from the maximum value of Bias_r. The better performance of the FY-4A LST product in autumn and winter and the worse performance in summer may be related to mean LST (minimum Bias_r in summer). The ability of the FY-4A LST product to capture changes in LST decreased as elevation increased (R gradually decreased), but the deviation from in situ observation gradually decreased (Bias and RMSE both improve greatly). This may also be related to the decrease in LST as elevation increased.

3.2. FY-4A LST Authenticity Test Using TCA

Using TCA can overcome the problem created by the difficulty of obtaining the standard reference dataset of quantitative remote sensing parameters on the surface at a regional scale with high spatial and temporal resolution; the error indicators for comparisons between remote sensing products and the unknown ground truth are computed in the analysis [21]. The matched hourly data triplets formed in this study, the 2 y FY-4A-CLDAS-ERA5-Land dataset with a total of 65.3 million triplets of data that we created, and the sufficiency of the quantity of data ensure the generality of the study results.

The results of the analysis showed that the correlation coefficient (Rho) between FY-4A LST and true surface temperature was greater in central-eastern, northern, and south-central Hunan Province and that areas with very high correlation coefficients were found mostly in the Hengyang basin in central Hunan. Rho decreased in the western and southern mountainous areas and the northern Dongting Lake area; values for the southern border mountainous areas and the northwestern border mountainous areas were significantly lower (Figure 4). Rho for Himawari-8 LST was greater than for FY-4A overall; the spatial distribution trended similarly.

Analysis of the other error indicators (Figure 5) showed that the sensitivity of FY-4A LST to the true surface temperature was high relative to other remote sensing parameters [38]. The distribution of Sensitivity was similar to that of Rho, and it was greater in the central-eastern, south-central and northern parts of Hunan Province except for the Dongting Lake area and some areas in the west; Stderr, which indicates the degree of error, was between 3 and 4 for most grids and was greater in most areas in the west and the Dongting Lake area in the north and very high in the high elevation mountainous areas in the northwest.

The FY-4A LST product showed stable deviation and weak signal interference. The value of fRMSE, which indicates variation in error, was greater in the central-western, northwestern, and southern border areas, and the greatest values were found mostly in the northwestern, southern, and southeastern borders. Deviation was steady and always less than the standard deviation of the true time series (~0.7). Snr_db, which characterizes the signal-to-noise ratio, was greater in the central-southern, central, and northern areas except for Dongting Lake, where the signal showed weak interference; the signal showed some distortion in the western and southern regions.

The distribution of TCA error indicators shows that FY-4A LST data is strongly influenced by topography, and parameters that indicate its accuracy are almost consistently correlated with terrain in Hunan Province, whereas parameters indicating error are inversely related to terrain. FY-4A LST has low accuracy in the western and southern mountainous areas of Hunan Province, indicated by low correlation, large deviation and strong signal distortion, especially in the northwestern and southern border areas; in the south-central, central, and northern areas other than Dongting Lake, the product is more accurate, especially in the Hengyang basin, where correlation was greater, deviation was less, and the signal-to-noise ratio was greater.

3.3. Analysis of the Mechanism of the Influence of Environmental Factors on the Performance of FY-4A/LST

In this study, environmental factor parameters and remote sensing LST error parameters were combined for analysis of the mechanisms through which environmental factors influence FY-4A LST data. The results of the analysis confirmed the results in Section 3.1 and Section 3.2 that described the performance of FY-4A LST products as being strongly influenced by topographic factors. The accuracy of remote sensing LST products varied significantly for different mean DEM classifications (Figure 6). Most indicators showed that the accuracy of remote sensing LST products gradually decreased as elevation increased (Rho, Sensitivity, Snr_db gradually decreased; fRMSE gradually increased). Only the parameter Stderr, which exists as a filtering process in the algorithm (Equation (9)), showed a trend of initially increasing and then decreasing as elevation increased, which indicated that the error of FY-4A LST was larger at medium elevations (436–1269 m). Rho for Himawari-8 LST was greater than for FY-4A LST, but the change trend was consistent between the two. Thus, the accuracy of FY4A LST gradually decreased, and the distortion of the detection signal gradually increased as elevation increased.

The results of elevation analysis using TCA differed from the results based on in situ measurement (Table 3). In the latter, the maximum station elevation category was also worst in capturing changes in LST but had the least deviation (as indicated by Bias, Bias_r, and RMSE). However, we think the results of TCA in our study were more dependable for the following two reasons. 1. There were only two in situ measurement stations in the highest elevation category, so the amount of data provided was limited. 2. The least values of Bias and RMSE may be related to the lowest LST value range at high elevation; Stderr in TCA for areas above 436 m shows a similar trend to Bias and RMSE, and Bias_r shows a gradual increase as elevation increases in all categories except the highest elevation category (Table 3 and Figure 6).

The performance of FY-4A LST varies with land cover. FY-4A data were more accurate for Cultivated land and Artificial surfaces and less accurate for Forests and Water bodies. Variation was greater for Forests and Wetlands (Figure 7). The accuracy of FY-4A LST gradually decreased as land cover information entropy increased, indicating that as land cover became more inhomogeneous, FY-4A LST decreased in accuracy. The trend of variation in accuracy for Himawari-8 LST with land cover was generally similar to that of FY-4A LST, but Himawari-8 was significantly more accurate when the surface water content was greater (accuracy was relatively higher for Water body and there was less variation for Wetland).

Landform, slope, soil moisture, and NDVI all affected the accuracy of FY-4A LST (Figure 8). For landform, the accuracy of FY-4A LST was best for Platform, followed by Hill and Plain, and it decreased for Mountain as relief increased, which is consistent with the trend of accuracy decreasing as slope increased. For soil moisture, the accuracy of FY-4A LST increased as surface soil moisture increased. For NDVI, the trend of variation in FY-4A LST was more complex. When NDVI was low (<0.6), FY-4A LST was uniformly accurate; however, when NDVI was high (>0.6), the accuracy of FY-4A LST decreased sharply as NDVI increased. This behavior indicates that lush vegetation on the ground surface has a greater influence on FY-4A LST accuracy than other factors.

4. Discussion

Many factors affect the accuracy of LST data retrieved from satellite TIR data, such as atmospheric profiles, sensor parameters, surface parameters, and systematic errors due to the parameterization step of the retrieval algorithm [41]. It is therefore important to test the authenticity and comprehensively evaluate the LST product, especially for new satellite products.

There have been many validation studies of the LST products of Landsat, MODIS, Himawari and other satellites [16,42], and the effects of the retrieval algorithm, day/night observation, season, and land cover type on the accuracy of remote sensing LST products have been documented, quantified and modeled. In this study, we conducted a comprehensive assessment of the FY-4A LST product, which has not previously been extensively validated. We used high-quality in situ measured data from CMA together with reanalysis products, which gave us a large quantity of data to ensure our research was superior to other similar validation studies [13,16,41]. The direct accuracy comparison analyzed more than 500,000 data points, and TCA used >65 million data points for a 2 y period; the study results were therefore more generalizable and provide a baseline reference for improving and promoting the FY-4A product.

The authenticity test in the study adopted direct comparison between FY-4A LST with in situ measurement and TCA between FY-4A LST with two reanalysis products. The different methods of analysis allowed us to make more comprehensive and more generalizable conclusions. The scale difference between different data products in the direct comparison may have affected the results. The resolution of FY-4A/AGRI (4 km resolution for both TIR bands and LST product) differs from the resolution of Himawari-8/AHI (2 km resolution for TIR bands and 0.045° resolution for LST product), and the remote sensing grid product scale is different from the in situ measured single point data scale. However, the research results of the TCA method partially compensate for the impact of scale differences on the results. In contrast to the Bias values that were within ±2 K according to the Quality Assessment Report of Copernicus Global Land Operations, the Bias values we obtained were generally greater than the values for in situ measurement. This result may be due to the sparse distribution of observation stations (thus increasing the scale mismatch between remote sensing data and in situ measurements) and the fact that most meteorological observatories were situated on hills at elevations that are not conducive to accurate remote sensing detection (Figure 6). However, the different but still similar observational accuracy of FY-4A LST when compared with Himawari-8 data, and the greater accuracy of the LST product than other remote sensing parameters [38], illustrate the capability of the TIR-based split-window algorithm of FY-4A/AGRI to retrieve accurate LST.

The seasonal and diurnal trends of FY-4A LST (Bias, RMSE; Table 3) were consistent with the findings of Li et al. [43] for Himawari-8 and MODIS LST in western China; the LST products in that study also showed the largest variation when the land cover was forest. The error coefficients (Bias, Bias_r, and RMSE) calculated in the comprehensive assessment by Fan et al. [24] using FY-3C/MWRI LST and in situ measurement in Hunan Province were similar to those we obtained in this study. Fan et al. [24] also found a trend of decreasing accuracy with increasing elevation in remotely sensed LST together with a trend of gradually increasing underestimation when daytime LST was >20 °C (descending orbit). Martin et al. [16] validated remote sensing LST products using in situ radiation measurement and found that the accuracy of such products varied with season, day/night, and land cover, similar to our findings, and that observation variations in subtropical and forest sites were similar to those of FY-4A LST.

There were large differences in remote sensing LST accuracy for different surface temperature ranges in our results (Figure 3 and Table 3), and the ranges of surface temperature were affected by factors such as day/night, season [44], land cover, soil moisture, and vegetation condition [45]. Analysis of the effects of environmental factors confirmed that the preceding factors had a large effect on LST accuracy. The poor performance of LST products in Dongting Lake area may be related to the interference of remote sensing signals at the meeting of land surface and water surface. However, the lack of water surface observation stations in the in situ measurements limits our in-depth analysis of this point. Another factor that affects LST is shade: surfaces that are shaded are significantly cooler than sunlit surfaces. The effects of this factor are incorporated in the effects of elevation, landform, and slope on LST.

FY-4A LST was less accurate than Himawari-8 LST, which indicates that the FY-4A product is in need of improvement. However, as the prototype of a new generation of Chinese geostationary meteorological satellites, FY-4A is inherently experimental, and therefore systematic errors due to instrument performance are unavoidable. It has been found that some TIR bands of AGRI are not very stable in terms of satellite calibration [46]. It has also been found that although the retrieval algorithm used to produce FY-4A LST is similar to that used for Geostationary Operational Environmental Satellites (GOES)-R LST, observational variation in FY-4A LST is much greater [47]; this may also be related to the relatively poor performance of the sensor.

LST is a fundamental surface parameter that is detected by the primary sensors of China’s new generation of geostationary meteorological satellites. This comprehensive evaluation of the FY-4A LST product confirms the satellite has a good detection capability, but it does not yet match foreign satellite products in terms of overall accuracy. We can improve the retrieval algorithm by incorporating the influencing factors we identified or perhaps by using data-driven models based on artificial neural network methods to retrieve quantitative parameters from the remote sensing Chinese satellites, which is the goal of our next research project.

In conclusion, FY-4A, the first satellite of the FY-4 series, has successfully completed its task of comprehensively prototyping China’s second-generation geostationary orbit meteorological satellite program and observation technologies. Its observation performance has exceeded expectations, and the data products are being used in live meteorological applications. FY-4B has also been launched and will soon be in operation, and we look forward to the increasingly important role that Chinese meteorological satellites will play in the future.

5. Conclusions

In this study, we directly compared the FY-4A LST product with in situ measured LST data and used TCA with other reanalysis products to test the authenticity of the FY-4A LST product, which has not previously been extensively validated, for accuracy. We used Himawari-8 LST for comparison. We analyzed the effects of several environmental factors on LST accuracy. We drew the following conclusions.

• The FY-4A LST product captured the surface temperature well for Hunan Province (R = 0.893, Rho = 0.915), but it generally underestimated LST (Bias = −0.6295 °C) and there was a large random error (RMSE = 8.588 °C, ubRMSE = 5.842 °C); observation accuracy was worse than for Himawari-8 LST.
• The FY-4A LST product performed better in terms of accuracy for the central-eastern area, the northern area except Dongting Lake, and the central-southern parts of Hunan Province than for other parts of the province. The greatest accuracy was for the Hengyang basin in central Hunan. Accuracy decreased in the western and southern mountainous areas and the Dongting Lake area, and accuracy was the lowest in the mountainous areas along the southern and northwestern borders.
• When the surface temperature is high (>25 °C), remote sensing detection will significantly underestimate LST, and accuracy is greatly affected by topography and terrain; product accuracy decreases as elevation increases, and the change trend is basically consistent with change in elevation. FY-4A LST is most accurate when the land cover is Cultivated land or Artificial surfaces and the landform is Platform. Accuracy changes between day and night and seasonally, and decreases as land cover becomes more heterogeneous, mountain relief increases, or slope and NDVI increase; accuracy increases as soil moisture increases.

FY-4A is an experimental satellite and its detection performance exceeded expectations and provided a variety of valuable observation products for China’s meteorological operations. However, our evaluation found that the FY-4A LST product is not as accurate as advanced foreign satellite products, and so there is still a need to improve the performance of domestic satellite instruments and to improve the retrieval algorithm by combining environmental factors so as to improve China’s meteorological detection capability.