With the rapid progress of remote sensing techniques, the need for remotely sensed images with high temporal, spatial, and spectral resolution has increased [1
]. In particular, remotely sensed images with high spatial resolution and frequent coverage are needed in the monitoring of global biophysical processes, which change quickly during growing season. However, satellite data with high temporal and spatial resolutions are lacking due to the cost, long revisit cycles [4
], frequent cloud contamination [5
], capacity of satellite platforms [6
], and technological difficulties in developing satellites. This situation presents significant disadvantages and challenges in observing and monitoring ground status in a timely and effective manner [7
]. Moreover, detecting the condition of rice growth in the southern area of China is difficult because of missing data caused by frequent cloudiness and precipitation. Meanwhile, the growth period of rice is short, and rice changes rapidly in its growing period. Hence, the shortage of remote sensing data on the main growth periods of rice hinders the development of research on rice, such as research involving rice condition monitoring and crop yield assessment.
The Thematic Mapper (TM), Enhanced Thematic Mapper Plus (ETM+), and Operational Land Imager (OLI) sensors aboard the Landsat satellites and the moderate-resolution imaging spectroradiometer (MODIS) sensor aboard the Terra and Aqua satellites provide two types of widely used satellite data [8
]. Landsat is suitable for long-term surface change monitoring at the regional scale due to its spatial resolution of 30 m. The most commonly used satellite sensor for mapping biophysical vegetation parameters and land cover types is Landsat [9
]. However, the 16-day revisit cycle of Landsat and the 35% average cloud cover of images [10
] have long limited the use of Landsat in studying global biophysical processes that evolve rapidly during the growing season [11
]. By contrast, MODIS has a high temporal resolution and covers the Earth multiple times a day. However, it has spatial resolutions of 250, 500, and 1000 m [12
], which limit the sensors’ capability to quantify biophysical processes in heterogeneous landscapes. By combining Landsat and MODIS data, we can capitalize on the spatial detail of Landsat and the temporal regularity of MODIS acquisition [13
Satellite image fusion is an effective, feasible, and inexpensive means to solve this problem. Satellite image fusion is the synergistic blending of multiple colocated images that possess distinct yet complementary attributes, and the goal is to produce a result that mitigates or transcends the individual limitations of each contributing dataset. Typical fusion merges low-temporal/high-spatial resolution (fine-resolution) data with high-temporal/low-spatial resolution (coarse-resolution) data (e.g., Landsat and MODIS) to increase the availability of detailed imagery. The most widely used fusion algorithm is the spatial and temporal adaptive reflectance fusion model (STARFM) [14
] developed by Gao et al. [11
]. The algorithm has achieved success in monitoring seasonal changes in vegetation [15
]. However, this algorithm does not explicitly address the directional dependence of reflectance as a function of the sun-target-sensor geometry described by the bidirectional reflectance distribution function (BRDF) [10
] and the mixed pixel problem. For the BRDF problem, Roy et al. [10
] proposed a semiphysical fusion approach that uses the MODIS BRDF/albedo land surface product and Landsat ETM+ data to predict ETM+ reflectance on the same, antecedent, or subsequent data [4
] to solve the limitation. To address the shortcomings of STARFM, Zhu [4
] developed the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM), which adds a pair of satellite data to deduce the trend of the change in land on the basis of STARFM. Studies [4
] have shown that ESTARFM improves the accuracy of the predicted fine-resolution reflectance, especially for heterogeneous landscapes, and preserves spatial details. The input data of ESTARFM limit the use of the algorithm. Consequently, flexible spatiotemporal data fusion (FSDAF) that requires minimal input data and can capture gradual land cover type change was proposed [16
]. Furthermore, Gevaert (Gevaert and García-Haro 2015) introduced the spatial and temporal reflectance unmixing model (STRUM), which can produce an accurate reconstruction of the normalized difference vegetation index (NDVI) trajectory with the unmixing-based method via experiments that simulate situations wherein few input high-resolution images are available. However, current STARFM-based models can only capture temporal changes caused by phenology and cannot capture changes in land cover type [4
]. Hilker et al. [15
] developed the spatial–temporal adaptive algorithm for mapping reflectance change (STAARCH) model. The STAARCH model employs Landsat images taken at the start and end dates of an observation period as inputs to predict Landsat surface reflectance. By selecting the optimal Landsat acquisition, this method can determine spatial changes from Landsat and temporal changes from MODIS. Afterward, Huang proposed the unmixing-based spatiotemporal reflectance fusion model (U-STFM) to estimate the reflectance change trend without reference to the change type or land cover change. This model was proven to be more capable of capturing both types of change compared with STARFM and ESTARFM [18
]. FSDAF [16
] also closely captures reflectance changes caused by land cover conversions.
Nevertheless, all these methods are linear models that do not consider reflectance of each endmember nonlinear change. The prior assumptions of current STARFM-based models is that the reflectance change rate of each endmember is stable [4
]. This assumption is reasonable for short periods, but it might cause larger errors in some time, such as phenological change of vegetation. Paddy rice in the same region might be in different phenological periods due to different planting/harvest schedules and local environmental conditions (soil water content, nutrition, and health) in southern China [12
]. The spatiotemporal fusion method must be made highly sophisticated to suit cases where vegetation phenology changes between the base time and the prediction time. This study proposes a nonlinear algorithm based on ESTARFM that considers phenological changes through the vegetation index curve for the fusion of Landsat-8 OLI and MODIS images. We divide the rice growth process into four different phenology periods and assume that the growth rate of rice in the same phenology period is stable. We reclassify similar pixels (neighboring pixels with the same land cover type as the central pixel are called ‘’similar” pixels) of rice on the basis of this theory to ensure that pixels that participate in fusion calculation (similar pixels) appear reasonable and resemble the central pixel. The new algorithm consists of four stages, including construction of enhanced vegetation index (EVI) time series, extraction of the rice phenology period, creation of new rules of searching for similar neighborhood pixels, and fusion with the modified algorithm using phenological information. This method is a promising choice for cases where vegetation phenology changes occur from the time of the available image pair to prediction and is helpful in monitoring the dynamics of surface vegetation in situations where data are missing.
The comparison of the fusion results with the remote sensing data demonstrates that our methods are more accurate than ESTARFM. The improved performance of the modified algorithm highlights the importance of selecting similar pixels within the same phenology period for paddy rice. The modified algorithm presented in this paper has several desirable properties. ESTARFM assumes that the reflectance change rate of each endmember is stable in the fusion period. This assumption is unreasonable in several situations because vegetation grows vigorously in the main periods, contributing to a dramatic change in reflectance. Moreover, reflectance change differs in different phenology periods. Our modified algorithm comprehensively considers the phenology of paddy rice by using more coarse-resolution images. This nonlinear change is well captured by the modified algorithm, which may be partly due to the use of division of different phenology periods in our algorithm to simulate the temporal evolution of paddy rice in images. This operation eliminates the assumption. Another problem in ESTARFM is that reflectance cannot be predicted when two contradicting changes occur within a coarse-resolution pixel simultaneously and compensate for each other. Gao [11
] believed that an approach is needed to use the difference in coarse-resolution image observations during a period, and small changes in coarse-resolution image observations ensure better prediction for a heterogeneous area. This problem does not exist in our modified algorithm for paddy rice pixels because the development and decline of paddy rice is divided into different types for fusion. Although the improvement of our algorithm over ESTARFM is not as significant as that over STARFM, the advantage of our modified method over ESTARFM is that we can accurately predict the reflectance of paddy rice when the phenology change occurs in the estimation period, and our algorithm can capture the reflectance changes of paddy rice that are not recorded in the based fine-resolution images using the rice phenological information from EVI time series data. By contrast, ESTARFM can only predict reflectance changes recorded in the based fine-resolution images because it fuses images based on two input image pairs.
Some limitations also exist in our study. We employed MODIS eight-day composite reflectance data to fuse with Landsat data due to heavy cloud contamination. There are several days’ interval of date between Landsat and MODIS pairs (four-day interval in the first pair and three-day interval in the second pair). Moreover, a four-day interval exists between the prediction and verification images which may cause errors in the result analysis. Moreover, the extraction of paddy rice pixels depends on the accuracy of classification, and any misclassification exerts an adverse impact on the entire algorithm. Finally, the study area contained other types of vegetation, and we did not take these non-paddy vegetation phenology change into consideration, which is a direction for our further work.
Two parameters in both algorithms limit large-scale applications, namely, the size of the moving window and the number of land cover types. The size of moving window decides the size of area of selecting similar pixels, and the number of land cover types is decided by the surface features in actual situations. Although we can set the parameters according to the homogeneity of land surface observed from Landsat images, different conditions of regional surface suit different parameters. If we can set a dynamically changing window size to suit different surfaces, it will improve the accuracy of spatiotemporal data fusion when the fusion area is very large (e.g., China or a global scale). Lastly, we used three fine-resolution images in this study because the study area ran across different paths and rows. However, sometimes very few fine-resolution images that were temporally close to the prediction time were available for use due to cloud and shadow contamination. These challenges need to be considered in further studies.
This research demonstrated the feasibility of improving the ESTARFM algorithm by adding phenological information to create a synthetic image with high spatial and temporal resolutions. We constructed the EVI time series curve from annual MODIS eight-day composite reflectance data to divide the growth of paddy rice into four main phenology periods. We established a new rule for selecting similar pixels using the phenology division and then fusing with the new rule. We tested the proposed algorithm through an experiment using real remote sensing data in the study area. Compared with the well-known fusion method ESTARFM, our algorithm showed better performance based on visual inspection and quantitative metrics, especially for NIR and red bands. The regression coefficient of prediction reflectance produced by the modified algorithm for the NIR band reached 0.9173, whereas that by ESTARFM was 0.8855. The regression coefficient of the former was 0.3665 compared with 0.2956 of the latter for the red band. We also verified the EVI results of two predicted images, and the regression coefficient of the modified algorithm was 0.8538 compared with the 0.7794 of ESTARFM. This indicates that our algorithm has better performance in EVI for paddy rice.
In summary, this study focused on eliminating the assumption that the change rate of each endmember is linear in the fusion period to improve the accuracy of prediction images for paddy rice. The modified algorithm using phenological information is helpful for structuring images of paddy rice with high spatial and temporal resolution in the main phenology periods, which is useful for monitoring intra-annual dynamic changes in vegetation in the absence of data. Furthermore, this algorithm enriches spatiotemporal data fusion methods and provides new ideas in improving spatiotemporal data fusion methods.