The Interannual Calibration and Global Nighttime Light Fluctuation Assessment Based on Pixel-Level Linear Regression Analysis

Abstract

The Operational Linescan System (OLS) carried by the National Defense Meteorological Satellite Program (DMSP) can capture the weak visible radiation emitted from earth at night and produce a series of annual cloudless nighttime light (NTL) images, effectively supporting multi-scale, long-term human activities and urbanization process research. However, the interannual instability and sensor bias of NTL time series products greatly limit further studies of lighting data in time series with OLS. Several calibration models for OLS have been proposed to implement interannual corrections to improve the continuity and consistency of time series NTL products; however, due to the subjective factors intervention and insufficient automation in the calibration process, the interannual correction study of NTL time series images is still worth being developed further. Therefore, to avoid the involvement of subjective factors and to optimize the Pseudo-Invariant Features (PIF) identification, an interannual calibration model Pixel-based PIF (PBPIF) is proposed, which identifies PIF by pixel fluctuation characteristics. Results show that a PBPIF-based model can reduce subjective interference and improve the degree of automation during the NTL interannual calibration process. The calibration performance evaluation based on Total Sum of Lights (TSOL) and Sum of the Normalized Difference Index (SNDI) shows that compared to the traditional PIF-based (tPIF-based) and Ridgeline Sampling Regression based (RSR-based) models, the PBPIF-based one achieves better performance in reducing NTL interannual turbulence and minimizing the deviation between sensors. In addition, based on the corrected NTL time series products, pixel-level linear regression analysis is implemented to maximize the potential of the NTL resolution to produce global Light Intensity Change Coefficient (LICC). The results of global LICC can be widely applied to the detailed study of the characteristics of economic development and urbanization.

1. Introduction

Nighttime light (NTL) images effectively depict the distribution of artificial light on the earth’s surface, and have become an important indicator of the intensity of urban social and economic activities, and has been widely used in urbanization research [1,2,3,4,5,6,7]. In particular, the unique advantage of NTL collections in space-time span makes it more straightforward to carry out a global economic assessment and regional development research, compared to traditional statistical index accounting, which usually takes administrative unit as a statistical unit, lacking effective and accurate spatial location information. Currently, NTL products that are widely used worldwide include Defense Meteorological Program/Operational Line-Scan System (DMSP/OLS) NTL datasets and National Polar-Orbiting Partnership/ Visible Infrared Imaging Radiometer Suite (NPP/VIIRS) Day/Night Band (DNB) datasets [8,9,10]. Elvidge et al. [11] and Miller et al. [12] have demonstrated the superior performance of the new generation of NPP/VIIRS DNB data in terms of light range, radiation calibration and resolution, especially compared to DMSP/OLS NTL data; nevertheless, the value of long-term historical DMSP/OLS NTL archives should not be underestimated, as it is an irreplaceable reference for recording global human activities, revealing the global economy and exploring urban development patterns in past decades [13].

In recent years, DMSP/OLS NTL datasets have been widely applied in urbanization and economic development research around the world, including urban impervious expansion [14,15,16], urban development models [17,18,19] or population and Gross Domestic Product (GDP) estimation [20,21,22,23,24]. However, two main defects and limitations can be recognized in DMSP/OLS NTL time series datasets: the interannual instability of the same sensors and the deviations between different sensors in the same year. On the one hand, the regional or global nighttime light intensity over a long time span usually shows a relatively stable trend. In fact, due to the interannual anomalous perturbation of the sensor, the observed annual Total Sum of Lights (TSOL) fluctuations from a certain sensor in the time series occasionally exhibit irregularity and randomness. On the other hand, the regional or global TSOL for a given year is constant, so the results observed by different sensors should theoretically be consistent. However, in NTL time series datasets, the TSOL recorded by different sensors in the same year is biased, due to the slightly different working conditions and performance between OLS sensors mounted on different satellites. Considering the enormous potential of NTL in describing socio-economic or human activities [25], consistent and comparable NTL time series images can help to further understand how human activities drive and respond to environmental changes [26]. Therefore, the systematic error correction (inter-annual correction) for comparative analysis of historical NTL time series datasets must be implemented.

For daytime optical remote sensing data, the methods of radiometric calibration mainly include absolute calibration and relative calibration. Absolute correction converts Digital Number (DN) value to radiation value taking advantage of transmission models or in-situ surface reflection measurement empirical models; relative correction achieves image matching by implementation of "constant" features of Pseudo-Invariant Features (PIF) between images [27]. Although these two types of calibration methods have achieved abundant results in daytime optical remote sensing data [28], a great challenge still remains for night light data due to the lack of onboard calibration systems and the difficulty of identifying PIF. At present, for the interannual correction method of NTL time series datasets, absolute correction based on surface reflectance measurement is rare [29], and the relative correction using PIF has gradually become widespread [30,31,32,33,34,35,36,37]. Similar to the PIF-based relative correction model for daytime optical remote sensing data, the traditional PIFs-based (tPIF-based) NTL interannual correction uses prior knowledge (usually referring to the speed of economic development) to select specific "PIF" targets and set reference images, and uses PIF offset rules (correction functions) between uncalibrated and reference images to correct global images. Obviously, the core assumption of tPIF-based correction is that the attribute characteristics (DN values) of PIF objects should be kept constant between different images. Since the intensity of illumination emitted by PIF objects, especially local and small-scale PIF objects, is not completely constant (e.g., High-pressure Sodium (HPS) lamps losing 20% brightness during their lifetime [38]), systematic and unknown deviations could arise during the correction process. Therefore, the pre-selection of constant PIF objects (pixels) is critical for the PIF-based interannual correction. Although the subsequent tPIF-based calibration methods differ in detail (calibration model, invariant target area), most of them follow the calibration paradigm proposed by Elvidge et al. [31]. Recently, scholars have been expanding the methods of PIF pre-selection and are gradually changing from regional PIF to special light sources, including specific industrial parks [38], streetlights [39] and artificial target light sources [40]. The above NTL interannual correction based on specific and small-scale light source object ensures the constant characteristic of PIF to a certain extent, but it is mostly suitable for local and small-scale light calibration. In order to avoid PIF recognition and to expand the applicable scenarios of the interannual correction model, based on the ridge regression model [41], Zhang et al. [13] proposed the Ridgeline Sampling Regression (RSR) model for the interannual correction by capturing the ridgeline function of the pixel density map between uncorrected and reference images. Theoretically, if there is no deviation between image pairs and no real change in NTL intensity, the ridge line of pixel density map will be in a perfect 1:1 diagonal line. However, the difference between images usually results in the non-linear distribution of ridges in density maps. Therefore, the correction function can be obtained by manually selecting point pairs on ridgelines with different DN values [13].

Although the tPIF-based interannual correction model has been continuously promoted and successfully applied to the interannual correction of regional and global NTL time series, the subjective uncertainties caused by artificial PIF recognition should still be avoided as far as possible during image processing. Another prominent limitation is that the existing PIF selection schemes mostly use local and small-scale DN value deviation rules to correct global NTL data while ignoring regional differences in light image deviation, which were observed by Pandey et al. [26]. The RSR-based method of NTL interannual correction by capturing ridge function from pixel density map avoids the interference caused by PIF recognition, but a mass of useless pixels are included in scatter plotting and calculation, which makes image correction more complex and less automated [13]. Considering the shortcomings of tPIF-based and RSR-based correction models in the process of correction, this paper proposes a pixel-based PIF (PBPIF) interannual correction model to identify PIF based on fluctuation characteristics of the pixels themselves. Differently from the existing PIF recognition and selection schemes, the PBPIF model generates multi-band NTL images by multi-band synthesis method, and carries out linear regression on the multi-band pixel values at each pixel position, one by one (Figure 1). According to the pixel-by-pixel regression coefficients, the automatic PIF recognition is realized in stable lighting areas worldwide, and the image calibration function is constructed based on the ridge features of PIF pixels. In addition, in order to evaluate the performance differences between the proposed PBPIF, tPIF-based and RSR-based calibration models, the global and national turbulence and sensor bias of NTL corrected by different models are compared. Finally, this paper further expands the global application of pixel regression based on the DMSP/OLS NTL time series images corrected by the PBPIF-based model. Finally, the paper further expands the application of PBPIF-based calibration results, performing linear regression on global stable light pixels and maps global Lighting Intensity Change Coefficient (LICC) from 1992 to 2013 to reveal global light fluctuation characteristics. The results of the research not only accurately reflect the long-term global land surface dynamic changes, but also provide important data to support global economic and urban development research.

2. Data

The DMSP/OLS NTL time series datasets are provided by the National Geophysical Data Center (NGDC), which is a stable lighting product in cities, the countryside and other places after eliminating the impact of accidental noise such as fire. The spatial resolution of the image is 30 arc seconds (the spatial resolution of the equator is about 1 km), and the DN value of the image represents the average light intensity, ranging from 1 to 63. The NTL datasets provide the longest span of light data to date, and the 22-year uninterrupted nighttime lighting image from 1992-2013 was rigorously screened and clouded. In fact, due to the long-time span, the images were collected from six different satellites: F10, F12, F14, F15, F16 and F18.

Although the NTL datasets eliminate transient light noises such as fires or auroras, there are still non-negligible deviations and noises in the NTL datasets due to the large time span, cloud interference, sensor performance degradation and gap between sensors. Figure 2 shows the Total Sum of Lights (TSOL) interannual trend for each scene from different sensors over a 22-year period. There are large differences and disturbances in the NTL time series datasets, especially in the observations of different sensors. For example, there are obvious differences and jumps in the interannual fluctuations of the images collected by F15 and F16 sensors during their respective time periods. At the same time, in some years, such as 1998-2008, there were two different light images from two different sensors in the same year. Theoretically, the distribution characteristics of surface lighting and TSOL measured by the two sensors at the same time should be very close. Conversely, due to the possible interferences and deviations mentioned above, the TSOL of these images from different sensors in the same year is quite different, and the deviations show diversity in different years.

3. Methods

3.1. Interannual Calibration of DMSP/OLS NTL Time Series Datasets

3.1.1. Automatic Recognition of PIF

The most critical parameter affecting the calibration performance of the PIF-based calibration model is the PIF. Previous PIF-based calibration models often set specific PIF regions with reference to economic statistics, social development levels, etc., as for instance in Sicily [31,32], Heilongjiang Province [33], Puerto Rico [34], Los Angeles [35], Lucknow region [37], South-west England [38], etc. Although the PIF region in the above model maintains a relatively stable level in the context of global economic development, there is still a risk of low change, as the positive correlation between the regional economic level and the TSOL is not fully matched. In addition, due to the excessive involvement of subjective factors the bias of PIF sample selection and the low degree of automation in PIF identification, an efficient multi-PIF sample and objective PIF selection method is worthy of being proposed. The widely used V4 version of DMSP/OLS NTL image has a global total of 7.25 × 10⁸ pixels ($43201\times 16801$), in which most of the stable light pixels experienced a light intensity change (increase or decrease) over 22 years. Therefore, in order to minimize the risk of PIF low change, the proposed PIF automatic recognition method at the pixel level needs to be able to efficiently select the PIF pixels with the smallest light intensity change among the globally stable light area during the past 22 years.

In this paper, linear regression is performed on each pixel location of the 22-year NTL dataset, and the regression coefficient is used to identify the light intensity change rate of the pixel location to achieve quantitative evaluation. Due to the partial overlap of light images in 22 years (some years have light images from different sensors), it is necessary to compare the time series combinations from different sensors and select the group with the smallest disturbance to perform the change detection based on linear regression. In theory, with the development of economy, from 1992 to 2013, the global total lighting brightness should show a more obvious upward trend that matches the level of urbanization [42,43,44]. Therefore, the TSOL linear regression of light images under different combinations was implemented from 1992 to 2013 to measure the continuity of time series light images. By comparing the coefficients and R² of linear regression under different combinations, this paper finally selected the combinations of F101992, F101993, F121994, F121995, F121996, F121997, F121998, F121999, F152000, F152001, F152002, F142003, F162004, F162005, F162005, F162006, F162007, F162008, F162009, F182011, F182012 and F182013 to synthesize multi-band (22 bands) global lighting images. In addition, in order to eliminate the interference of non-light area on PIF pixel recognition and ensure that the acquired invariant target area has stable light and small fluctuation in the 22 years’ time, zero-value pixels in the light images are not considered in the synthesis process. Theoretically, the change coefficient of the global stable light pixels should have a normal distribution, that is, the high-growth and high-attenuation pixels are in the normal distribution of the two tails (Figure 3). Considering the characteristics of PIF used to construct correction function (light intensity should keep low change rate) and according to the sensitivity analysis results, this paper chooses about 10% of the pixels (corresponding to a coefficient $\pm 0.05$) as PIF pixels. By widely selecting 22-year weakly changing light pixels as PIF objects on a global scale, PIF samples can be greatly enriched, reducing the risk of low PIF changes and avoiding sample bias.

3.1.2. Reference Image Selection

Until now, the existing interannual calibration methods for DMSP/OLS NTL datasets have been based on relative radiation correction. In the calibration process, the selected reference image is used as a standard, and uncorrected images are empirically calibrated to match reference images and improve the comparability of time series images. Therefore, the selection of reference images is particularly critical in the relative radiation calibration process. Actually, in long-term sequence NTL images, differences between images (including differences caused by actual surface changes and differences caused by noise, sensors, etc.) increase with time span. Naturally, the greater the time span between the selected reference image and the image to be corrected, the greater the difference between the images. In order to reduce the influence of time interval on image pairs, we refer to the selection of Zhang et al. [13] and take the F152000 of NTL time series as the reference image. On the one hand, the observation time of F152000 is in the middle of 1992–2013 so it can minimize the time span between the reference image and uncorrected images to reduce the “burden” of calibration. On the other hand, there are two NTL images of F14 and F15 in 2000. Among them, the TSOL of NTL image from F14 sensor is low, and the F152000 is used as reference image because of its better continuity with F12 and F15 sensors.

3.1.3. Calibration Function Setting

The PIF pixels of NTL images from F101992 to F182013 are extracted based on the results of PIF automatic recognition. The RSR is used to determine the interannual calibration function of the NTL time series image to improve the consistency of data collections. The RSR extracts the ridgeline by placing the reference image and the uncorrected image on the two axes of the scatter plot, respectively. The distorted ridgeline in the image scatter plot captures the deformation relationship between the reference image and the uncorrected image, which can be used to develop the correction equation.

In fact, in the previous RSR-based correction method, it is computationally intensive to incorporate all the pixels of the global NTL image into the scatter plot of the image pair. Moreover, besides a small number of PIF pixels, the image scatters plot also contains massive actual DN change pixels on the surface, which are not necessary to be included in the calculation. Since the massive useless pixels are included in the scatter plot, the complexity of automatically extracting the ridgeline is steep and difficult to achieve. Therefore, Zhang et al. [13] used the manually extracted data pairs to reduce the amount of ridgeline extraction calculations in the ridgeline extraction, which increased the subjective factors in the whole calibration process. Differently, the PBPIF proposed in this paper only retains about 10% of stable light pixels, which greatly reduces the calculation amount in the ridgeline extraction process, making the automatic ridgeline extraction possible. In addition, for the scatter plot of the extracted PIF pixels, the reference image pixel values correspond to a series of approximate uncalibrated image pixel values, which can be averaged to further reduce the computational complexity of ridge fitting. Finally, based on the deformation characteristics of ridgelines, cubic polynomials are used to define correction models:

$${\mathrm{y}=\mathrm{ax}}^3{+\mathrm{bx}}^2+\mathrm{cx}+\mathrm{d}$$

(1)

where y is the pixel value of the reference NTL image, x is the pixel value at the relevant location in the image to be corrected, and a, b, c and d are coefficients, respectively, which are estimated by the least square method.

3.2. Evaluation of Calibration Results

In order to evaluate the effectiveness of the interannual correction of DMSP/OLS NTL images, a common method is to compare the continuity of TSOL of time-series NTL images [45]. TSOL is the sum of nighttime radiance and brightness in the region and is an important indicator to measure the overall trend of regional economic development. It is worth noting that TSOL is not a real physical quantity: by way of example, values are biased, as the high latitudes have more TSOL per square kilometer than low latitude countries. Considering that the latitude bias of TSOL is relatively stable and systematic, it still has a potential in the assessment of the performance of interannual corrections. Besides, the image collection of sensors in some years overlaps, so the difference between nighttime light images from different sensors in the same year is another important method to measure the correction effect. The Normalized Difference Index (NDI) is usually used to evaluate the difference of DN values between two periods of images in adjacent years. The larger the absolute value, the greater the interannual variation of the brightness of the light, and the greater the possibility of abnormal data fluctuation. The Sum of the Normalized Difference Index (SNDI) is the accumulation of NDI on the time axis, which can reflect the abnormal fluctuation level of the entire sequence image. TSOL, NDI and SNDI are defined as:

$$\mathrm{TSOL}={\displaystyle \sum }_{\mathrm{i}}^{\mathrm{n}}{\mathrm{DN}}_{\mathrm{i}}$$

(2)

$${\mathrm{NDI}}_{\mathrm{t}}=\left\{\begin{array}{cc}\frac{\left|{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{m}\mathrm{t}}-{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{n}\mathrm{t}}\right|}{{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{m}\mathrm{t}}+{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{n}\mathrm{t}}}& {\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{m}\mathrm{t}}+{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{n}\mathrm{t}}\ne 0\\ 0& {\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{m}\mathrm{t}}+{\mathrm{T}\mathrm{S}\mathrm{O}\mathrm{L}}_{\mathrm{n}\mathrm{t}}=0\end{array}\right.$$

(3)

$$\mathrm{SNDI}={\displaystyle \sum }_{\mathrm{i}=1}^{12}{\mathrm{N}\mathrm{D}\mathrm{I}}_{\mathrm{t}}$$

(4)

where ${\mathrm{DN}}_{\mathrm{i}}$ is the light image pixel value of the i position; ${\mathrm{NDI}}_{\mathrm{t}}$ is the NDI of year t, ${\mathrm{TSOL}}_{\mathrm{mt}}$ and ${\mathrm{TSOL}}_{\mathrm{nt}}$ are TSOL of two light images from m and n sensors in t year, respectively; SNDI is the sum of 12 years of NDI, and t are 1994, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 and 2007, respectively.

3.3. Global Light Intensity Change Analysis

Due to the unique potential of night lighting in reflecting human vitality and revealing economic development on the surface, the study of the global economy and urbanization based on night lighting has been widely concerned and carried out. Among them, research on the interannual variation of light intensity has been continuously promoted, and abundant achievements have been reported [46,47,48,49,50]. Limited to the continuity defect of DMSP/OLS NTL datasets, the light intensity trend analysis usually carries out TSOL comparison at national or regional scales. With the rapid economic development and in-depth analysis, TSOL change analysis at the global level has been unable to fully display the huge information hidden in the lighting data, and cannot meet the needs of sophisticated urbanization research. Therefore, based on the 22-years NTL collection calibrated by PBPIF-based model, the pixel value linear regression is used to measure the change characteristics of light intensity at the pixel scale (Figure 4). Specifically, we synthesized 22 corrected images from 1992 to 2013 and performed OLS regression on each pixel position to generate the LICC of each pixel position. The LICC is derived from the light intensity regression coefficient for 22 years at each pixel location, which can reflect the change rate of surface light intensity in the past 1992-2013. By mapping the global LICC, we can more accurately analyze the global light distribution and its spatial heterogeneity over the past 22 years.

4. Results

4.1. Interannual Correction Results of NTL Datasets

Figure 5 shows the ridgeline extraction results of F101992-F182013 images, with F152000 taken as the reference image. The feature of ridgeline reflects the effect of correction function on uncorrected images, which can be roughly divided into three stages. In the stage of low DN value of uncorrected images (approximately lower than 20), the slope of ridgeline decreases continuously, reflecting that the gain amplitude of the correction function for the light intensity decreases as the DN value increases. In the middle of the uncorrected pixel DN value (in the range 20–45), the slope of the ridgeline is stable, reflecting that the gain amplitude of the correction function is consistent at this stage. For the stage of higher DN value of uncorrected pixels (DN values higher than 45), the slope of the ridgeline tends to show an upward trend, reflecting that with the rising of DN value pixels, the gain amplitude of correction function for corrected pixels increases continuously. In addition, the fitting degree of the ridgeline to scatter points also reflects the difference of correction function in different DN value stages: at the lower or higher DN value stage, the scatter points are more concentrated and closely matched with the ridgeline; for the middle range of DN value, the scatter points are characterized by a partial discrete distribution, and the degree of fitting with ridgeline decreases.

Since the relationship between the reference image and the uncorrected image reflected by the ridgeline is nonlinear with bending, the cubic polynomial can well capture the variation characteristics of the ridgeline. The correction function parameters and correlation coefficients of the uncorrected images of F101992 to F182013 are shown in

Table 1. The NTL time series datasets correction function parameters.

Sensor	Year	y = ax3 + bx2 + cx + d				Adj.R2
		a	b	c	d
F10	1992	0.0005	−0.0512	2.3196	−5.6617	0.985
	1993	0.0006	−0.0593	2.4654	−5.7124	0.994
	1994	0.001	−0.0982	3.3487	−8.7948	0.967
F12	1994	0.0005	−0.0484	2.169	−5.0818	0.995
	1995	0.0004	−0.0396	2.0380	−5.1433	0.996
	1996	0.0005	−0.0486	2.2144	−5.3353	0.995
	1997	0.0004	−0.0382	1.9484	−4.2920	0.995
	1998	0.0004	−0.0398	2.0398	−5.8782	0.997
	1999	0.0003	−0.0299	1.8015	−4.7800	0.991
F14	1997	0.0004	−0.0447	2.2826	−3.1678	0.996
	1998	0.0005	−0.0519	2.3574	−3.8349	0.991
	1999	0.0004	−0.0429	2.2022	−3.4844	0.992
	2000	0.0006	−0.0565	2.3019	−3.789	0.989
	2001	0.0007	−0.0662	2.5309	−4.3559	0.991
	2002	0.0003	−0.0324	1.9512	−2.2158	0.995
	2003	0.0004	−0.0395	2.0087	−2.3651	0.991
F15	2000	0	0	1.0000	0	1.000
	2001	0.0004	−0.0374	1.9148	−4.1631	0.994
	2002	0.0004	−0.0396	2.0486	−5.3467	0.995
	2003	0.0006	−0.0582	2.3813	−2.0850	0.992
	2004	0.0007	−0.0671	2.5810	−2.7208	0.994
	2005	0.0005	−0.0498	2.2629	−2.2188	0.993
	2006	0.0006	−0.0570	2.3323	−1.6420	0.995
	2007	0.0008	−0.0784	2.9209	−4.1009	0.987
F16	2004	0.0004	−0.0396	2.0524	−3.3894	0.997
	2005	0.0006	−0.0583	2.4086	−2.7747	0.995
	2006	0.0005	−0.0491	2.2482	−2.8847	0.994
	2007	0.0005	−0.0483	2.2178	−4.8194	0.995
	2008	0.0005	−0.0479	2.2036	−4.3790	0.995
	2009	0.0006	−0.0580	2.4550	−4.5812	0.996
F18	2010	0.0005	−0.0455	2.0468	−8.3139	0.994
	2011	0.0004	−0.0395	2.0682	−5.7123	0.995
	2012	0.0004	−0.0411	2.1747	−7.1854	0.995
	2013	0.0005	−0.0480	2.2920	−6.9268	0.996

. The R² of each correction function is higher than 0.96, which reflects the superiority of cubic polynomial in capturing ridgeline. It is noteworthy that among all correction functions, R² of F101994 one is the lowest (0.967), and the fitting effect is lower than that of other uncorrected images. Compared with the scatter plot of “F101994-F152000”, the main reason for the low fitting effect is the lack and dispersion of scatter points in the middle part of DN value of uncorrected images.

4.2. Evaluation of Interannual Calibration Performance

4.2.1. PIF Recognition Results

As mentioned earlier, the core of the PBPIF-based calibration model is the automatic identification of PIF pixels worldwide. The multi-sample PIF pixels filtered by the light intensity change coefficient is the most prominent advantage of the proposed correction model. An essential condition for PIF pixels is to maintain a relatively stable intensity of light radiation over a long period of NTL collection. Figure 6 is a comparison of the mean DN values of PIF pixels in raw and calibrated NTL datasets. It can be found that the mean of PIF pixels of NTL images before correction is less than that after correction, but the coefficient of variation (CV) is much higher than that after correction. In statistics, CV is a standardized measure of probability distribution dispersion, and is widely used in geoscience and social statistics research [51]. The results show that after PBPIF-based model correction, the DN value of PIF in the whole time series is more concentrated, and the DN value is also increased slightly. The stability and constancy of PIF are important factors influencing the quality of PIF. The smaller CV value of the PIF in the NTL dataset indicates that the global PIF auto-recognition proposed in this paper have great advantages and potentials to avoid the risk of low PIF changes.

4.2.2. Global and regional TSOL continuity

As mentioned earlier, the most intuitive and prominent problem of DMSP/OLS NTL time series images is the turbulence and discontinuity of TSOL on the time axis. Therefore, the contrast TSOL continuity on the time axis can intuitively reflect the difference in the correction performance of different correction methods. In order to compare in a more comprehensive way the continuity and consistency of TSOL on the time axis, the global and different countries are used as spatial units for statistical TSOL. Selected countries include China, United States, Russia, France, Australia, Brazil, India, Egypt, Saudi Arabia and South Africa, which are representative countries on all continents. Meanwhile, the selected countries also have diversity in economic development speed and economic volume, such as China being the largest developing country, and during the period of study (1992–2013), the economic volume and economic development speed have undergone tremendous changes; the United States has maintained the world’s largest economy during the NTL data coverage period, and is one of the richest, most developed and highest living standards countries in the world; South Africa is one of the largest and most developed economies in Africa, with the highest level of development and the most modern infrastructure in Africa. The trend line of TSOL in the world and 10 countries with different development backgrounds are shown in Figure 7.

Results show how, compared to raw NTL time series images, the TSOL continuity of the corrected images has been greatly improved. On the one hand, the smoothness of time series TSOL from the same sensor has been improved a lot; on the other hand, the deviation of TSOL from different sensors in the same year has also been reduced. Although different calibration models can effectively improve the continuity of the raw NTL datasets, the performance of RSR-based and PBPIF-based calibration is better than that of tPIF-based model, especially the tPIF-based correction results cannot effectively alleviate the interannual instability of sensors. The TSOL continuity of RSR-based and PBPIF-based calibration results is similar, but PBPIF-based calibration performs better on the F18 sensor.

It is noteworthy that the TSOL of countries with different backgrounds also shows diversity after interannual correction. We found that the TSOL continuity of developing countries in China, Brazil, India, Egypt, Saudi Arabia and South Africa was stronger than that of developed countries. In fact, during 1992–2013, the above-mentioned developing countries have made remarkable progress in domestic economic development and urbanization development. Therefore, although the economic volume of the above-mentioned developing countries lags behind that of developed countries, they achieved substantial growth in 22 years due to their small initial total economic volume (1992), which led to a significant increase in TSOL. Developed countries, such as the United States, have already been the largest economies in the world before the start of the study period. The huge economic volume cushions the fluctuation of economic development, resulting in a more stable TSOL change than developing countries. In addition, among the above countries, the fluctuation of TSOL in Russia is disordered and the TSOL in F142002 is extremely low. Previous studies have pointed out that in the high latitudes of the Northern Hemisphere (above ${50}^{°}$ N), large-scale cloudiness, long sunshine time and aurora will interfere with the data quality and reduce the number of effective light pixels, resulting in poor performance of model correction [26]. Specifically, as a stable lighting product, V4 version NTL images exclude short-term light sources such as sunlit, moonlit, cloud interference and aurora [52]. Due to the higher probability of the above interference factors in high latitudes, the number of effective light pixels in data products is unstable. Especially for F142002, the light pixels above ${50}^{°}$ N were not recorded, which led to the sharp decrease of TSOL in Russia in 2002.

4.2.3. Comparison of SNDI

Besides intuitively comparing the continuity of TSOL in time series, SNDI is also widely used to evaluate the different sensor biases in the same year in NTL images. Generally, the same region TSOL in a particular year is a relatively stable value, so even if observed by different sensors, the results should be kept in a very close range. Therefore, SNDI can quantitatively evaluate the correction performance of the model by accumulating TSOL differences among sensors in all years.

To this end, the SDNI of the raw NTL and corrected NTL images by different calibration models in more than 230 countries were counted, respectively. The results showed that at the global scale, the SNDI mean value of the interannual corrected NTL images was significantly lower than that of the raw NTL images (Figure 8). Among them, the SNDI mean value corrected by PBPIF-based model is 45% lower than that of raw image, 32% lower after RSR-based model correction, and 43% lower after tPIF-based model correction. The global SDNI comparison results show that PBPIF-based and the tPIF-based correction models have similar performances in reducing the deviation between sensors and are better than the RSR-based model.

In addition, the three calibration models can effectively reduce SNDI in most countries or regions of the world (Figure 9). After calibration by the three calibration models, about 92.5% of the countries or regions have different degrees of decline in SNDI. Specifically, for the raw NTL images, the low SNDI (SDNI < 0.5) country or region only accounts for 2%, and the relatively low SNDI (SDNI < 1.2) country or region accounts for 51%; for the images corrected by tPIF-based model, the low SNDI country or region only accounts for 50%, and the relatively low SNDI country or region accounts for 88%; for the images corrected by RSR-based model, the low SNDI country or region only accounts for 22%, and the relatively low SNDI country or region accounts for 83%; for the images corrected by PBPIF-based model, the low SNDI country or region only accounts for 52%, and the relatively low SNDI country or region accounts for 88%. The comparison results show that the NTL images corrected by PBPIF-based and the tPIF-based model greatly inhibited SNDI between countries or regions, but the PBPIF-based model performed better. Although the results of RSR-based model correction have been greatly improved compared with the raw image, the SNDI suppression is slightly inferior to the other two correction models.

Furthermore, from the SDNI distribution of the country or region, we find that the low-SDNI units are mainly distributed between latitudes of 10 to 50 degrees, while the SNDI in some countries near the equator is still greater than 0.5 after calibrating. These countries mainly include South Sudan, Uganda, Central African Republic, Congo, Liberia, Ivory Coast in Africa, Sri Lanka, Cambodia, Papua New Guinea in Asia, Costa Rica, Panama, Cuba and Haiti in Central America. Considering the special climatic conditions near the equator, the blocking effect of cloudiness on lighting may be an important cause of this phenomenon.

4.3. Global Night Lighting Intensity Changes

4.3.1. General Characteristics

Based on the pixel level OLS regression analysis of DMSP/OLS NTL images from 1992 to 2013, the 22-year global LICC in stable lighting areas were produced for the first time (Figure 10). The global LICC fluctuated between −3.65 and 4.27 for 22 years (mean value is 0.268), which indicates that despite the spatial heterogeneity of global nighttime LICC changes over 22 years, the overall LICC showing an upward trend, the DN value increases by 0.268 per year. In terms of the coefficient composition of global stable light pixels, about 30% of stable light pixels have experienced a light intensity decline in 22 years, while the intensity rising pixels accounted for the majority, reaching about 70%. Moreover, pixels with LICC below 0 are mainly distributed in North America and Eastern Europe, and pixels with LICC greater than 0 are more widely distributed, mainly including East Asia, South Asia, Western Europe, North America, and Eastern South America. Specifically, among the pixels with rising light intensity, the light pixels with a growth rate higher than the average value of 0.268 are mainly distributed in East Asia, Southeast Asia, West Asia, Europe, and southeastern South America; the pixels whose growth rate is lower than the average are mainly concentrated in South Asia, Western Europe and southeastern North America.

In addition, Figure 11 shows the average of LICC (avgLICC) with the nation as statistical unit. At the national level, 88.5% of countries or regions have avgLICC greater than 0, while only 11.5% of countries or regions have avgLICC lower than 0. Among the countries where avgLICC is below 0 (except for countries in high latitudes and near the equator that may be disturbed by clouds and other factors), Ukraine, Moldova, Uzbekistan, Tajikistan and Zimbabwe experienced a decline in GDP during the study period, which also coincided with the results of the fluctuation coefficient.

In fact, the amount of stable light pixels is neglected when the avgLICC is counted under the national statistical unit, which often results in higher avgLICC in developing countries with small pixel amount but high fluctuation coefficient. Therefore, we have calculated the sum of the LICC (sumLICC) by country or region on a global scale (Figure 12). The results show that on a global scale, the sumLICC in Canada is the lowest, while China has the highest sumLICC. For China, since the reform and development in the 1980s, economic and urbanization has continued to advance, and its economic volume and economic growth have remained stable at a high level. During the study period, China’s GDP growth rate remained at an average annual rate of 10.27% (2.8% of the world average). Compared with 1992, the total GDP in 2013 increased by 22.5 times (3 times of the world average) and surpassed Japan as the world’s second-largest economy in 2010. High-speed economic development and huge economic volume make it indisputable as the most representative country with positive fluctuation of night light intensity in 22 years. Correspondingly, the average annual GDP growth rate in Canada during the study period was only 2.58%, and the total GDP increased by about 3.1 times. The lower GDP growth rate and the special geographical location (located in high latitudes) make the sum of the LICC in the development process at a lower level.

The countries with the high level sumLICC include the United States and India. The United States, as the world’s largest economy, is recognized as one of the most developed economies in the world. During 1992–2013, the average annual GDP growth rate was maintained at 2.59%, and the total GDP expanded by 2.6 times. India, BRICS Members, during the study period, their economic growth rate is second only to China, and the total domestic GDP has expanded by about 6.5 times, making it one of the fastest growing countries in the world. In addition, the number of countries where sumLICC is at a relatively high level has increased, including Mexico, Brazil, Spain, France, Italy, Germany, Poland, Egypt, Saudi Arabia and Iran. Among these countries, Brazil is a member of the BRICS, ranking first in South America in GDP, and Mexico’s economic development has been developing rapidly since its entry into the North American Free Trade Area (NAFTA) in 1994. Spain, France, Italy, Germany and Poland, as traditional developed countries, maintained relatively stable GDP growth during 1992–2013, with an average annual GDP growth rate of 2%, and the total GDP increased by about 2.5 times compared with 1992. The average annual growth rate of GDP was 4.5%, 2.9% and 2.7% respectively, and the total GDP increased by 6.89 times, 5.44 times and 7.33 times respectively compared with 1992.

4.3.2. Regional Characteristics

Previous studies on time series NTL fluctuation mainly focused on national or regional scales, only revealing the overall change characteristics and trends of spatial units. To break through the limitation of space unit and maximize the utilization of NTL data, a global night light fluctuation analysis based on pixel-level is proposed in this paper. Figure 13 shows the LICC distribution in six typical regions around the world, which can reflect the fluctuation characteristics of NTL in the region. The area in Figure 13a is located in the southeastern part of the United States and is one of the areas where nighttime lights are densely distributed. The distribution of high-value LICC has a ring structure, such as Atlanta and Charlotte. The pixels with high-value coefficients are mostly around the periphery of the urban core area and stacked into a circular structure. For Figure 13b, it is located in central Europe and includes Italy and its surrounding areas, with a high percentage of stable lighting coverage. Among them, the high-value LICC has formed an agglomeration belt from Venice in the east to Turin in the west in Italy. The agglomeration zone is located in the Po Plain in northern Italy, which is the most developed and densely populated area in Italy [53]. Figure 13c depicts the distribution of LICC in the Middle East, particularly in Egypt, Israel, Palestine, Saudi Arabia, Lebanon and Syria. The high-valued LICC pixels in the area formed two clusters, one along the Nile and the estuary delta in Egypt; the other was the Jerusalem and Amman provinces on both sides of the Jordan River. Compared with other areas, the climate in this area is dry and precipitations are limited. With tropical desert climate as the main climate, water resources play a decisive role in urban location and development [54], and ultimately form a regional high-value LICC coordinated with the distribution of water resources.

In South America, this paper chooses cities in southeastern Brazil including Brasilia, Sao Paulo and Rio de Janeiro as case areas (Figure 13d). Although Brasilia is the capital of Brazil, its lighting area and amount of high-value LICC pixels are smaller than those in coastal areas of Sao Paulo and Rio de Janeiro. Sao Paulo is Brazil’s largest and most economically prosperous city in South America, with the largest lighting area and the largest number of high-value LICC in the region [55]. For Rio de Janeiro, once the capital and the second largest city in Brazil, the LICC of Macae, the center of the coastal oil industry in the east of Rio de Janeiro, is higher than that of Sao Paulo. For Asia, this paper chooses South Asia and East Asia, which have the largest lighting area, as the case areas of LICC comparison (Figure 13e,f). In South Asia, India has the largest stable lighting area. India is one of the fastest growing economies in Asia [56]. Its stable light pixels form the northern cluster centered on New Delhi, the central cluster centered on Mumbai and Hyderabad, and the southern cluster centered on Bangalore. The case areas in East Asia include eastern China, Taiwan and the Korean Peninsula. With the rapid development of economic globalization and regional economy, East Asia has gradually grown into the leader of global economic development, especially China. The eastern part of China not only has the largest number of high-value LICC pixels, but also has relatively aggregated distribution, which is quite different from that of other parts of the world. Specifically, in the process of urbanization, China’s eastern region has gradually formed China’s two largest urban agglomerations: the Yangtze River Delta urban agglomeration and the Pearl River Delta urban agglomeration. During the urbanization process, cities in the estuary delta are expanding with each other, and urban agglomerations with compact spatial organization and high degree of integration are gradually realized, which reflected on the LICC is the aggregation of high-value coefficients. Conversely, the distribution of high-value LICC in North China is discrete and presents a pattern of scatter distribution.

5. Discussion

5.1. PBPIF-Based Calibration Model and Comparison of Results

As mentioned above, the tPIF-based model assumes that the light intensity in a specific region is stable, and the global pixel correction is performed using the deviation characteristics of DN values in the region. The RSR-based model uses density scatter plot of image pair to extract ridgeline as conversion function between NTL images. The PBPIF-based model proposed in this paper synthesizes the advantages of tPIF-based and RSR-based models to achieve more efficient and automatic time series NTL calibration. The calibration procedures for the above three NTL interannual calibration methods are different. Among them, the tPIF-based model uses the prior knowledge to artificially select the constant area of the light intensity, and uses the deviation relationship between uncorrected images and reference images in the invariant region to correct the time series NTL in the whole research area. The advantage of this calibration model is that the processing flow is simple, and the calculation amount is small, while the disadvantage includes strong subjective participation and low automation. The RSR-based model uses the density scatter plot of all pixels of the reference image and uncorrected image to extract ridgeline manually, and then determines the correction function by fitting ridgeline. The RSR-based model avoids the subjective error caused by PIF selection, reduces the degree of manual participation during the correction process, and makes up for the shortcomings of tPIF-based model. However, there are still some shortcomings in this method. On the one hand, the correction process needs to plot the density scatter plot of all the pixels ($1.45\times {10}^9$ pixels of the two images), which greatly increases the difficulty of data processing. On the other hand, due to extremely dense pixel scatter, the ridgeline can only be extracted by manual screening, which reduces the degree of automation of the processing process. The PBPIF-based model proposed in this paper can automatically find PIF pixels on the global scale by using the fluctuation characteristics of the pixel itself, which greatly reduces the subjective participation. Meanwhile, the global selection of PIF pixels can avoid the errors caused by the use of local invariant target regions in the traditional PIF model to correct the global NTL. Compared with the raw NTL images, the number of PIF pixels identified decreases sharply, which makes it possible to automatically extract ridgelines, or to fit the distribution of scatters.

In addition, TSOL and SNDI at global and national scales were compared to assess the calibration benefit of PBPIF-based and the current common interannual correction models. The results show that RSR-based model and PBPIF-based model are better than tPIF-based model in eliminating the turbulence of NTL images in time series. The TSOL interannual curve of NTL corrected by RSR-based model and PBPIF-based model is smoother and more synergistic with the fluctuation curve of GDP (Figure 7). For SNDI, which measures the deviation between sensors, PBPIF-based works best, followed by tPIF-based and RSR-based models. In addition, the SNDI suppression results of the three calibration models for the national unit reflect that the correction performance of the three correction models is more effective at the mid-low latitudes, while at higher latitudes (Russia, Nordic countries, Greenland, etc.) the performance is general due to the known problems of high latitude NTL data. In the middle and low latitudes where most of the population and cities are concentrated, the number of light pixels accounts for about 80% of all NTL images, and the data is less disturbed by clouds, thus maintaining high quality. Therefore, profit by the wide range of PIF automatic screening at the pixel level, PBPIF-based model can obtain more PIF samples in low and middle latitudes to improve the effect of correction function. However, for countries with high latitudes and near the equator, the lighting data is disturbed by factors such as clouds, aurora and sunshine, which degrades the data quality, resulting in irregularities and differences in deviations from different sensors compared to mid-low latitudes. Therefore, the correction effect of the three correction models in high latitudes is gradually decreases. In fact, due to the natural conditions and climatic conditions of the high latitudes, the population density and urban density are lower than those in the middle and low latitudes. Although it is difficult to reduce the SNDI to the ideal level in the high latitudes for the three calibration models, the results of PBPIF-based calibration can still be used for time series analysis and vitality monitoring of urbanization worldwide considering the low population and urban density in the region.

5.2. Fluctuation Characteristics of NTL

At present, the results of the change analysis based on the corrected time series NTL datasets are constantly enriched. However, these studies tend to focus on large unit scales such as countries and regions. During the study period, the change of average light intensity and the total amount of light was used to measure the fluctuation of light, which concealed the difference of light change in the interior space of the unit, and is not conducive to mining more detailed differences in lighting changes and urban development. With the rapid development of the economy and the continuous advancement of urbanization, the analysis of the lighting fluctuations within the spatial unit can help reveal the differences in urban development and a more comprehensive understanding of the urbanization process [57]. Actually, the spatial resolution of the NTL images has reached about 1 km, which has great potential for developing more fine-scale characteristics of light intensity fluctuation. In this paper, the light fluctuation analysis based on pixel regression is implemented for corrected NTL images, which can release the information contained in NTL images and improve the precision of night light intensity fluctuation research.

From the LICC statistics of countries and regions worldwide, it can be found that the sumLICC on national or regional units can better reflect the real changes of lighting between countries than the avgLICC. In fact, there are significant differences in the distribution of global light pixels among countries, especially for stable light pixels. On the one hand, for countries with large territory, more light pixels will inevitably increase the dispersion of the overall pixel value. On the other hand, for developing countries with smaller territory, during the study period, most of the lights were concentrated in a few developed cities in the region and maintained rapid growth. Hence, the avgLICC ignores the significance of the pixel amount in light fluctuations, due to the huge difference in the number of light pixels between countries. Through the summarization of all stable lighting pixels LICC in the unit, the sumLICC can more accurately depict the lighting fluctuation characteristics of the global national scale for 22 years. The results of the national-scale sumLICC show that the growth of light intensity in emerging economies represented by China in recent years is larger than that of traditional developed countries. According to data released by the International Monetary Fund (IMF), the economy of developed economies grew by only 2.7% in 2007, and the economy of emerging and developing economies grew by 8%, and gradually became a “source of world economic stability” in the context of economic globalization [58].

In addition, there are substantial differences in LICC within the city between emerging economies represented by China and traditional developed countries represented by the United States. Figure 14 shows the distribution of LICC in four representative cities or regions in the United States and China. For a highly developed country like United States, the fluctuation coefficient of the city and its surrounding areas has the following characteristics: (1) the LICC of the urban core area is close to 0, and the fluctuation is weak; (2) there are ring-shaped high-valued LICC pixels around the core of the city, such as Las Vegas, Atlanta, and Philadelphia; (3) for the periphery of the ring-shaped LICC, the LICC in the suburban area is much lower than the urban area. The LICC of Chinese cities or regions is obviously different from the above-mentioned cities or regions and has the following characteristics: (1) the LICC in the urban core area is higher than that in the United States, and the LICC of the Central Business District (CBD) is close to 0; (2) LICC in urban transition zone and suburban area is higher than the core area, and shows a growth and integration trend, which is different from the three-tier classification of the United States cities; (3) the development of urban integration has been strengthened and presented the development mode of urban agglomeration, and the overall lighting intensity of the region has increased rapidly. As mentioned earlier, the saturation defect of the DMSP/OLS sensor results in that the DN value of the lighting pixel exceeding 63 will not be recorded correctly (more than 63 is still recorded as 63). For the most intense urban core area of NTL, the DN value is limited to 63 for a long time, which in turn leads to a low LICC. It should be emphasized that the area and light intensity of urban core area in the United States were significantly higher than that of the Chinese city in 1992. Obviously, urban core areas in developed countries bear more problems caused by the saturation defects of OLS sensors than those in developing countries, which results in that the pixels of LICC approaching 0 in urban core areas in the United States are larger than those in developing countries.

Another notable difference between Chinese and American cities is the difference of LICC in suburban areas. Specifically, LICC in urban transitional and suburban areas of China does not show a rapid decline trend, and the urban integration development model is more obvious. In the process of urban development in China, coordinated urban development and urban agglomeration have gradually become an important mode of urban development [59]. Cities with close geographical location tend to present the pattern of cluster development, and the mutual expansion of cities makes the original suburban areas change dramatically with the development of urbanization. Generally, the differences in the initial development, growth rate and development mode between the traditional developed cities and the newly developing cities represented by China and the United States can be effectively captured by the pixel-based fluctuation analysis.

6. Conclusions

Rapid development of the global economy and the reality of regional differences in development have placed higher demands for time series analysis based on NTL. In recent years, with the no-longer-used collection of DMSP/OLS NTL data, the development of sensor technology and the rising demand for night lighting monitoring, a series of new lighting data (NPP/VIIRS and Luojia-1), were launched, expanding the research and application of night lighting data. Although the radiation sensitivity and spatial resolution of the new generation of night light data are greatly improved compared with NTL, the long-term historical NTL datasets is still an indispensable data source for scholars to carry out urbanization research. In order to remedy the interannual instability of NTL datasets and achieve more fine-scale global light fluctuation analysis, an interannual correction model (PBPIF-based model) based on pixel regression is proposed to calibrate the global time series NTL data. In addition, TSOL and SNDI at global and national scales are used to assess the reliability of interannual calibration results based on pixel regression. Finally, based on the corrected time series NTL datasets, this paper uses pixel regression to analyze the fluctuation characteristics of the global stable lighting pixels.

The results show that the PBPIF-based model proposed in this paper can minimize the participation of subjective factors and improve automation during the calibration process. The evaluation results of the calibration performance show that the calibration model can improve the continuity of time series NTL efficiently, reduce the deviation between sensors and improve the accuracy and reliability of NTL images for time series analysis. Meanwhile, the PBPIF-based model has also achieved better correction results in comparison with the existing two widely used calibration models. Although the wide selection of PIF pixels globally can best avoid PIF low variation and satellite observation interference [60], the use of a strictly constant physical calibrator will be the key to the next phase of NTL interannual correction. Finally, based on the corrected time series NTL datasets, the global LICC is produced (Figure S8). Pixel-level LICC can maintain the resolution of light data and reveal the characteristics of economic development and urbanization process more precisely than traditional light data change analysis, which can be used as a new data for subsequent urbanization research.