Intercalibration and Gaussian Process Modeling of Nighttime Lights Imagery for Measuring Urbanization Trends in Africa 2000–2013

Abstract

Sub-Saharan Africa currently has the world’s highest urban population growth rate of any continent at roughly 4.2% annually. A better understanding of the spatiotemporal dynamics of urbanization across the continent is important to a range of fields including public health, economics, and environmental sciences. Nighttime lights imagery (NTL), maintained by the National Oceanic and Atmospheric Administration, offers a unique vantage point for studying trends in urbanization. A well-documented deficiency of this dataset is the lack of intra- and inter-annual calibration between satellites, which makes the imagery unsuitable for temporal analysis in their raw format. Here we have generated an ‘intercalibrated’ time series of annual NTL images for Africa (2000–2013) by building on the widely used invariant region and quadratic regression method (IRQR). Gaussian process methods (GP) were used to identify NTL latent functions independent from the temporal noise signals in the annual datasets. The corrected time series was used to explore the positive association of NTL with Gross Domestic Product (GDP) and urban population (UP). Additionally, the proportion of change in ‘lit area’ occurring in urban areas was measured by defining urban agglomerations as contiguously lit pixels of >250 km², with all other pixels being rural. For validation, the IRQR and GP time series were compared as predictors of the invariant region dataset. Root mean square error values for the GP smoothed dataset were substantially lower. Correlation of NTL with GDP and UP using GP smoothing showed significant increases in R² over the IRQR method on both continental and national scales. Urban growth results suggested that the majority of growth in lit pixels between 2000 and 2013 occurred in rural areas. With this study, we demonstrated the effectiveness of GP to improve conventional intercalibration, used NTL to describe temporal patterns of urbanization in Africa, and detected NTL responses to environmental and humanitarian crises. The smoothed datasets are freely available for further use.

1. Introduction

Urbanization can be broadly defined as the transition of settlements from rural to urban environments and the growth of existing metropolitan areas. The proportion of the world’s population living in urban areas is projected to be 66% by 2050 [1] and sub-Saharan Africa currently has the world’s highest annual urban growth rate of any continent at 4.2% [2]. Urbanization is known to impact a range of socio-economic issues including public health, education, environmental quality, and economic development [3]. The public health effects of urbanization are complex with both negative and positive outcomes. For example, numerous studies have indicated that urban environments may have a mitigating effect on malaria transmission [4], while at the same time increasing the incidence of dengue fever [5]. Thus, innovative and reliable methods and datasets for monitoring spatiotemporal changes in urban areas are paramount.

Nighttime lights imagery (NTL) [6], maintained by the National Oceanic and Atmospheric Administration (NOAA), offer a unique viewpoint for studying urban trends. These data, available as annual composite images from 1992 to 2013 provide a means for spatiotemporal analysis on a global scale. Although noise removal and other corrective processing are applied to the NTL imagery by NOAA to make them usable for analyses, inherent shortcomings remain. Foremost among these is the lack of inter- and intra-annual calibration between satellites. The satellite sensors that collected these data lacked on-board calibration capabilities and a system for recording in-flight gain changes [7]. When observing raw imagery, this causes annual fluctuations in recorded brightness rather than the gradual increase expected with typical growth in urban populations. Elvidge et al. [7] developed an empirical procedure to allow ‘intercalibration’ of the NTL data. Often referred to as the invariant region and quadratic regression method (IRQR), this method has been applied across a number of settings [8,9,10,11,12,13]. It is a regression based method that relies on a high gain reference image and a reference area where illumination has changed little over time.

An alternative intercalibration method was developed by Liu et al. [11]. These authors built on the IRQR method by applying intra- and inter-annual corrective algorithms and make use of a thresholding technique [12] that relies on land use/land cover data to extract urban information. Wu et al. [13] presented an alternative strategy for applying the IRQR method that included pixel saturation correction and the use of the power-law function for regression analysis. Li et al. [14] used an automatic algorithm to extract reference area pixels and entered them into a linear regression model where outliers were iteratively discarded to refine the intercalibration equation. Finally, Zhang et al. [15] employed a novel sampling strategy along the ‘ridgeline’, i.e., the densest part of plots generated between the reference and target images, to derive calibration models. Regardless of the approach, the intercalibration methods reviewed here all showed improved performance as indicated by their respective evaluation procedures. In general, method evaluation was based on the use of GDP, which has demonstrated a positive linear relationship to NTL in various studies [14,15,16,17]. However, the shortcomings of these alternative intercalibration methods is that they were conducted within a limited geographic scope, required multiple reference datasets such as land use/land cover or population, and were not sufficiently validated.

Precisely intercalibrated NTL data is ideal for mapping urban extents as Li and Zhou [18] describe in their recent systematic methodology review of this subject. Others, such as Ju et al. [19] have focused on the characterization of urban dynamics in China using pixel-based time series trajectories to identify five distinct patterns of urban growth. Similarly, Zhang and Seto [20] identified urbanization typologies on a worldwide basis and validated the accuracy of their results with multispectral imagery. Ma et al. [21] both analyzed and predicted urban development at the municipal level using three different regression models to measure the relationships between NTL and population, GDP, built-up area, and electric power. Finally, Cauwels et al. [22] applied NTL to the concept of urban agglomerations by using a threshold method combined with a segmentation function that identifies clusters of contiguously lighted pixels.

The primary objective of this study was to generate an intercalibrated time series of annual NTL images at the continental scale for Africa from 2000 to 2013 for use in measuring changes in urbanization over time. This particular period was chosen to align with significant reductions in the incidence of malaria in Africa, which occurred primarily as a result of scale-up of large-scale interventions, but also due to environmental changes and economic development, including urbanization [23]. Since substantial inter-annual noise remained even after IRQR intercalibration, we evaluated the use of Gaussian process methods (GP) to generate a smoothed series of NTL images free of temporal noise signals. We then demonstrated the utility of the smoothed dataset for describing patterns of urbanization in Africa and studying relationships between NTL and economic and population indices.

2. Materials and Methods

2.1. Data Source

The Nighttime lights imagery used for this study was produced by the U.S. Air Force as part of their Defense Meteorological Satellite Program (DMSP). The Operational Linescan System (OLS) sensors on DMSP satellites have specialized low light detection capabilities across visible and near infrared wavelengths. During the 22 years spanning 1992 through 2013, six satellites were active and their sun-synchronous near-polar orbits provided global coverage twice per day. The NOAA National Centers for Environmental Information (formerly National Geophysical Data Center) has been the custodian of the DMSP-OLS data archives since 1992. More recently in 2010, NOAA produced a complete time series consisting of cloud-free composites made using the archived data for each calendar year by satellite. This product is known as the version 4 DMSP-OLS Nighttime Lights Time Series and is freely available on the NOAA website [6]. The composites are provided on 30 arc second image grids spanning from −180 to 180 degrees longitude and from −65 to 75 degrees latitude. The digital numbers (DN) have a 6-bit dynamic range with values from 0 to 63. They also contain four bands representing distinct levels of data processing: raw average of the DN values, a cloud-free coverage tally, average DN of cloud-free light detections multiplied by the percent frequency of light detection, and stable lights. We utilized the stable lights band which consists of the average of the visible band DN values and represents areas with persistent lighting. It includes gas flares from petroleum operations, but ephemeral events such as fires have been discarded and background noise from other sources has been removed.

For this study, the NTL stable lights composites from satellite-years 2000–2013 were used (

Table 1. Stable lights composites for DMSP-OLS satellite-years 2000–2013.

Year	Satellites
	F-14	F-15	F-16	F-18
2000	F142000	F152000
2001	F142001	F152001
2002	F142002	F152002
2003	F142003	F152003
2004		F152004	F162004
2005		F152005	F162005
2006		F152006	F162006
2007		F152007	F162007
2008			F162008
2009			F162009
2010				F182010
2011				F182011
2012				F182012
2013				F182013

). Google Earth Engine [24] provided easy access to Version 4 annual composites of the DMSP-OLS Nighttime Lights Time Series and the image processing tools required to perform the bulk of the analysis.

2.2. Image Processing

Not all the variation observed in the stable lights composites of NTL can be attributed to actual changes in urbanization [7]. For instance, variation across time and space in the DN values recorded can also be explained by human activities other than urbanization, by physical phenomena like the refraction of light or by measurement error due to the use of different satellites. Here we explain the steps taken to address the different sources of variation.

2.2.1. Intercalibration

Since the NTL satellites had no standard for calibration of their sensors, the data were pre-processed for temporal analysis. Elvidge et al. [7] developed an empirical procedure for intercalibration of NTL annual composites from different satellite-years. Also known as the IRQR method, it is a regression based method that relies on a reference area where illumination has changed little over time and an accurate baseline image dataset. Sicily is the standard reference area for intercalibration since it has an even spread of data across the full dynamic range and its population is relatively stable with only a 0.17% growth rate from 2001 to 010 [10]. Likewise, satellite year F12-1999 has relatively high brightness values and is used as the baseline with images from all other satellite years adjusted to match its data range.

To implement the adjustment, the region of Sicily was resampled on each NTL image to a standard 0.825 km grid. This grid cell size was chosen based on the area of a 30 arc second pixel at Sicily’s latitude. The relationship between each satellite-year image and the baseline image F12-1999 was fit using the following quadratic regression model:

$$X_{i,0}=C_0+C_1{X}_{i,j}+C_2{X}_{i,j}^2$$

(1)

where X_i,0 is the DN of the ith grid cell in the baseline image, X_i,j is the DN of the ith grid cell in satellite-year image j, and the parameters C₀, C₁ and C₂ are the intercept, linear, and quadratic coefficients, respectively. For all images, DNs values less than 2 were excluded in keeping with the author’s method [7]. Figure 1 shows an example regression scattergram for satellite-year F18-2013 versus F12-1999 for Sicily. Regression coefficients generated by the model were then used to adjust the DN values observed for Africa. For each year, a set of calibrated values were computed using Equation (2):

$$X_{i,j}^A=C_0+C_1{X}_{i,j}+C_2{X}_{i,j}^2$$

(2)

where $X_{i,j}^A$ is the calibrated digital number of the ith pixel cell in satellite-year image j of Africa. To ensure calibrated DN values were within the 0–63 range, calibrated values that exceeded 63 were set to 63 and where the minimum value was greater than zero, the original zeroes were restored to preserve the background ‘darkness’ matrix.

2.2.2. Gaussian Process Smoothing

Once the satellites have been intercalibrated, there are still discrepancies between DN values at the same location-year recorded by different satellites. The occurrence of such discrepancies suggests that the observed data are in fact a noisy realization of an actual but unobserved NTL signal. This relationship can be expressed as

$$X_{i,j}^A=X_{i,j}^S+{\epsilon }_{i,j}$$

(3)

where $X_{i,j}^A$ is the observed NTL signal, and $X_{i,j}^S$ is the actual and unobserved NTL signal of the ith pixel at satellite-year $j$ and ${\epsilon }_{i,j}$ the associated observation error. The intercalibration step does not eliminate the error term, rather, it results in the error being rescaled to the same magnitude across the different satellite-years.

Gaussian process methods [25] are commonly used with models similar to that in Equation (3) for solving inference problems in time series [26,27,28] where the aim is to distinguish an unobserved signal or latent function from noisy data. The main assumptions of such models are that the noise terms ${\epsilon }_{i,j}$ are independent and identically distributed, while the realizations of the latent function $X_{i,j}^S$ are not independent and follow a multivariate normal distribution. In our case, the second assumption guarantees that the observations and latent function are correlated across time.

Here we estimated $X_{i,j}^S$ by fitting a Gaussian process with a covariance kernel [29] composed of a linear and an exponentiated quadratic term to each of 22 satellite-years (Table 1). While the first kernel implements the estimation of the NTL trend across time, the second allows identification of non-constant increments/decrements of the DN values. Additional details on our use of Gaussian process modeling and kernel function types can be found in Appendix A.

2.2.3. Annual Averaging

From 2000 to 2007, there were multiple satellites operating in any given year (Table 1). For the uncalibrated (UC) and intercalibrated (IC) test cases used throughout this study, DN values for each satellite during this time period were intra-annually averaged to produce a single annual image.

2.2.4. Gas Flare Removal

Gas flares are present in the stable lights band of the Version 4 NTL dataset and, given they are not generally associated with human settlements, they were removed. In a previous study, Elvidge et al. [30] estimated flare volume and evolution and mapped their locations and extent. As part of this study, flare locations as polygonal masks have been made available on NOAA’s website as shapefiles, a common spatial data format [6]. These were downloaded, merged, and used to mask all known flares. While these have not been updated since 2009, this dataset represents the best estimate of the locations of gas flares available. Since most gas flares in African occur in marine environments or the remote Sahara Desert, newer flares should not result in significant inaccuracies.

2.2.5. Blooming Correction

Probably the most serious non-temporal errors in the NTL images are caused by ‘overglow’ and ‘blooming’, which is the diffusion of urban light into rural areas and the magnification of light by reflective surfaces such as water and sand, respectively. In either case, the errors are not entirely due to reflected light. Additional sources of this error are the composite building process and image registration errors [31]. It was noted that overglow occurring on a single image dataset can be compounded in composite images. Also, numerous small errors in geo-registration may cause brightly lit urban cores to grow slightly at their peripheries. In both cases, the result is overestimation of urban extents and settlements on land and their deceptive extension into uninhabitable aqueous environments.

Pixel blooming caused by water reflectance from inland water bodies and coastal waters was accounted for by masking via the World Water Bodies and World Countries datasets [32], respectively. Mediating urban overestimation due to overglow is considerably more problematic. Typical solutions to this problem involve use of low light thresholding which is intended to filter out a portion of the lowest predicted DNs. The goal of these somewhat empirical approaches are to remove ‘noise’ associated with overglow from the urban boundaries without fragmenting the urban core or excluding genuine smaller settlements [33]. We have chosen to forgo the use of thresholding due to its uncertain efficacy.

2.2.6. Re-Projection

NTL composites are produced on a 30 arc second grid, so the land surface area of the pixels naturally differs according to their geographic latitude. For example, in Kampala, Uganda which is essentially on the Equator, a pixel represents approximately 0.86 square kilometers, whereas in Cape Town, South Africa (34°S latitude) pixels represent 0.71 square kilometers. This becomes problematic with spatial analysis requiring surface area measurements. When investigating temporal changes in urbanization, this is quite often the case. Thus, the processed NTL composites were projected to the Mollweide equal-area map projection for analytical purposes. This is a pseudo cylindrical map projection where the accuracy of area representation takes precedence over the accuracy of angle and shape. The composites were projected to a 1 km cell size using the nearest neighbor re-sampling technique. This step was only applied to the imagery used for the urban growth analysis described below.

2.3. Method Validation and Urban Growth Analysis

To determine relative success of the intercalibration method, the ‘sum-of-lights’ index (SOL) [7] was calculated for each satellite year. The SOL is simply the sum of the predicted DNs for an NTL image within a region of interest. Once calculated by region, it is plotted as a time series. Success is indicated by a relative convergence of SOL values in years where two satellite products are available, and a relatively continuous growth trend across the time series (Figure 2 and Figure 3).

For procedural validation, the IRQR and Gaussian process time series were compared as predictors of the satellite F12-1999 annual composite image of Sicily. Given the assumption that NTL values have not changed in Sicily 2000–2013, the F12-1999 image can be considered the gold-standard ‘observed’ values for 2000–2013 in our study. When the IRQR and GP time series are similarly considered ‘predicted’ values, the mean squared error (MSE) measurement can be used to validate and compare methods. Given that the Gaussian process calculations were based on the entire time series, MSEs were calculated using data from all years.

This enhanced intercalibration method was also used to evaluate the positive association of the intercalibrated and processed NTL time series to GDP and urban population on a continental and country-wise basis. These relationships have been demonstrated in earlier studies [34,35] and therefore are used as an indication of the efficacy of our own methods. Statistics for GDP per country were sourced from the World Bank [36] using the US dollar price in July 2016 as the benchmark. Corresponding urban population data were also obtained from the World Bank [37]. The correlation of SOL to these factors was compared both before and after intercalibration. The relationship between uncalibrated and annually averaged (UC), intercalibrated and annually averaged (IC), and intercalibrated and Gaussian process smoothed (GP) NTL with GDP and UP was estimated using linear regression. Models were fit at continental scale and for sub-Saharan African countries. Since GDP and UP data were not available for Somalia and Western Sahara, they were not included in the continental analysis.

The amount of growth in urban and rural settlements was also calculated both in absolute and proportional terms. Total urban and rural statistics are based on the area represented by all ‘lit’ pixels (DN > 0), whereas urban agglomerations were defined as contiguously lit pixels with an area >250 km² (~97 mi²).

Figure 4 provides a flow chart outlining the work flow for NTL intercalibration, image processing procedures, and validation and analysis. Figure 5 provides an enhanced view of the intercalibrated 2013 nighttime lights image for continental Africa.

3. Results

As described above in Materials and Methods, the regression based IRQR intercalibration method relies on a reference area where illumination has changed little over time and an accurate baseline image dataset. Using Sicily as the reference area and satellite year F12-1999, regression models were generated for all satellite-years (

Table 2. Regression coefficients for intercalibration.

Satellite	Year	${\mathit{C}}_0$	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{R}}^2$
F12	1999	0	1	0	1
F14	2000	1.2445	1.3076	−0.0051	0.9334
F14	2001	0.3811	1.3103	−0.0050	0.9461
F14	2002	1.2242	1.1542	−0.0030	0.9262
F14	2003	0.8802	1.2381	−0.0039	0.9444
F15	2000	0.1832	1.0418	−0.0010	0.9410
F15	2001	−0.7078	1.1191	−0.0015	0.9617
F15	2002	0.1354	0.9587	0.0008	0.9662
F15	2003	0.3589	1.4992	−0.0078	0.9336
F15	2004	0.7187	1.3200	−0.0050	0.9485
F15	2005	0.7567	1.2666	−0.0040	0.9377
F15	2006	0.9387	1.2660	−0.0040	0.9409
F15	2007	1.6464	1.2480	−0.0038	0.9056
F16	2004	0.3607	1.1809	−0.0032	0.9153
F16	2005	0.1794	1.3906	−0.0060	0.9402
F16	2006	0.1955	1.1322	−0.0017	0.9233
F16	2007	0.9177	0.8841	0.0017	0.9483
F16	2008	0.6750	0.9773	0.0001	0.9456
F16	2009	1.9043	0.9740	−0.0007	0.8381
F18	2010	2.9053	0.4593	0.0070	0.8404
F18	2011	3.1449	0.6453	0.0036	0.8129
F18	2012	2.1239	0.5975	0.0054	0.9369
F18	2013	2.1382	0.6683	0.0039	0.9372

). Following gas flare removal and blooming correction, these regression coefficients were used to adjust the raw DN values for NTL images of Africa. These steps were followed by Gaussian Process smoothing to produce the final intercalibrated time series.

3.1. Method Evaluation

3.1.1. Sum-of-Lights Index

Figure 2 and Figure 3 show SOL plotted for uncalibrated and IRQR (only) NTL series, respectively. Successful intercalibration is indicated by a relative convergence of SOL values in years where two satellite products are available. Another indication of successful intercalibration is a clear continuous growth trend in light values across the time series. Figure 3 illustrates how IRQR calibrated datasets for Africa visually comply with these criteria, despite noticeable discontinuity introduced by satellite F18.

When annual averaging was applied in addition to IRQR intercalibration, it resulted in a smoother SOL growth trend between most years, with exceptions occurring in the transition years between satellite F16 and F18 (2008–2010). However, when Gaussian process smoothing was applied using all intercalibrated satellite-year images, it yielded a smooth trajectory of continuous SOL growth (Figure 6).

3.1.2. Validation Using the Invariant Region

For procedural validation, the uncalibrated (UC), IRQR intercalibrated (IC), and GP smoothed time series were compared as predictors of the invariant region dataset: satellite-year F12-1999. The mean squared error was calculated for each of these test cases using DN values for Sicily (

Table 3. MSE for intercalibration test cases vs. F12-1999.

Case	UC	IC	GP
MSE a	22.76	18.22	11.38

^a Mean squared error.

).

Whereas conventional intercalibration exhibited considerably lower MSE than the uncalibrated, the Gaussian process approach was clearly the best predictor of the reference dataset with an MSE half that of the UC test case. While these results are not evidence of improved accuracy, they do substantiate the use of Gaussian process smoothing to produce refined NTL time series while observing the principles of the IRQR method.

3.2. Correlation of SOL with GDP and Urban Population

In terms of the relationships between the NTL layers (UC, IC and GP) and GDP and urban population, the UC layer showed the least correlation, while IC and GP case adjustments each incrementally increased the strength of the relationship. The regression results and root-mean-squared error (RMSE) values for continental Africa are listed in

Table 4. Coefficient of determination and RMSE for SOL vs. GDP and urban population.

Intercalibration Test Cases	GDP		Urban Population
	${\mathit{R}}^2$	RMSE a	${\mathit{R}}^2$	RMSE b
Uncalibrated (UC)	0.847	990,522	0.803	1,122,507
Intercalibrated (IC)	0.925	360,601	0.902	412,637
IC + Gaussian process (GP)	0.983	158,827	0.9997	22,241

^a Root-mean-squared error.

. The coefficients of determination indicate that the SOLs in the GP case have a stronger linear relationship with GDP than those from the IC case. Thus, when Gaussian process smoothing was applied, R² increased from 0.92 to 0.98 and from 0.90 to 0.9997 for GDP and urban population, respectively. Corresponding RMSE values also decreased dramatically. The high R² and relatively low RMSE value for the Gaussian process SOL vs. urban population clearly indicates the advantage of this enhanced intercalibration method. Figure 7 and Figure 8 graphically illustrate these improvements.

Regression analyses performed at the country level yielded similar results. Figure 9 and Figure 10 display regression results for these analyses. The GP test case is shown to perform notably better than the IC case in nearly all cases with positive with correlation coefficients ranging from 0.72 to 0.997 for GDP and 0.76 to 0.9996 for urban population. The Gambia and Eritrea were two exceptions where the correlation coefficient for the IC method exceeded that of our GP method in the case of GDP.

Other exceptions were the notably poor and negative correlations of SOL to both GDP and UP for Zimbabwe and the Central African Republic. Figure 11 and Figure 12 illustrate these atypical temporal patterns of SOL in relation to GDP and UP.

3.3. Urban Growth Analysis

The absolute and percent changes in lit and agglomerated area for the 2000–2013 time period have been calculated to model total settlement (urban and rural) and urban growth, respectively. These are shown in

Table 5. Absolute and percent change in lit and agglomerated area, 2000–2013.

Country	Total Area of Lit Pixels (km2)				Area of Urban Agglomerations (km2)
	2000	2013	Change a	% Chg	2000	2013	Change	% Chg
South Africa	202,098	240,581	38,483	19	156,608	192,501	35,893	23
Nigeria	68,470	96,054	27,584	40	41,870	58,744	16,874	40
Sudan	24,622	49,307	24,685	100	16,449	33,055	16,606	101
Angola	7086	21,845	14,759	208	3800	10,884	7084	186
Mozambique	5566	16,119	10,553	190	1439	6777	5338	371
Ethiopia	8888	17,866	8978	101	2129	5403	3274	154
Kenya	14,194	22,527	8333	59	8289	15,410	7121	86
Ghana	23,708	30,384	6676	28	15,134	21,451	6317	42
Tanzania	10,228	15,272	5044	49	3238	5662	2424	75
Congo, DR	6105	11,020	4915	81	2745	4489	1744	64
Cote d’Ivoire	39,350	43,443	4093	10	18,737	23,493	4756	25
Niger	3864	7804	3940	102	975	1710	735	75
Botswana	7946	11,869	3923	49	2134	4811	2677	125
Senegal	6226	10,083	3857	62	2790	4815	2025	73
Congo, R	2371	5800	3429	145	1622	2919	1297	80
Namibia	9611	12,960	3349	35	2692	4219	1527	57
Chad	1127	4393	3266	290	233	1180	947	406
Zambia	10,667	13,726	3059	29	5870	7455	1585	27
Burkina Faso	3086	5820	2734	89	741	1509	768	104
Mali	3444	6163	2719	79	803	1224	421	52
Gabon	3206	5372	2166	68	707	2335	1628	230
Benin	3354	5464	2110	63	1511	2930	1419	94
Cameroon	6672	8621	1949	29	2423	2705	282	12
Swaziland	5180	7060	1880	36	3818	6204	2386	62
Malawi	4819	6583	1764	37	1895	1482	−413	−22
Eq Guinea	158	1753	1595	1009	0	631	631	-
Uganda	4055	5604	1549	38	1834	2508	674	37
Liberia	490	2029	1539	314	252	510	258	102
Rwanda	694	1948	1254	181	346	905	559	162
Mauritania	1912	3070	1158	61	699	891	192	27
Lesotho	1562	2681	1119	72	344	1588	1244	362
Sierra Leone	371	1132	761	205	0	280	280	-
Togo	2431	3167	736	30	1164	1408	244	21
Somalia	1495	2210	715	48	0	614	614	-
Guinea	2275	2973	698	31	511	476	−35	−7
Gambia	445	1030	585	131	300	548	248	83
Madagascar	2931	3468	537	18	710	925	215	30
Burundi	501	961	460	92	275	398	123	45
Eritrea	1512	1938	426	28	417	506	89	21
Central Afr Rep	829	1175	346	42	266	266	0	0
Djibouti	408	677	269	66	0	340	340	-
Guinea Bissau	197	216	19	10	0	0	0	-
Zimbabwe	19,625	16,847	−2778	−14	10,270	9287	−983	−10

^a Sorted by absolute change in lit area, descending order.

and illustrated as bar graphs in Figure 13 and Figure 14. The red bars represent absolute change in lit area and the blue bars indicate change in agglomerated area. In general, the growth in total lit area per country exceeds that of agglomerations, though it is possible for growth in urban agglomerations to be greater than growth in total lit area. Although this seems counterintuitive, given that agglomerations have been filtered by size, merging of small urban clusters over time can produce such results. This can be illustrated with an example of growth analysis in Kenya’s Nairobi metropolitan area (Figure 15). Note that in Figure 15a the lit area in the year 2000 is derived from all cluster sizes, whereas in Figure 15b agglomerations in 2000 only include two clusters that exceed the 250 km² filtering threshold. In both cases, the yellow areas outside the dashed lines of the year 2000 polygons represent the area of absolute change. In this case, the urban/peri-urban communities surrounding Mount Kenya merged with greater Nairobi by 2013. Consequently, agglomerations increased by 3936 km², while lit area increased by only 2919 km². Countries displaying greater growth in agglomerated than lit area include Cote d’Ivoire, Swaziland, Lesotho, and Djibouti (Figure 13 and Figure 14).

4. Discussion

For the first time, we report here on the production of an intercalibrated open access NTL dataset spanning continental Africa 2000–2013 using a GP statistical approach. This improved dataset can be applied to a broad range of disciplines including public health, economic development, and environmental monitoring. While NTL data offer an opportunity to measure and map the human footprint, in its raw format, these data are difficult to interpret and can lead to spurious conclusions.

Gaussian process smoothing, the key enhancement of our intercalibration method, yielded a more intuitively smooth increase in SOL over Africa and was less noisy than that produced by using IRQR and annual averaging alone. While there is no gold standard short of calibration with known light sources as ground truth [38] against which to validate this approach, the SOL plots indicate a relatively effective intercalibration. However, what is achieved through IRQR intercalibration is not the elimination of errors, but their re-scaling such that they have the same magnitude across satellite-years. Through the use Gaussian process methods, independent temporal noise signals have been separated from latent functions in the annual NTL datasets. We have attempted to validate the GP method results against the premises of the conventional IRQR method. That is, the assumption that the reference dataset for the invariant region represents the NTL brightness across the time series. While invariably information is lost in the process, the overall result is an NTL time series for Africa that is standardized and comparable across both time and space.

The success of intercalibration with GP smoothing was also evaluated by comparing the resultant time series to known indicators of urbanization: GDP and UP. Improvements in the correlations found between the SOL and these indices when using the GP method provide further support for its use. When the same relationships are explored on a national basis, it provides sub-regional insights into urban and economic growth patterns, as well as the consequences of political and humanitarian events. While we have not compared GDP/UP figures to NTL values in countries outside Africa, we would expect similar relationships in these countries. The country based regression analysis for sub-Saharan Africa yielded generally positive results with only a few exceptions. The GP method performed notably better than the IC method in nearly all cases with correlation coefficients exceeding 0.9 for 80% and 90% of the countries for GDP and urban population, respectively. The exceptions were The Gambia and Eritrea, where positive correlation coefficients for the IC method exceeded that of our GP method for the GDP regression. For Eritrea, a partial explanation for this anomaly is the fact that GDP data for 2012–2013 are unavailable. However, also at the root of this data omission are a series of military disputes and border disputes with Ethiopia and Djibouti during our study period. In the case of The Gambia their economy, and therefore GDP, struggled between 2000 and 2004 and the unsmoothed IC dataset reflected this short term trend more accurately than the globally smoothed GP case.

Among the national statistics there were two countries with notably poor negative correlations to both GDP and UP: Zimbabwe and the Central African Republic. Notable is Zimbabwe’s unusual patterns with the SOL decreasing from 2000 through 2011 when it begins to recover. This pattern coincides with a disputed presidential and parliamentary election in 2008 preceded by periods of political unrest. The GDP trended slightly downward during the lead up to the election and then rose sharply thereafter. However, UP rose steadily from 2000 to 2013, even while SOL was decreasing. The Central African Republic also displays an unusual pattern, with relatively low SOL values, an urban population that rises steadily, and GDP that rises but drops abruptly in 2013 in response to a coup d’état. In general, anomalies such as these can be traced to the effects of insurgent warfare, political unrest, and humanitarian crises on nighttime lighting. Bennett and Smith [39] studied socioeconomic changes that caused reductions in NTL in post-Soviet Russia and war-torn Syria, while Li et al. [40] observed similar reductions as a result of the Islamic State of Iraq and Syria (ISIS) insurgency in Northern Iraq. Humanitarian crises often come in the form of drought, famine, earthquakes, and tropical storms. Gillespie et al. [41] examined the effects of tsunami damage in Sumatra (2004–2008) using NTL imagery. Several authors have suggested NTL based metrics for monitoring at risk populations. For instance, Coscieme et al. [42] propose an NTL based index of regional disparity and Li et al. [43] similarly suggest a nighttime light variation index (NLVI) to predict the risk of armed conflicts. All these insights underscore the potential for well calibrated NTL data as a tool for monitoring the regional effects and outcomes of such events.

Urban growth analysis using the improved NTL time series data indicate that the extent of urban growth appears to vary widely across African countries, with an overall trend of increasing areas of the continent being lit, particularly around urban agglomerations. We have described basic patterns of urban growth in Africa by computing changes in the area of lit NTL pixels and agglomerated pixels (>250 km²) on a national basis. The lit area serves as an indicator of overall growth of human settlements, whereas the agglomerations are meant to be a measure of large metropolitan areas, a proxy for urbanization.

Most sub-Saharan countries exhibited substantially greater growth in lit area than agglomerated lit area. Exceptions included Cote d’Ivoire, Swaziland, Lesotho, and Djibouti which all showed greater growth in agglomerated area. In Cote d’Ivoire, this may be due the effects of civil wars occurring in 2002 and 2011, with increased agglomerated area resulting from migration away from centers of conflict to urban areas. In the case of Swaziland and Lesotho, this is likely related to their proximity to South Africa which has the highest growth rates. In both cases, lit area growth was concentrated along borders with South Africa and internal major roadways. Thus, merging of smaller, patchy urban areas caused a relatively high increase in agglomerated area. In the case of Djibouti, most growth in lit area occurred in the city of Djibouti causing it to surpass the 250 km² threshold.

A variety of other anomalous patterns are expressed by countries with minimal growth in lit area many of which likely reflect human events, as previously discussed. Zimbabwe uniquely showed decreases in both lit and agglomerated area which were likely the effects of persistent political unrest during the study period. Malawi increased in lit area while decreasing its agglomerated area, apparently due to disaggregation of the city of Zomba metropolitan area. Finally, other notable anomalies are the Central African Republic and Guinea Bissau. The Central African Republic had no change in agglomerated area, while Guinea Bissau had no agglomerated area as of 2013 despite a modest 19% increase in lit area. In both of these situations, annual growth patterns may be attributed to their relatively sparse populations and the margin of error associated with the processed NTL data.

While we have demonstrated the utility of our enhanced approach to intercalibration of NTL imagery, for the purpose of this study we focused on imagery up to 2013. Continued growth in NTL applications hinge on new NTL data sources such as the Visible Infrared Imager Radiometer Suite (VIIRS) of the National Polar-Orbiting Operational Environmental Satellite System launched in 2011 [44,45,46]. Although nighttime lights imagery from VIIRS are superior to DMSP-OLS data in both spatial resolution (0.5 km) and dynamic range (14-bit), it is currently only available as monthly composites and only since 2014. Furthermore, VIIRS data processing methods are relatively unexplored, with relatively few publications to date that make use of the imagery. Thus, DMSP-OLS will likely remain the de facto standard for nighttime lights based investigations for a number of years. As part of this study, integration of VIIRS data with DMSP-OLS data was briefly explored as way to extend the time series to 2015. However, the relative complexity of data processing necessary to interface the two datasets was prohibitive and deemed beyond the scope of this study. Future research on data processing of VIIRS imagery and their integration with DMSP-OLS data is vital if NTL based research and application development is to continue. Toward this end, release of a VIIRS NTL time series of processed annual composites similar to DMSP Version 4 would be of great benefit to the scientific community.

5. Conclusions

The NTL dataset has been widely used during the past two decades with the number of related publications increasing dramatically as reported in the systematic literature review by Huang et al. [47]. With the release of the Version 4 DMSP/OLS Nighttime Lights Time Series dataset by NOAA in 2010, the publication rate increased further with 25 papers published in 2013 alone. Clearly, these numbers speak to both the unique worth of the dataset and the benefits of open access to high quality annual NTL composites. Numerous publications have proposed methods to correct well-documented calibration deficiencies in the version 4 composites, and here we add our own refined technique.

Here we report on the use of Gaussian process methods to estimate NTL latent functions free from noise signals prevalent in IRQR intercalibrated datasets. The method was validated by comparing uncalibrated, intercalibrated, and GP smoothed time series as predictors of the invariant region dataset. We have also demonstrated the value of the improved time series by investigating the established relationship of NTLs to GDP and urban populations, by describing basic patterns of urban growth in Africa, and considering its potential to detect and monitor environmental and humanitarian crises. Finally, the intercalibrated NTL time series (2000–2013) has been made available to global health researchers as well as the broader scientific community in the hopes of spawning continued growth in NTL based research (see Supplementary Materials).