Utilizing Geospatial Data for Assessing Energy Security: Mapping Small Solar Home Systems Using Unmanned Aerial Vehicles and Deep Learning

Abstract

Solar home systems (SHS), a cost-effective solution for rural communities far from the grid in developing countries, are small solar panels and associated equipment that provides power to a single household. A crucial resource for targeting further investment of public and private resources, as well as tracking the progress of universal electrification goals, is shared access to high-quality data on individual SHS installations including information such as location and power capacity. Though recent studies utilizing satellite imagery and machine learning to detect solar panels have emerged, they struggle to accurately locate many SHS due to limited image resolution (some small solar panels only occupy several pixels in satellite imagery). In this work, we explore the viability and cost-performance tradeoff of using automatic SHS detection on unmanned aerial vehicle (UAV) imagery as an alternative to satellite imagery. More specifically, we explore three questions: (i) what is the detection performance of SHS using drone imagery; (ii) how expensive is the drone data collection, compared to satellite imagery; and (iii) how well does drone-based SHS detection perform in real-world scenarios? To examine these questions, we collect and publicly-release a dataset of high-resolution drone imagery encompassing SHS imaged under a variety of real-world conditions and use this dataset and a dataset of imagery from Rwanda to evaluate the capabilities of deep learning models to recognize SHS, including those that are too small to be reliably recognized in satellite imagery. The results suggest that UAV imagery may be a viable alternative to identify very small SHS from perspectives of both detection accuracy and financial costs of data collection. UAV-based data collection may be a practical option for supporting electricity access planning strategies for achieving sustainable development goals and for monitoring the progress towards those goals.

1. Introduction

Energy access is a global issue. Sustainable development goal (SDG) 7.1, outlined by United Nations [1], proposed to achieve “access to affordable, reliable, sustainable, and modern energy for all” by 2030. However, as the latest progress report in 2020 [2] pointed out, extrapolating from the current progress towards electrification, there would still be 620 million people without basic access to electricity by 2030. Although the great majority of the global population has grid-connected electricity access, the high costs of traversing challenging topographies make grid extension to remote, rural communities less affordable or efficient [3].

In attempting to reach SDG 7.1, solar home systems (SHS) have become a promising solution as an alternative to grid extension [4]. SHS are solar-photovoltaic-based systems that provide electricity to individual homes. Compared to solar farms where large solar arrays are installed and generated power is transmitted and regulated by large power grids, a SHS (a solar panel and associated equipment that provides power to a single household) is typically a small solar panel fixed to the top of the household roof and not connected to the electric grid. Off-grid solar systems are often cost-effective solutions for rural communities far from the grid.

A crucial resource for targeting further investment of public and private resources, as well as tracking the progress of universal electrification goals, is shared access to high-quality data on individual SHS installations including information such as location and capacity. With this information, decision-makers can make more informed decisions about electrification options, such as grid extensions, mini/micro grids, and stand-alone systems [5]. Despite the importance of such data, unfortunately this information is often of limited availability or only accessible by expensive and time-consuming surveys [6] or incomplete self reports [7].

To address this widespread obstacle to energy access tracking and planning [8], generally three approaches to small SHS data gathering are available: (1) on-the-ground surveys (people going door-to-door to collect information), (2) information from retailers of solar panels or government agencies, or (3) remotely sensed data using overhead photography (satellites, aircrafts, or UAVs). Ground surveys are usually too expensive (this will be discussed in Section 6) and gathering data from governments or utilities is often hindered by either the fact that such data are not collected or that there is not the willingness nor the incentive to share the data. This leaves remote sensing approaches as a promising path to data collection. Previous studies have recognized that SHS are visible in overhead imagery and when remote sensing data sources are combined with machine learning tools, it may create a suitable approach to data collection for some SHS. Utilizing high-resolution satellite images, it was shown that medium to large installations of solar panels can be identified with high accuracy [9,10,11,12]. However, significantly smaller sizes of SHS are used in rural areas of low- and middle-income countries. The majority of the global population using off-grid solutions use SHS under 50 W [13], which typically occupies an area of 0.3 m${}^2$ [14]. This is a challenge even at the highest resolution of commercially available satellite imagery (typically around 0.3 m/pixel [15]). This is a major limitation of the collection of location and capacity data for very small SHS. Therefore, unmanned aerial vehicles (UAVs) offer a higher-resolution alternative to satellite platforms and are a potential alternative solution to filling this information gap. A more detailed discussion of the existing literature on solar panel detection using remote sensing data is presented in Section 2.

Contributions of This Work

In this paper, we critically analyze the viability of using drone-based aerial imagery to detect common, but physically small, energy infrastructure as an alternative to satellite imagery or manual surveys from both an assessment accuracy and cost effectiveness perspective. Our contributions closely follow the three research questions that we propose to answer: (i) what is the detection performance of SHS using UAV imagery; (ii) how expensive is the drone data collection, compared to satellite imagery; and (iii) how well does drone-based SHS detection perform in real-world scenarios?

For investigating (i), the detection performance of SHS using UAV imagery, as there exists no public dataset to directly answer this question, we develop a drone-based solar panel detection dataset covering various flying altitudes and flying speeds and train a deep learning detection algorithm to detect the SHS. We then evaluate the performance and robustness of this algorithm. For research question (ii), regarding the comparative cost of UAV based data collection, we conduct a cost/benefit analysis of UAV-based SHS data collection and compare it to common alternatives such as satellite imagery and aerial photography. For research question (iii) regarding the performance of UAV-based data analysis for SHS assessment under real-world conditions, we annotated and evaluated SHS detection performance on UAV imagery from Rwanda to quantify the performance of UAV imagery in a real-world setting.

The main contributions of this work are summarized as follows:

• The first publicly available dataset of UAV imagery of very small (less than 100 Watts) SHS (Section 3). We collected, annotated, and openly shared the first UAV-based very small solar panel dataset with precise ground sampling distance and flight altitude. The dataset contains 423 images, 60 videos and 2019 annotated solar panel instances. The dataset contains annotations for training object detection or segmentation models.
• Evaluating the robustness and detection performance of deep learning object detection for solar PV UAV data (Section 5). We evaluate the performance of SHS detection performance with a U-Net architecture with a pre-trained ResNet50 backbone. We controlled for the data collection resolution (or 1/altitude): sampling every 10 m of altitude across an interval from 50 m–120 m. We controlled for the dimension of panel size by using 5 diverse solar PV panel sizes
• Cost/benefit analysis of UAV- and satellite-based solar PV mapping (Section 6). We estimate a cost-performance curve for comparing remote sensing based data collection for both UAV and satellite systems for direct comparison. We demonstrate that using the highest resolution satellite imagery currently available, very small SHS are hardly detectable; thus, even the highest-resolution commercially available satellite imagery does not present a viable solution for assessment of very small (less than 100 Watt) solar panel deployments.
• Case study in Rwanda illustrating the potential of drone-based solar panel detection for very small SHS installations (Section 7). By applying our models to drone data collected in the field in Rwanda, we demonstrate an example of the practical performance of using UAV imagery for solar panel detection. Comparing the results to our experiments with data collected under controlled conditions, we identified the two largest obstacles to achieving improved performance are the resolution of the imagery and the diversity of the training data.

2. Related Work

Larger solar PV arrays, especially as compared to very small SHS, have been shown to be detectable in satellite imagery. Recent work demonstrates the potential of automatically mapping Solar PV arrays using remote sensing and machine learning from individual household SHS (5–10 kW [6,9,10,16,17]) to utility-scale (>10 MW [12,18]) installations, while no past studies have focused on smaller solar arrays that are being deployed for those transitioning to electricity access for the first time, which could be 100 W or less—an order of magnitude smaller than all past studies.

The earliest work on solar panel segmentation used traditional machine learning methods like support vector machines (SVM), decision trees and random forests, with manually-engineered features like mean, variance, textons, and local color statistics of pixels [6,9,19]. Yuan et al. [16] applied deep neural networks (DNN) for solar panel detection from aerial imagery. Advances in convolutional neural networks (CNNs) [20,21] in large scale image datasets like ImageNet [22] have also propelled the field of solar panel segmentation (i.e., pixel-wise classification) forward. Convolutional networks for image classification were the first type of CNNs to be used for a coarse form of solar panel segmentation by [11,16,23]. True segmentation CNNs for solar PV identification soon followed including SegNet [24] and activation map based methods [10]. U-net structures [25] were also quickly adopted in this field further increasing model detection performance [7,26].

UAVs have been used for solar PV monitoring and management on solar farms. However, this has typically been limited to situations where the locations of the solar PV is already known, such as solar panel monitoring in large solar farms. UAV-based solar array segmentation has been used for inspecting the arrays of solar farms, using texture features and a clustering algorithm for the analysis [27]. Other research on drone-based solar panel detection used a thermal camera to identify potential defects in solar arrays at the site [28,29]. There have also been techniques for monitoring solar PV farms using satellite imagery, such as tracking particulate matter deposition and effects on generation efficiency [30]; however, this is the exception since solar PV condition monitoring typically requires UAV data due to high image resolution requirements. In these settings, both thermal and optical cameras have been used. Thermal cameras are used in this setting to identify abnormal temperature profiles that may indicate malfunctioning solar arrays [31,32,33]. Optical UAV imagery has also been used to identify damaged or dust covered solar cells at solar farms [34,35,36,37,38]. None of the these UAV-based studies released their imagery datasets.

Access to energy at a larger scale has also been explored using night-time lights imagery. This use of 750 m-resolution data has been used extensively to generate estimates of which communities are grid-connected and even to explore the reliability of those grid connections [39,40,41]. This approach is not able to investigate the small-scale off-grid systems, however, as some of these SHS are less than a meter in diameter. In this work, we do not focus on identifying grid-connected communities, but are looking for communities and individual households that receive their power through off-grid solutions.

3. The SHS Drone Imagery Dataset

Drone datasets containing annotated images of solar panels, and especially very small (≤100 W) solar panels do not exist to our knowledge. We summarize the most relevant publicly-available UAV-based datasets in the appendix; however, none of these datasets contain solar panel annotations, making them unsuitable for training automated techniques to identify solar PV cells in UAV imagery.

Since no existing public data were available for our study, we collected a new dataset (DOI: https://doi.org/10.6084/m9.figshare.18093890) specifically for exploring the performance of UAVs (also know as drones) under controlled conditions for identifying very small solar PV. We describe our dataset, test sites, and equipment below. Below are our data design considerations:

• Adequate ground sampling distance (GSD) range and granularity. Due to variations in factors such as hardware and elevation change, the GSD of drone imagery can vary significantly in practice. Therefore, we want our dataset to contain imagery with a range of image GSDs (shown in

  Table 1. Details of the dataset collected. Altitude and corresponding GSD are listed. Img: Images. Vid: Videos. In total there are 423 images, 60 videos, and 2019 annotated solar panels.

  Altitude	GSD	# Img	# Vid	# Annotated PV
  50 m	1.7 cm	58	6	248
  60 m	2.1 cm	63	7	289
  70 m	2.5 cm	47	8	227
  80 m	2.8 cm	60	8	295
  90 m	3.2 cm	44	8	214
  100 m	3.5 cm	47	9	230
  110 m	3.9 cm	56	4	278
  120 m	4.3 cm	48	10	238

  ) that are sufficient to represent a variety of real-world conditions, as well as detect SHS.
• Diverse and representative solar panels. As solar panels can have different configurations affecting the visual appearance (polycrystal or monocrystal, size, aspect ratio), we chose our solar panels carefully so that they form a diverse and representative (in terms of power capacity) set (

  Table A1. Details of the solar panels used in the experiments. X-crystalline (cell composition): Polycrystalline or Monocrystalline, which have slightly different colors. L: Length. W: Width. T: Thickness.

  Brand	X-Crystalline	L (mm)	W (mm)	Aspect_Ratio	Area (dm${}^2$)	T (mm)	Power (W)	Voltage (V)
  ECO	Poly	520	365	1.43	19	18	25	18
  ECO	Mono	830	340	2.45	28.3	30	50	5
  Rich solar	Poly	624	360.8	1.73	22.6	25.4	30	12
  Newpowa	Poly	345	240	1.44	8.3	18	10	12
  Newpowa	Poly	910	675	1.35	61.5	30	100	12

  ) of actual solar panels that would be deployed in developing countries.
• Fixed camera angle of 90 degrees and different flying speeds: To investigate the robustness of solar panel detection as well as data collection cost (that is correlated with flying speed), we want our dataset to have more than one flight speed.

Data Collection Process

UAV. A DJI mini 2 was used to collect all our data except for the Rwanda case study (in which we used previously published data [42,43]). The DJI mini 2 UAV was selected due to its flexibility in operational mode, high camera quality, and low cost. It has 3 modes of flight, with a maximum speed of 16 m/s. It has the capability to hover nearly stationarily, resulting in high image quality in low wind conditions. It uses 1/2.3${}^{″}$ CMOS sensor (4 k × 3 k pixels) with a wide range of possible shutter speeds. It shoots 4 k video (3840 × 2160) up to 30 fps. We fixed the gimbal (angle of camera) to be vertically downward (90 degrees).

Solar panels. We purchased 5 solar panels in total with various physical sizes and PV materials to be representative of the types of SHS that are typically installed for off-grid solar projects in communities transitioning to electricity and include the low end of feasible use (around 10 W). More detailed specifications of our solar panels are presented in the appendix.

Flight plan. Imagery data were collected in the flight zone within Duke Forest in Durham, NC. Due to regulations of the federal aviation agency (FAA) under part 107, we were allowed to occupy class G airspace that includes altitudes lower than 120 m. Therefore, the height of all the data collected was capped at 120 m. Additionally, to prevent collisions with trees, we set the lower limit of our flight height to be 50 m.

Annotation. The data were manually annotated with polygons drawn around the solar panel by a human annotator. These polygons were converted to pixel maps for network training. All stationary images were labeled and a small subset of the videos were annotated.

We present examples of our collected dataset here for reference in Figure 1.

4. Post-Processing and Metrics

For the evaluation metrics, we use standard object-wise detection precision and recall, and summary metrics, as described below. As our models are U-Net-based segmentation models (code available at https://github.com/BensonRen/Drone_based_solar_PV_detection) that produce pixel-wise confidence scores, we aggregate the pixel-wise predictions into objects based on thresholding the pixel-wise confidence scores into binary images, grouping those binary images into groups of pixels (objects), and morphologically dilating those objects into groups of neighboring objects, a process illustrated in the Figure 2. The resulting set of objects and corresponding confidence scores then serve as an input to a scoring function that compares each detected object to ground truth to determine whether they are a true positive, false positive, or false negative object, (illustrated in appendix Figure A2). We consider a predicted object to be a true positive if its intersection over union (IoU) with a ground truth object is 0.2 or greater. Intersection over union computes ratio of the intersection of the area between two objects with the union of the area included in the two objects.

While all PR-curves can be found in the appendix, we present summary statistics including the maximum $F_1$ score and average precision (AP). $F_1$ is the harmonic mean of precision and recall and is frequently used as a measure of the accuracy of the detection/classification where an $F_1$ score of 1 implies perfect precision and perfect recall. Average precision is a summary statistic representing the area under the PR-curve. An average precision of 1 signifies perfect precision and recall as well. They are defined as follows:

$$\begin{array}{cc}\hfill {F_1}_{max}& =\underset{\tau }{max}2\times \frac{P\left(\tau \right)\times R\left(\tau \right)}{P\left(\tau \right)+R\left(\tau \right)}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill AP& ={\int }_0^{R_{max}}P\times dr\hfill \end{array}$$

(2)

Here, P is Precision, R is Recall, $R_{max}$ is the maximum recall that the model can achieve (usually less than 1 due to object-wise and IoU limitations), and $\tau$ is the operation confidence threshold for object-wise scoring that we sweep to produce the PR curve. The $F_1$ score uses the harmonic mean to penalize lower values in either precision or recall. For example, if a detector achieves 90% precision and 10% recall, the $F_1$ score is 18% (a simple mean would yield 50%).

5. Experiment #1: Solar Panel Detection Performance Using UAV Imagery

As we describe the three experiments included in this study, recall that we are investigating three goals: (i) the detection performance of SHS using UAV imagery and computer vision techniques; (ii) the cost of drone data collection compared to satellite imagery and other data collection approaches; and (iii) performance in a real-world scenario using data from Rwanda. This section investigates the first (i) of these three questions, and subsequent sections explore the remaining two.

The primary goal of this first set of experiments is to estimate the achievable accuracy of small SHS detection models when they are applied to UAV imagery. We also evaluate how their accuracy varies with respect to two operational settings: the speed of the UAV, and the ground sampling distance (or resolution) of the UAV’s imagery. We hypothesize that these two factors will have a significant impact on the achievable accuracy of recognition techniques. Low image resolution can reduce detection accuracy and higher UAV speed can result in motion blur in the imagery, which can also negatively impact detection accuracy. These two factors are likely to vary in practice, making them an important consideration for practitioners and researchers who wish to employ automated recognition models.

To estimate the accuracy of contemporary PV detection models on UAV imagery, we applied a proven object detection model called U-Net [25] to the UAV Solar PV dataset we created for this experiment. The U-Net has been employed in many recent studies involving object recognition in overhead imagery, including solar PV mapping [26]. Due to the relatively small size of our UAV dataset, we begin by further training our ImageNet-pretrained U-Net model on a satelliteimagery solar PV panel dataset [23]. We then fine-tune the model on our UAV data (more details about this in the appendix). We evaluated the performance of this fine-tuned detection model as we varied both the resolution of the imagery and the flight speed at which the imagery was collected. To evaluate the model’s performance, we split our data into separate training, validation, and testing sets, which is a widely-used approach to obtain unbiased generalization performance estimates of machine learning models [44]. Full details of the training and data handling procedure can be found in the Appendix A.

5.1. Detection Performance Comparison over Imagery Resolution

To investigate the performance as resolution varies, we train and test image detection models multiple times, varying the resolution of the image for each scenario. Note that we define image resolution as the GSD of the UAV imagery (other optics-focused definitions exist [45], but are less relevant for this study).

To evaluate the performance change due to changes in the resolution of the imagery, we fly our drone at various heights from 50 m to 120 m (the maximum legally allowed height in our jurisdiction), resulting in an image resolution range of 1.6 cm to 4.5 cm per pixel. We divide our flying height (altitude) into 20 m-interval groups for which we evaluate performance. We trained and tested each possible image resolution to evaluate the performance impact of flying at different altitudes (resulting in varying image resolutions).

Additionally, we compare the performance across UAV image resolution to common satellite imagery resolutions, which are a potential alternative approach to collecting data on very small scale solar PV. We simulate the highest commercially available satellite imagery resolution (30 cm) as well as high resolution aerial imagery resolution (7.5 cm and 15 cm) to provide a thorough comparison to UAV performance. To simulate these alternative image resolutions, we resize the high-resolution UAV imagery to the desired lower resolution (Details in Appendix A.3 list 3).

Results

Even at the highest flying altitude, as shown in Figure 3b, UAV-based solar PV detection performance had an Average Precision of 0.94 (at 4.5 cm ground sampling distance). While this high level of performance is reasonable given that the UAV resolution is high enough to capture multiple pixels of solar panel content for each SHS, these results massively outperformed satellite imagery resolutions. As the resolution decreases from our UAV imagery to satellite imagery, the detection performance drops monotonically. At 7.5 cm (high-resolution aerial imagery), performance remains high. However, at a resolution of 15 cm and above, the detection capability of our neural network degrades heavily with only about half of the panels being detected (the maximum recall is 0.6) and at the typical highest satellite resolution (30 cm), the performance drops to near zero. Therefore, we conclude that at the best commercially available satellite resolution (30 cm), very small SHS PV panels are not feasible to detect. UAVs offer a feasible alternative for small solar PV detection as does high resolution aerial imagery.

5.2. Detection Performance with Respect to Resolution Mismatch

As demonstrated in the previous section, very small solar PV detection performance depends greatly on resolution, but across UAV resolutions (1.7 cm to 4.3 cm per pixel in our data collection), performance was less variable. In practice, the resolution may change for multiple reasons. Elevation of the terrain may change, the slope of the landscape and angle of the image may change over the target area and we may fly UAVs at varying heights for data collection (also possible if collected by different operators). Therefore, the resolution may not always be identical. In this section, we investigate the impact of mismatch between training image resolution and test image resolution.

To accomplish this, we divide our flying height (altitude) into 20 m-interval groups for which we evaluate performance. We trained and tested all the possible pairwise combinations of imagery resolution to evaluate the performance impact of a mismatch between training and testing altitude.

5.2.1. Results

The results are shown in the Figure 4, where we can see the algorithm’s performance with different training and testing image resolution pairings. When the training resolution and test resolution matched, the average precision was 90%. This can be seen in the diagonal pattern (with the only exception in training of 1.6–2.2 cm ground sampling distance). The off-diagonal performance (when there was considerable mismatch between the training and test resolution) was generally lower in performance, illustrating the importance of having a similar resolution during model training and testing.

5.3. Solar Panel Detection Performance with Respect to Flight Speed

Apart from resolution, another important controllable variable for drone imagery operation is the speed at which to collect the imagery. The faster the drone flies, the shorter the time span, and hence the lower the cost of data collection. However, greater speed may also introduce additional noise into the data due to motion blur [46], impacts on the stability of the flight, reduced exposure time, and forcing the use of a higher ISO. Each of these may lower the performance of solar panel detection. Here, we aim to provide a controlled experiment to investigate the change in performance if we fly at different flight speeds and compare how flight speed impacts detection performance.

The UAV we used for these experiments, the DJI mini 2, has 2 operational flight speeds: “Normal” mode and “Sport” mode. Normal mode has an average speed of 8 ms, and Sport mode has an average speed of 14 s. Continuous videos were taken, and individual frames were cut from the video and labeled for evaluating the performance change when flight speed changes (models only operate on individual image frames rather than continuous video).

Results

Figure 5 shows the degradation in detection performance with respect to flying speed. From the plot, we can see that the detection performance does drop with respect to the flying speed as all performance statistics (AP, ${F_1}_{max}$ and $R_{max}$) are worse for sports mode than the normal mode. However, as the resolution differs, the performance drop monotonically increases as the resolution becomes lower. We also note that with the normal mode (slow speed) flying, the detection performance is kept the same comparing to stationary mode.

6. Experiment #2: Cost Analysis of UAV-Based Solar Panel Detection and Comparison to Satellite Data

6.1. Cost Analysis: Methods

Having demonstrated that UAVs can accurately identify very small solar PV panels under laboratory conditions, a remaining question is whether using UAVs for this purpose is cost-effective. In this section, we estimate the costs of using UAVs for small solar panel detection and compare the estimated costs with that of commercial satellite data and aerial photography. Although we present the operational trade-off of detection performance using small solar PV detection, these costs may be relevant to a variety of similar analyses of optical overhead UAV imagery data.

To estimate the cost of UAV mapping (in unit of $), we first identify two key characteristics of the final product: (1) image resolution and (2) size of the area imaged. Nearly all our cost estimates are a function of these two product specifications (e.g., cost per km${}^2$ at a resolution of 0.03 m). With a fixed resolution and total area, the total amount of work required (total flight time) can be determined. Although we split our cost estimation into five major categories listed in Figure 6, not all of them are dependent on the image resolution or size (such as legal costs). We made a few assumptions regarding operational parameters in order to provide representative cost estimates for a generic case:

• The UAV is operated five days each week, six hours per day (assuming an eight-hour work day, and allowing two hours for local transportation and drone site setup).
• Each UAV operator rents one car and one UAV; the upfront cost of the UAV is amortized over the expected useful life of the UAV.
• Total UAV image collection time is capped at three months, but multiple pilots (each with their own UAV) may be hired if necessary to complete the collection.
• UAV lifetime is assumed to be 800 flight hours (estimate from consultation with a UAV manufacturer).
• A sufficient quantity of UAV batteries is purchased for operating for a full day.
• The probability of inclement weather is fixed at 20%. and no operation would be carried out under those conditions.

Using these assumptions, we can translate total required flight time for a given data collection into total working hours, the number of pilots needed and the number of UAVs needed. Together, this information can be combined to estimate each individual cost type we enumerate in Figure 6.

We distinguish four categories of costs, as illustrated in Figure 6, for UAV data collection:

• Legal and permit: The legal and permitting cost of getting the credentials for flying in a certain country or region. As an example, in the US, although state laws may vary, at a federal level, flying for non-hobbyist purposes (class G airspace, below 120 m) requires the drone pilot to have a Part 107 permit, which requires payment of a fee as well as successful completion of a knowledge test. The legal and permitting costs are inherently location-dependent, and cost variation may be large.
• Transportation: The total transportation cost for the drone operator. For the purposes of our estimate here, we assume one drone pilot (thus, total data collection time is a linear function of area covered). Note that this category includes travel to and from the data collection location, which is assumed to include air travel, local car rental, car insurance, fuel costs, and (when the operational crew is foreign to the language) a translation service.
• Labor-related expenses: Umbrella category of all labor-related costs including wages and fringe benefits or overhead paid to the drone pilot, as well as boarding and hotel costs.
• Drone-related expenses: Umbrella category of all drone-related costs including purchase of the drone, batteries, and camera (if not included with the drone).

The details of the price assumptions are outlined in

Table 2. Table with detailed breakdown of the cost assumptions in each category for US-based operation. References embedded with hyperlinks. Note that nearly all of the unit prices in this chart vary greatly with the operational location of interest, and therefore anyone referencing this cost estimate framework should adjust according to local conditions and prices. A detailed calculation and explanation of our cost assumptions are given in the appendix and can be found at our code repository.

Category	Item	Unit Cost	Unit
Legal and permit	Part 107 certificate	$150 [47]	/pilot
	Pilot training for exam	$300	/pilot
	Drone registration fee	$5 [48]	/drone × year
Transportation	Car rental	$1700	/month
	Car insurance	$400	/month
	Fuel	$3 [49]	/gallon
	Flight ticket	$2000	/pilot
	Driver/translator	0 in US	/pilot
Labor related	Wage	$40	/hour × pilot
	Benefit	$20 [50]	/hour × pilot
	Hotel	$125	/night
Drone related	Drone	$27,000	/drone
	Camera	$0	/drone
	Battery	$3000	/drone
	Data storage	$130	/5 TB

. In each row, we specify the major cost item and the unit price. We put additional details in the appendix.

6.2. Cost Analysis: Result

We present our cost estimate in Figure 7. There were three types of drones used in [42] and we use “Ebee Plus” to estimate our total drone operation cost because it is able to map the largest area per flight (or per unit time) among the three. From Figure 7, we see that the relationship between the total cost and the total area of interest after some threshold is nearly linear (i.e., as the fixed cost is averaged out over a larger amount of area). We also see that the overall cost of using a UAV to map high-resolution imagery is non-trivial: mapping an area equivalent to the size of the Federal Capital Territory (7000–8000 km${}^2$) around and including Abuja, the capital area of Nigeria, would cost around 6 million dollars in total.

One of our assumptions of the price estimation is that the pilots are not within day-trip distance to the place of interest and need lodging during operation. Local pilots are also an option and are increasingly available, which would yield lodging and other travel related expense savings. At 0.03 m resolution, the lodging fee, at most represents about 15% of the total operation cost, which would not impact our conclusion or the relative ranking of UAV, satellite, or aerial imagery cost. More information regarding lodging fee is provided in the Appendix A.

Apart from the drone operation cost, the core question we aim to answer concerns the cost and performance trade-offs between using drone-based high-resolution imagery or satellite imagery for a given setting or application. Therefore, we combine these two elements together in Figure 3 for direct reference. Note that the price estimates for commercial offerings are estimated from online sources, government agency reports, and expert consultation, but we recognize these estimates; estimates which we believe are reasonable for the purpose of this study, but are certainly imperfect. The exact price would vary based on negotiated rates, changing business models, and on specific parameters of the desired data collection including the region of interest, business status (e.g., for nonprofit or for-profit use), etc. Currently, the highest resolution (0.3 m) commercially available satellite imagery is provided by private companies. We collected estimates of satellite imagery cost per unit area (USD per km${}^2$) and compared these with our estimated UAV average cost per km${}^2$ with the same resolution. Note: these costs estimates will vary based on negotiated rates and size of the imagery order.

From Figure 3, we draw several conclusions: (1) The satellite imagery costs are typically much lower than UAV imagery cost, although they also have significantly lower resolution, which results in extremely poor performance in very small SHS detection. (2) The unit price of UAV imagery is lower both when the total area increases sufficiently (to a limit where the average fixed cost per unit area falls to essentially zero, so that the relationship between total cost and coverage area becomes essentially linear) and when the resolution requirement drops. (3) The cost of UAV imagery is comparable to satellite imagery when operating over a large area (>50 km${}^2$) with lower image resolution (4–7.5 cm); which also achieves excellent performance according to the performance plot.

In addition to the cost and performance of satellites and UAVs, Figure 3 also provides the same information for one more approach: imagery collection from piloted aircraft. To estimate the costs of the piloted aircraft approach, we collected data from two government reports [51,52]), and also information on the Hexagon (HxGN) Imagery Program. That report indicates that the cost is $100–200 per km${}^2$ and the resolution ranges from 7.5 cm to 30 cm. However, we note that these cost and resolution figures are based on historical data collection in the US, conducted for government agencies and for a large coverage area. Therefore, the average costs may differ in a different setting, such as in South Asia or sub-Saharan Africa. The Hexagon Imagery Program’s price reflects only the archival imagery that was collected in previous piloted missions and is limited in geospatial and temporal coverage compared with satellite imagery.

For context, we also provide data points of cost for ground-based surveys. Since ground surveys are usually conducted one household unit at a time (rather than by one unit area, km${}^2$, as in aerial surveys), the costs are typically calculated per household. Researchers and organizations conducting such surveys [53,54,55] reported that ground survey costs varies widely between $20–$422 per household, depending on the level of detail of the survey. Comparing those estimates with UAV cost estimates, as long as the area of the drone survey has an average density of more than 5 households per km${}^2$, then UAV-based data collection would be cheaper than ground surveys (assuming at 3 cm GSD, 50 km${}^2$ in total area for the UAV data collection). For comparison, the average household density in Rwanda is around 120 households per km${}^2$ and over 90% of the population of lives in an area where household density is greater than 78 households per km${}^2$ (Details can be found in Appendix A.4) which is much higher than our 5 households per km${}^2$ price parity point for UAV and ground based surveys. Piloted flights (as shown in Figure 3) are also a potentially cost-effective data collection methodology and are also less expensive than ground surveys for regions with a density of 15 households per km${}^2$ or higher. Although from these estimates, UAV costs are significantly lower than ground surveys, ground surveys can provide a much larger variety of information such as economic and demographic information, which would be nearly impossible to collect from raw overhead imagery alone.

In conclusion, although satellite imagery are cheaper and logistically easier to access, satellite imagery simply cannot adequately capture the small solar PV as Section 5 shows. UAV imagery has the advantage of unmatched high resolution and therefore, even at somewhat higher costs, is far more reliable in very small SHS performance in the SHS detection task. While tasks other than small SHS would likely have different performance with respect to image resolution, the trend we show here may be similar for cost estimates relevant to other applications (such as agriculture [42], wildlife conservation [56], emergency response [57], and wildfire monitoring [58], among others). Additionally, as we found the biggest cost element is the human operator-related cost, which is indispensable currently due to the legal requirement of human-guided operation within the line of sight [59]. However, if automated UAV flights become possible (or did not require on-site operators), then the cost of drone-based operations would become significantly lower, potentially giving UAVs a large cost advantage compared to satellites.

The cost estimates provided here were collected regarding data collection in the US as the numerous cost estimates and underlying operational assumptions that go into such estimates are readily available. This may be an optimistic estimate when considering the heterogeneity of applications globally due to differences in operational expenses, transportation time, and local regulations. Additionally, weather conditions and flight performance at different altitudes [60] may affect the flight speed and hence impact costs. We encourage the use of the calculations presented in Appendix A.5 to create estimates for a specific data collection process.

7. Experiment #3: Case Study: Rwanda SHS Detection Using Drone Imagery

The last of our three questions was to evaluate how well UAV-based SHS detection performs in real-world scenarios. To test the feasibility and robustness of using UAV imagery to detect small SHS in rural areas, we conduct a real-life case study in Rwanda to detect the SHS that villagers in developing countries are currently using, using the UAV imagery provided by RTI [42,43]). As this imagery was collected from multiple rural agricultural areas in Rwanda representing different agroecological zones without any prior information on the prevalence or locations of SHS, the model performance achieved is expected to be more representative of the actual performance when this method is applied to a new location of interest. The drone was flown at a fixed height yielding a GSD of 0.03 m/pixel, which was also covered in the range of our controlled experiments above. We are also making our annotations of these Rwanda data public for reproducibility.

We manually labeled the solar panels in the Rwanda imagery. We found 214 solar panels in total and split them into a training set of 114 panels and validation set of 100 panels. We limited the fraction of images without SHS to 10% of the total available to maintain class balance. The dataset contains around 21,000 image patches of 512 × 512 pixels, of which 1% has solar panels. In total, the model required 2 days to train on the 10% background data and the 114 solar panels using an NVIDIA GTX2080. We trained 120 epochs to ensure convergence, which led to 0.25 million images passing through our network.

Case Study: Result

Figure 8 shows randomly selected examples of our model predictions. Qualitatively, the majority of the SHS were found with few missing SHS and false positives. Out of the nine solar panels present in the sample imagery, eight of them are correctly detected and one of them is missed in the example given. From the example, we can also see that the network can find panels reliably in two distinct roof types present in the Rwanda data.

Quantitatively, from the PR curve in Figure 9 we can see that the performance achieves a maximum $F_1$ score of 0.79, and average precision of 0.8 and maximum recall of around 0.9. By properly thresholding the confidence score, we can achieve a Recall of 0.89 (detecting 89% of the solar home systems) with precision of 41%. Lower precision means more false positives, but in a post-processing the detections could be manually reviewed quickly and most eliminated without minimal intervention. Alternatively, recall and be sacrificed for precision, reducing the number of false positives (potentially dramatically so), but sacrificing some recall, meaning that the number of solar panels that were correctly identified would decrease. This amounts to moving around the PR curve in Figure 9 to find the appropriate balance for a given application.

Although there is a performance gap between the lab controlled experiment and the Rwanda case study, it is encouraging to see that recall, the fraction of very small SHS successfully identified remains high in both cases, suggesting that UAV based data collection may be a viable approach for providing information to decision makers working in sustainable development.

While these results are encouraging, there remain several limitations or challenges around this application that include regulatory requirements [59] to maintain visual line of sight, which necessitates a trained drone operator; limitations in battery technology that bottlenecks the overall flight time of each mission [61]; and limited autonomous navigation and control systems [62]. More specifically in sub-Saharan African countries (where the majority of people without access to electricity are located), Washington [63] states that the lack of trained operators, lack of government regulations, and privacy concerns are the major challenges to be anticipated for drone technologies to be widely and safely applied to benefit wider communities. However, this has been changing in recent years as more countries in sub-Saharan Africa have developed standard processes for obtaining flight permits and local capacity for operating drones to collect and process imagery for analytical purposes has been expanding rapidly in many countries.

8. Conclusions

Small solar home systems that provide transitional electricity access to communities are too small for any commercially available satellite imagery to capture. We demonstrate that UAV imagery is a viable alternative to map these small solar home systems to provide critical information to stakeholders working to improve electrification in developing countries including the identification of potential markets and information to track progress to electrification sustainable development goals. Through controlled experiments examining the impact of altitude and speed, we tested the technological and financial viability of UAVs for this purpose. We investigated the robustness of drone-based small SHS detection with respect to both resolution (detection performance changes minimally within typical UAV operational altitude) and flying speed (detection performance drops with higher speed), estimated the cost of operation and found comparable cost with satellite imagery given a sufficiently large region to map. We also evaluated UAV small solar detection performance on a case study in Rwanda that demonstrated that drone-based small SHS detection is a viable approach to supply crucial information about local SHS conditions for energy access projects, successfully identifying nearly 90% of solar panels with moderate false positive rates that could be reduced through post-processing.

The evidence from this study suggest that UAVs are a technically viable and financially reasonable approach to collect data for small energy infrastructure like solar home systems. The information about the location and characteristics of small SHS in developing countries collected from drones may provide evidence for decision-making around energy access planning for reaching the SDG 7.1 of universal access to electricity by 2030.

Future works: There are at least two potential future directions of this work. The first is developing a larger, more diverse collection of UAV data on small energy system objects such as solar PV, diesel generators, electric water pumps for irrigation, and distribution poles and lines. The dataset we share controlled for a large number of possible experimental variables to form fundamental conclusions. A larger and more diverse set of imagery could provide an application-focused benchmark for future algorithm development. The second potential direction of this work is for a larger set of case studies, preferably with even more diverse geographies beyond the regions of Rwanda that were available for this work to explore algorithm robustness.