A Practical Methodology for Generating High-Resolution 3D Models of Open-Pit Slopes Using UAVs: Flight Path Planning and Optimization

Abstract

High-resolution terrain models of open-pit mine highwalls and benches are essential in developing new automated slope monitoring systems for operational optimization. This paper presents several contributions to the field of remote sensing in surface mines providing a practical framework for generating high-resolution images using low-trim Unmanned Aerial Vehicles (UAVs). First, a novel mobile application was developed for autonomous drone flights to follow mine terrain and capture high-resolution images of the mine surface. In this article, case study is presented showcasing the ability of developed software to import area terrain, plan the flight accordingly, and finally execute the area mapping mission autonomously. Next, to model the drone’s battery performance, empirical studies were conducted considering various flight scenarios. A multivariate linear regression model for drone power consumption was derived from experimental data. The model has also been validated using data from a test flight. Finally, a genetic algorithm for solving the problem of flight planning and optimization has been employed. The developed power consumption model was used as the fitness function in the genetic algorithm. The designed algorithm was then validated using simulation studies. It is shown that the offered path optimization can reduce the time and energy of high-resolution imagery missions by over 50%. The current work provides a practical framework for stability monitoring of open-pit highwalls while achieving required energy optimization and imagery performance.

1. Introduction

Unparalleled augmentation has been witnessed in the field of Unmanned Aerial Vehicles (UAVs) in the past decade. UAVs have been successfully incorporated into modern mining operations throughout the world. The increasing availability of cost-effective drones and skilled personnel have led to their use for multiple applications in open-pit mining operations. UAV technology is the perfect tool for mining companies, due to the vast and remote nature of operations and their hazardous environment. As a result, many companies have already been using drones for surveying, volumetric analyses, and visual inspection of inaccessible locations (e.g., Rio Tinto [1], BHP Billiton [2], Newmont [3], and Barrick [4]). Other applications, such as designed vs. actual survey of ramps and benches [5], detecting the height of safety berms, blast fragmentation analysis and determining the accuracy of drill holes [6], and vegetation [7], and surface moisture [8] monitoring have been presented in the literature. Completing all of these tasks with aerial missions can result in significant benefits for mining operations in terms of time and resource utilization; however, available technologies for using drone images by application of artificial intelligence, computer vision, and machine learning algorithms have not yet been developed to their highest capacities. This is due to the lack of sufficient resolution in acquired aerial images in most cases. For example, detection of tension cracks and monitoring their evolution remains a critical problem for geotechnical engineers at open-pit mines despite all of the advances in slope monitoring techniques. Current slope monitoring practices rely primarily on instrumentation, such as inclinometers, prisms, crack extensometers, or remote sensing technologies like LiDAR and slope stability radar (SSR); however, the instrumentation is an unsafe and laborious exercise and remote sensors like SSR are not able to directly visualize cracks regardless of being a state-of-the-art slope movement monitoring technology. Furthermore, capital and installation costs associated with an SSR system are unaffordable for small- to medium-scale mining operations [9]. It seems that UAV imaging would be an ideal tool for automation of this process; however, tension cracks are usually narrow geometrical features that are not detectable on low resolution images usually taken from a constant, high altitude (around 400 feet) by popular drones currently on the market. In this regard, the Mining Automation Lab at the University of Nevada, Reno conducted a project (supported by the National Institute for Occupational Safety and Health, NIOSH) to use UAVs for high-resolution imaging and to develop state-of-the-art machine learning techniques for automated identification and characterization of cracks. The technology can be used as an early-warning system that alerts geotechnical engineers about the location of new cracks, and subsequently the potential for related failures so that necessary safety steps can be executed to avoid them [10].

This paper describes a part of the project wherein a practical setup was developed to use popular, cost-effective drones for the systematic generation of high-resolution images and 3D models for large open-pit mines. The setup consists of a ground station mobile application that assists the user in designing an imaging mission. It generates a series of flying waypoints at a constant, short distance above the ground to take high-resolution images. Images are processed by photogrammetry software to generate three-dimensional (3D) models of the pit, which are used for detecting and monitoring tension cracks [11].

2. High-Resolution Imaging

2.1. Photogrammetry

The science of obtaining 3D information on physical world objects from their digital photographs is referred to as photogrammetry [12]. Mathematical relationships for flight planning and estimating parameters of the flight for taking overlapping photos of an area are discussed in the literature [13,14]. Photogrammetry has been a fundamental approach for generating required 3D models from overlapping images in many applications. A methodology for generating a high-resolution orthomosaic of an active landslide area using a micro UAV fitted with an image sensor was demonstrated by Torres et al. [15]. The 3D form of a scene can also be estimated from a series of 2D digital images using Structure from Motion (SfM) [16]. Lucieer et al. [17] provided an example showing the use of SfM where a 3D point cloud of topography in a real-world coordinate system was generated with an accuracy of 3 cm from overlapping UAV photographs of the terrain. In the field of agriculture, where financial and technical challenges of satellite sensors exist, photogrammetry is used to derive quantitative remote sensing models on crop stress and water spills using UAVs equipped with standard thermal and multispectral imaging sensors guided by autonomous GPS navigation [18].

Agisoft PhotoScan [19] is used in this project for photogrammetric processing of images and generating dense point clouds, digital elevation maps (DEM), geo-referenced orthomosaics, and contours. Processing steps for creating these items from overlapping images collected by UAVs are shown in Figure 1. A computer server equipped with eight NVIDIA Ti 2080 graphic cards, 256 GB RAM memory, dual 16-core 2.1 GHz CPUs, and 30 TB SSD storage is used in this research to process photogrammetry calculations in a timely manner.

Although photos are automatically geotagged by the UAV and contain latitude, longitude, and elevation of the point where the photo has been taken in its metadata, there are substantial errors in the geo-location information of images, due to inaccuracy of the drone’s GPS. These errors need to be minimized in order to create an accurate geo-referenced 3D model of the mine. Ground control points (large visual signs on the ground with surveyed coordinates) are used to improve the accuracy of generated models in a global coordinate system.

2.2. Flying Challenges in an Open-Pit Mine Environment

It is crucial to fly over a surface mine in an optimized way due to the mine’s huge scale and the limited capacity of batteries on board drones. During a flight mission, the UAV takes off from a starting point (called Home), visits and takes photos at locations (called waypoints), and returns Home. Waypoints are at a constant distance above the ground (e.g., 30 m for 1 cm/pixel resolution with DJI (Dà-Jiāng Innovations) Mavic drone) with sufficient forward and lateral overlapping among images (around 80%). Hence, thousands of images need to be taken in a typical mine mapping mission.

Because of the relatively low altitude of flights above the ground, the required resolution, and the fact that the terrain of an open-pit changes continuously, common sources of global terrain models, such as the Space Shuttle Radar Topography Mission (SRTM) [20], ASTER Global Digital Elevation Model [21], or JAXA’s Global ALOS 3D World [22], would not be applicable to develop a flight mission for an open-pit mine. Furthermore, these terrain models are of low resolution (30 m/pixel) and were generated many years ago. Two feasible options are installation of LiDAR range finder sensors on the drone or importing a previously generated high-quality terrain map. The latter is selected to maintain practicality of missions on simple drones like Mavic and Phantom. The imported terrain can be a mine Toe-Crest computer-aided design (CAD) model or a DEM resulting from photogrammetry.

Figure 2 depicts the developed procedure for generating high-resolution maps for an open-pit mine. There are two stages of flights in this process. First, an area mapping flight is conducted at the highest permitted altitude by designing a grid mission and using ground control points (GCPs). Then, photogrammetry is used to create the updated terrain model of the mine. Since the geometry of benches and walls constantly change in an open-pit mine, this stage needs to be repeated periodically. Second, the surface terrain obtained in the first stage is used as an input to fly the drone precisely at a required elevation above an area of interest to generate high-resolution models of that area. In both flight stages, each waypoint is defined based on the forward and side overlap required between images. Unlike the constant flight altitude in Stage 1, elevations of waypoints in Stage 2 are set at a constant distance above the mine terrain model, which is derived from Stage 1. This distance is determined based on the required ground resolution and camera specifications. Another difference between the two flights is that Stage 1 always follows a parallel path, while Stage 2 follows an optimized path that will be explained later in the paper.

2.3. Ground Station Mobile Application

Development of the operating procedure for the Android flight control system is explained in this section. While Android Studio V.3.1.4 has been used as the developing environment, DJI Android SDK [23] has been utilized for interacting with and sending flight commands to the UAV’s flight controller. Furthermore, Google Maps’ API platform [24] has been an interface for flight planning and navigation. Updated versions of all three kits have been implemented in JAVA.

DJI Mavic Pro (Figure 3) has been used as a test platform for conducting test flights using the application during the development and testing stages. It is a commercially available quadcopter fitted with an image sensor.

Table 1. DJI Mavic Pro specifications [25].

Flight Time	27 min	Image	4000 × 3000 px
Navigation	GPS + GLONASS	Weight	734 g
Battery	3830 mAh, 11.4 v	Charging Time	45 min
Max. Altitude	120 m	Max. Speed	40 mph

lists the main specifications for this UAV. It has a high-quality image sensor for capturing aerial imagery and videos, and it automatically geotags these images with the WGS 84 coordinate system, which aides in photogrammetric processing of data.

Figure 4 depicts the flight planning interface of the designed mobile application. It can import 3D terrain models generated using photogrammetry. As shown in Figure 4, the most up-to-date 3D model of the mine has been overlaid as a reference for the user on Google Maps, which initially did not have a trace of on-going mining operations. Subsequently, the user can plan and edit UAV mapping missions based on the imported terrain. Once flight parameters are input by the user, the application uploads the path to the drone’s flight controller, which will execute the mission and bring the drone back to its initial launch position. Once the drone lands, the mobile application automatically downloads captured images and uploads them to the project server where they will be processed for the next stages of the project.

The size ($L_x$, $L_y$,) of the ground image taken by the UAV flying at height h above the ground depends on h and the size of the image sensor, as shown in Figure 5 [13,14]. It can be computed as follows:

$$L_x=2h\tan \frac{\alpha }{2}$$

(1)

$$L_y=\frac{L_x}{\rho }$$

(2)

where $L_x$ is the width of the image on the ground, $L_y$ is the length of the image on the ground, $\alpha$ is the angle of view of the camera, $\rho$is the aspect ratio ($\rho =\frac{I_x}{I_y}$), $I_x$is the width of the image expressed in pixels, and $I_y$ is the length of the image expressed in pixels.

In addition, Ground Sampling Distance (GSD), which is the distance between centers of two pixels of an image measured on the ground is given by Equation (3).

$$GSD=\frac{L_x}{I_x}=\frac{2h\tan \frac{\alpha }{2}}{I_x }$$

(3)

From Equation (1), it can be concluded that the size of the image on the ground is directly proportional to the altitude of the UAV above the ground. Another important parameter is the proportion of overlapping between images along the length and width of the area. The industry-wide recommendation is to have 85% forward (length) overlap and 70% side (width) overlap [26]. The distance between two overlapping images along the length and the width of an area of interest (Figure 6) can be determined as:

$$a=d\times \left(1-\frac{q}{100}\right)$$

(4)

$$b=d\times \left(1-\frac{p}{100}\right)$$

(5)

where $a$ is the distance between two images along the width of the mapping area, $b$ is the distance between two images along the length of the mapping area, $d$ is the size of the image on the ground, $p$ is the percentage of forward overlap, and $q$ is the percentage of side overlap.

2.4. Validation of Images and Models

The developed application has been validated by conducting a case study at the East Keystone Trail Head near Reno, Nevada. The objective of this experiment is to validate the ability of the software to generate high-resolution terrain maps of the area by following the terrain at a constant altitude. The East Keystone trail is a suitable site for this experiment because it has a hilly terrain and is sparsely populated. GCPs were laid out over the approximately 0.09 km² area, as shown in Figure 7a to minimize errors in generated 3D models. X-marks are made with survey-grade spray paint that has a bright color like orange, as shown in Figure 7b. The size of each X-mark is 1 m × 1 m. A wooden peg was installed at the center of the X-mark to be used as a survey point. In order to easily spot GCPs, survey stakes with ribbons, as shown in Figure 7c, were placed near them. An integrated global navigation satellite system (GNSS) was used for surveying. This unit is comprised of a Trimble R8 as the receiver and a Trimble TSC3 as the data collector.

Placement and surveying of GCPs took around 1.5 h. Once this was completed, the Stage 1 flight was conducted over the area with a DJI Phantom 4 Pro UAV. Collected images from the flight were processed using PhotoScan to generate a point cloud, a DEM, and an orthomosaic of the area. The orthomosaic and the DEM were then imported into the designed mobile application to plan and execute the Stage 2 flight with the terrain-awareness feature enabled for an area of interest [27]. Characteristics for Stage 1 and Stage 2 flights are shown in

Table 2. Flight parameters for Stage 1 and Stage 2 flight test.

Parameters	Stage 1	Stage 2	Parameters	Stage 1	Stage 2
Altitude (AGL)	60 m	36.5 m	Forward Overlap	80	80
Area	0.0658 km2	0.0104 km2	Side Overlap	80	80
Max. Speed	7 m/s	5 m/s	Terrain-Awareness	No	Yes
Flight Time	15 min	7 min	No. of Images	119	84

, and the estimated UAV path for the Stage 2 flight is depicted by the red polyline in Figure 8. The altitude for the Stage 2 flight was set to a lower value of 36.5 m compared to 60 m for the Stage 1 flight to generate sub-millimeter resolution models. It can be observed in Figure 8 that the Stage 1 generated terrain map of the test site has been overlaid on Google Maps to aid the user in flight planning. Since the terrain-awareness feature was enabled, flight planning was based on altitude values derived from the generated DEM of the area. DJI Mavic Pro was used for this test flight. The Android application ran on an Asus Zenpad with 1.3 GHz CPU and 2 GB RAM.

It was observed during the Stage 2 flight that the UAV was able to follow the terrain by continuously maintaining an above ground altitude of 36.5 m, as shown in Figure 9. This means that the Mavic Pro was able to adjust its flying altitude autonomously based on the imported DEM. Without terrain-awareness, if the drone was flying at 36.5 m over the same area, it would crash into the hill. The benefit of imaging the terrain at lower altitude is noticeable in Figure 10. The resolution of the DEM is almost equal to the theoretical GSD when the flying altitude of the UAV was 36.5 m. Furthermore, both DEM and orthomosaic resolutions increased by more than 50% compared to those from the Stage 1 flight (Figure 10).

This is a significant advantage because the Mavic Pro UAV is able to generate higher resolution models even though its imaging sensor is smaller than that for the Phantom 4 Pro UAV (20 MP resolution). The resolution of 6.3 mm/pix is sufficient for detecting tension cracks and measuring drill hole accuracy from an orthomosaic. In this case study, the considered area was relatively smaller than that for an actual mine. It can be observed from Table 2 that if the area of interest increases, the number of images taken, and the flight time required to do so rises significantly. With less flight time available for each battery, it is a major challenge to manage UAV battery power consumption for mapping mining operations. For such missions, it is crucial to calculate the number of batteries required beforehand and also to charge batteries simultaneously during the mission using an on-field power source like a car battery. This problem is addressed in the next section by developing an optimization algorithm to map a greater area using a given UAV with the same amount of battery power or map the same area with less battery consumption.

3. Materials and Methods

An optimized flying mission can be defined as completing the flight with minimum power consumption and/or at the shortest travel distance, which can be formulated as a classical Traveling Salesman Problem (TSP). In a TSP, the salesman has to visit all cities (represented by nodes on a graph) in a tour such that each city is visited exactly once and the total distance traveled is minimized [28]. This is a challenging, nondeterministic, polynomial-time problem, which can be solved by different approaches, such as simulated annealing [29], neural networks [30], and Tabu search [31]. Integer linear programming methods have also been proposed with ways to eliminate sub tours [32]; however, integer programming is not a suitable choice when the number of cities increases to more than one hundred. An application of a genetic algorithm for solving a 3D variation of the TSP has been presented by Meneses et al. [33].

In general, the speed of reaching a solution can be increased by using metaheuristics for estimating the initial solution in cases with evolutionary algorithms. A 3D path planning approach using genetic algorithms for multiple drones has been presented by Ergezer and Leblebicioglu [34]. They proposed a solution that plans drone flights through some desired areas while avoiding other undesired areas. Besada-Portas et al. [35] designed an evolutionary trajectory for multiple UAVs in a military environment that can work online, as well as offline. It includes constraints based on a drone’s dynamics and the environment with the goal of minimizing the risk to the UAV from enemy missiles. Pehlivanoglu and Hacioglu [36] proposed a vibrational mutation operator designed to improve the speed of path planning using a genetic algorithm. The advantage of such an algorithm is that when the solution is unstable/jammed in local minima, it helps to remove the repetitive iteration.

3.1. Experimental Study

A power consumption model for the UAV is necessary to optimize drone battery performance. Experiments were conducted to study battery performance when the drone flies in different directions and encounters varying wind speeds.

The power consumption model for a UAV can be calculated in one of two ways, known as White-box model and Black-box model approaches. A microscopic behavior model for a drone can be derived by conducting a comprehensive study of motor performance, aerodynamic environment, and battery systems. Such an approach is called the White-box model approach. A significant number of aerodynamic parameters relating to the body of the drone and sophisticated experimental setups are required in this method, and the method is specific/limited to the particular drone [37]. In the Black-box model, a multivariate regression model for drone power consumption is derived by conducting empirical studies [38]. This model is more flexible and acceptable for flight mission optimization. Moreover, parameters in this approach are easier to measure; hence, it is a more desirable approach. Since most mining companies use commercial drones, this approach is more suitable for mining applications because these drones do not come with development kits [39].

In this project, a statistical approach was used for deriving the energy consumption model for the DJI Mavic Pro. In a UAV, powering motors to lift the UAV’s body consumes a significant portion of total available energy. The rest is consumed in controlling the UAV’s motion in different directions. An experimental analysis was performed to study on-board battery power consumption during the drone’s movements in different directions while encountering varying wind speeds. A drone’s movements can be categorized as hovering, horizontal, and vertical motion. Each drone flight is a combination of these three motion types. To measure the rate of power consumption during these motions, multiple experiments were conducted. In each experiment, the drone was programmed to ascend and descent repeatedly at its maximum speed in a vertical direction. Then, it is moved horizontally at maximum speed without altering its altitude. DJI Mavic Pro was used as a test platform for these experiments. The UAV’s speed relative to the ground in vertical and horizontal directions, battery power, and barometers readings are recorded for each experiment from telemetry data. Figure 11 depicts recorded data from one of the above experiments for the DJI Mavic Pro test drone. Barometer readings show the drone’s altitude above the ground. Accelerations in both vertical and horizontal directions were also calculated from time series data. During vertical movements, the drone deviated slightly from a straight path, due to errors in GPS modules. Similarly, during horizontal movements, the UAV was not moving at an exact constant altitude, due to barometer errors. However, the average of these deviations was less than 0.1 m, which is considered negligible.

It was observed that power consumption steadily increases as the drone moves in an upward direction and decreases as it descends. GPS errors also generate minimal horizontal speed data during vertical motion, which have been filtered. It was observed that power consumption is relatively stable during horizontal motion at a constant altitude. The hovering motion has been considered by recording power consumption between these horizontal and vertical motions.

Powering drones is similar to power electric ground vehicles, but being aerial, they face some distinctive challenges. To elaborate, the wind is a major environmental factor that affects power consumption of the Mavic Pro during experiments. Therefore, another set of experiments was designed to model and measure battery performance when the drone flies in the same direction as the wind and in directions opposite wind. Wind speed and direction were recorded with a wind speed meter. The frequency of wind speed measurements was 2 s. Flights were made at a local park, but on different days to consider different wind speeds and to make the study more comprehensive. The drone was made to fly closer to the ground to minimize errors in wind measurements, due to elevation differences between the wind speed meter and the drone. The average wind speed of each flight has been used as a representative speed for the entire flight. Experiments were conducted for three wind speed cases—17.28 km/h, 18 km/h, and 19.8 km/h.

It can be observed in Figure 12 that the DJI Mavic Pro’s power consumption increases when it is flying in the direction opposite to the wind (i.e., with a headwind) and decreases when it is flying in the same direction as the wind (i.e., with a tailwind); however, the relationship is not linear. Aerodynamics confirm that flying with a headwind decreases power consumption, due to an increase in relative flow over propellers, and hence, an increase in a translational lift [40]. On the other hand, at higher wind speeds, aerodynamic drag can outweigh translational lift [39]. During the experiments described above, when the drone was flying with a headwind at its maximum speed, more drag is created than translational lift resulting in more power consumption.

3.2. Regression Model of Power Consumption for the DJI Mavic Pro

Assuming a constant wind speed at all locations of the experiment, a multivariate linear regression model (Equation (6)) was derived using altitude values, horizontal and vertical speeds, horizontal and vertical accelerations collected from the empirical studies conducted in the previous section. The coefficients of the model were estimated by a standard regression method. Estimated battery power consumption $P$, in Watts, is given by:

$$\begin{array}{cc}P=103.67+& {\left[\begin{array}{c}0.9\\ 0.048\\ 0.204\\ 0.128\end{array}\right]}^T \left[\begin{array}{c}{v}_{xy}\\ v_{xy}^2\\ v_{xy}.a_{xy}\\ v_{xy}.wind\end{array}\right]+{\left[\begin{array}{c}-0.764\\ 0.006\\ -0.505\\ 0.034\end{array}\right]}^T \left[\begin{array}{c}{a}_{xy}\\ a_{xy}^2\\ a_{xy}.a_z\\ a_{xy}.wind\end{array}\right]\\ & +{\left[\begin{array}{c}-2.54\\ -2.359\\ 1.167\\ 1.987\end{array}\right]}^T \left[\begin{array}{c}{v}_z\\ v_z^2\\ v_z.a_z\\ v_z.wind\end{array}\right]+{\left[\begin{array}{c}-2.493\\ -0.016\\ -0.075\\ -0.08\end{array}\right]}^T \left[\begin{array}{c}{a}_z\\ a_z^2\\ wind\\ win{d}^2\end{array}\right]\end{array}$$

(6)

where $v_{xy}$ and $a_{xy}$ are magnitudes of speed and acceleration vectors in the horizontal direction, respectively; and $v_z$ and $a_z$ are magnitudes of speed and acceleration vectors in the vertical direction, respectively. The wind vector in the horizontal direction is denoted by$wind$. If wind direction and speed are considered constant, total energy consumed within a flight with duration T can be estimated by P × T.

3.3. Model Validation

The model presented in Equation (6) was validated by conducting a new set of experiments in which the same Mavic Pro was programmed to fly in both vertical and horizontal directions under different wind conditions. Wind speed during this flight was 5.76 km/h. It was observed that the given model was able to capture power consumption trends with respect to wind speed and direction of movement, as shown in Figure 13a. The average error of estimation for power consumption is 0.7% with a standard deviation of 0.11. Figure 13b depicts total energy consumption for the flight, as the ultimate goal is to optimize the entire flight of the drone. It was observed that the error in estimating total energy consumption at the end of the flight is 0.6%. These low levels of error in estimating power and energy consumption show good accuracy of the power consumption model for the drone allowing the model to be used to determine which direction would consume relatively more battery power and not to predict actual power consumption values for the given UAV while flying. This approach can also be repeated for any other commercially available drone.

3.4. Path Optimization

The given UAV flight path planning and optimization problem is such that the UAV has to visit each waypoint to capture a picture exactly once and return to the starting point with the minimum trip cost (i.e., lowest power consumption). As described earlier, this is a classic traveling salesman problem (TSP), and the literature contains numerous genetic algorithm solutions.

Genetic algorithms (GAs) are based on Darwin’s theory of evolution, which states that the survival of an organism is defined by the “survival of the fittest” rule [41]. The energy consumption model derived in the previous section has been used as a fitness function in the given optimization problem. Figure 14 presents the flow of the proposed algorithm, whereas its main components are discussed in the next sections. The objective of the algorithm is to determine a UAV flight path with the least energy consumption, which also fulfills constraints of the TSP.

3.4.1. Fitness Function

The flight mission consists of a series of waypoints, and the UAV has been outfitted with a camera at the bottom to take pictures at each point in the flight. If i − 1, i, and i + 1 are three consecutive points in a proposed flight path, then the fitness of that path can be determined by Equation (7).

$$\mathrm{Fitness} \mathrm{Function}={\sum }_{i=2}^{n-1}{E}_{i,i+1}+\alpha {\theta }_{i-1,i,i+1}$$

(7)

where $E_{i,i+1}$ is the energy consumed by the UAV while moving from waypoint i to waypoint i + 1, and ${\theta }_{i-1,i,i+1}$ is the vector angle between path segments (i − 1, i) and (i, i + 1). Preliminary studies proposed that if the path has a significant number of sharp turns, then the drone consumes more time and consequently more energy in changing its direction. Therefore, α has been included as the penalty for sharp turns between two path segments. Its value can be increased if sharp turns are not required. The fitness of a route is being interpreted in an inverse way such that routes with the least fitness values are better than those with higher values. The aim is to select individual routes that consume less power and have a smaller amount of sharp turns.

3.4.2. Chromosome Structure

Each chromosome has been defined by a series of waypoints in which each waypoint has a latitude, longitude, and elevation in the World Geodetic System (WGS 84). Such representation is called Path Representation and possibly the most natural representation of a tour [42]. This representation is also more suitable because it facilitates the use of many popular genetic operators like partially mapped crossover (PMX), cycle crossover, order crossover, and genetic edge recombination crossover (ER). Each chromosome should contain all points in it so that each point is visited by the UAV. Furthermore, each waypoint should occur in a chromosome exactly once so that the UAV would visit each point only once. This chromosome structure has been implemented such that the drone, after visiting all waypoints, will return to its initial start point. Therefore, the first and the last genes in every chromosome should be entered the same depending on the location selected as a start point. If waypoint i is at the j-th position in the chromosome, then waypoint i is the j-th point to be visited.

4. Results

A case study was developed by deriving centerlines of benches for an open-pit mine in Nevada. Mine bench centerlines were manually extracted from the mine’s point cloud using PhotoScan software. These centerlines were then elevated 36.75 m above the ground as the required image resolution is 1.27 cm. To ensure the minimum forward overlap of 80% between successive images along the bench, the maximum distance between two consecutive points was restricted to 7.6 m (using Equations (1)–(5)). From these centerlines, two different regions (R1 and R2) were selected and optimized UAV flight paths for these regions were determined, as shown in Figure 15b and Figure 16b, using the designed GA in Matlab [43].

The maximum number of iterations was set at 10,000 and mutation operators implemented were swap, flip and slide. Using Equations (1)–(5), parallel paths for taking overlapping images with 80% forward and side overlap were determined for R1 and R2, as shown in Figure 15a and Figure 16a. The number of waypoints calculated in parallel and optimized missions for region R1 were 70 and 80, respectively; and for region R2, were 63 and 81, respectively.

Flight simulations for parallel and optimized paths were performed using the DJI assistant 2 software, which is the provided flight simulator for Mavic Pro [23]. Once both optimized and parallel missions have been designed, the UAV could be connected to the simulator. Initial and final battery readings were recorded from telemetry data once the drone has landed. The time of flight was determined as the difference between the time the mission started and time the drone landed back at the take-off point. The simulator is running on a Windows PC with 16 GB of RAM.

From Figure 17, it can be observed that battery percent and mission time for both regions are 50% less for the optimized path compared to the parallel path. Moreover, in the case of the optimized path, the UAV is programed to fly exactly over benches to image only the area of interest (horizontal bench surface in this case, with a final aim to monitor tension cracks in the next step of the research) saving both time and battery power. It can be seen that total battery power consumed in a particular UAV flight is directly proportional to flight time. This simulation study validates the proposed UAV flight path optimization algorithm by determining UAV paths for high-resolution imaging of an area that consume less battery power compared to a conventional parallel path.

5. Discussion

In this study, a methodology has been presented to generate high-resolution 3D models of open-pit slopes using low cost DJI drones in an optimized manner. These drones are widely used in the Earth science industry [44,45,46,47]. Methods presented in this paper have been validated using experimental and simulation studies. A comparison was performed between the resolution of 3D models generated using Stage 1 (constant elevation) and Stage 2 (proposed method) flights. The resolution of DEMs and orthomosaics generated from Stage 2 were double compared to Stage 1 3D models. Furthermore, all autonomous UAV flights were performed safely without any accidents. With this validation study, it can be concluded that the proposed software can generate millimeter resolution 3D models of hazardous inaccessible open-pit slopes without any risks to personnel who are responsible for the surveys and measurements to obtain multiple parameters of mine slopes (For an extensive list of influencing factors, see, e.g., Zare Naghadehi et al. [48,49]). Li and Ling [50] have presented an approach for modifying a consumer drone, such as DJI, by fitting it with an external computer equipped with a camera and Wi-Fi link; however, in this approach, the flight of the drone could not be controlled by the external computer. The developed application is significantly beneficial from the point of view of its application in a real mining environment. It can not only plan and execute flights autonomously for any DJI drone, but also collect high-resolution images of mine terrain with a low-resolution camera sensor. The generation of 3D model after the Stage 1 flight using photogrammetry is a time-consuming process which in case of large open-pit mines could take days to complete. However, this is a one-time step because the same 3D model could be used for several Stage 2 flights in future unless the terrain is changed significantly. It is also difficult to use the application in case of high winds weather conditions. It is advisable to conduct drone flights when the wind speed is around 2–4 m/s. Quadcopters have a finite battery time [51], which limits their operation in large open-pit mines for achieving centimeter-scale accuracy. The battery power consumption of the UAV has been studied and modeled using several experimental studies. For a test flight, model-predicted drone power consumption was compared with actual on-board battery power consumption. The average error of estimation was found to be 0.7%, which validates the drone power consumption model. The presented methodology can also be repeated for determining the power consumption model of any other consumer drone.

Furthermore, a genetic algorithm (GA) has been used to compute optimized UAV flights for high-resolution imaging of open-pit slopes using the battery power consumption model. A simulation study was conducted to test battery power and time consumed by UAV flight paths computed by the GA for two different regions in a mine. Results showed that the proposed power consumption model and GA-computed UAV flights consumed 50% less time and battery power as compared to normal parallel flights for both regions. It can be seen in Figure 11, that the UAV consumes more energy when flying in a vertical direction than in a horizontal direction. Optimum paths for regions R1 and R2 (Figure 15b and Figure 16b) have significantly greater portions of horizontal segments as compare to vertical segments. Moreover, these paths reduce the number of turns the UAV takes in a flight mission. These two factors are the main reasons for lower flight time and battery power consumption in optimized paths. These results are similar to those presented in a recent paper [52] that used GAs to estimate energy-efficient UAV flight paths. The optimized path with more horizontal segments and fewer turns is more suitable for open-pit mines—where only the horizontal part of each bench is imaged sequentially. These paths can be computed for detecting and monitoring tension cracks on mine benches or for measuring the positional accuracy of drill holes on a production bench using a drone. The centerline of benches in an open-pit mine can be automatically extracted from the DEM, as shown in Winkelmaier et al. [11]. Wind speed during simulation studies was 0 m/s, and hence, not considered, although speed and direction of the wind are parameters in Equation (6). The optimal flight plan would certainly change depending on wind conditions [39]. These study results show that high-resolution imaging using UAVs in open-pit mines can be optimized using battery power consumption modeling and the proposed genetic algorithm.

6. Conclusions

In this article a high-resolution imaging setup based on low-trim UAVs, with the aim to monitor stability conditions of open-pit highwalls, was studied. A ground station mobile application was designed using Android Studio, Google Maps API, and DJI Android SDK for generating high-resolution terrain maps of open-pit mines. The application was able to plan and execute area mapping operations for remote surveying of any area with UAVs. The application was also able import 3D terrain models generated from photogrammetric software. A terrain-awareness feature was incorporated into the application to maneuver the UAV over a non-planar terrain. This feature enables the drone to fly at altitudes as low as 10 m to capture millimeter-level resolution images of an area. It is able to accomplish this task with two flight stages. Stage 1 flight was a traditional high-altitude mapping operation, which outputs 3D models of the area. Stage 2 flight was planned using these models to adjust the drone’s flying altitude based on ground elevation. The entire method was been tested and validated at a local test site, in a case study that generated 3D models with 6.3 mm/px resolution.

An empirical study was performed to model battery performance of the test UAV considering various flight circumstances. A linear regression model for the power consumption of the drone was derived and validated for a test scenario. Furthermore, an optimization algorithm was proposed to solve the problem of path planning, which resulted in a path that consumes less battery power. A case study, based on an open-pit mine in Nevada, was presented. The algorithm was tested by performing simulation studies to determine the optimum path for high-resolution imaging of benches at the host mine. It is safe to conclude that the proposed algorithm can generate tours that consume approximately 50% less battery power and time for a given area.