Mapping of Potential Fuel Regions Using Uncrewed Aerial Vehicles for Wildfire Prevention

Abstract

This paper presents a comprehensive forest mapping system using a customized drone payload equipped with Light Detection and Ranging (LiDAR), cameras, a Global Navigation Satellite System (GNSS), and Inertial Measurement Unit (IMU) sensors. The goal is to develop an efficient solution for collecting accurate forest data in dynamic environments and to highlight potential wildfire regions of interest to support precise forest management and conservation on the ground. Our paper provides a detailed description of the hardware and software components of the system, covering sensor synchronization, data acquisition, and processing. The overall system implements simultaneous localization and mapping (SLAM) techniques, particularly Fast LiDAR Inertial Odometry with Scan Context (FASTLIO-SC), and LiDAR Inertial Odometry Smoothing and Mapping (LIOSAM), for accurate odometry estimation and map generation. We also integrate a fuel mapping representation based on one of the models, used by the United States Secretary of Agriculture (USDA) to classify fire behavior, into the system using semantic segmentation, LiDAR camera registration, and odometry as inputs. Real-time representation of fuel properties is achieved through a lightweight map data structure at 4 Hz. The research results demonstrate the effectiveness and reliability of the proposed system and show that it can provide accurate forest data collection, accurate pose estimation, and comprehensive fuel mapping with precision values for the main segmented classes above 85%. Qualitative evaluations suggest the system’s capabilities and highlight its potential to improve forest management and conservation efforts. In summary, this study presents a versatile forest mapping system that provides accurate forest data for effective management.

1. Introduction

The number of uncontrolled wildfires around the globe is increasing due to a rise in global temperature and extreme weather brought on by climate changes in recent years [1,2]. Longer periods of high temperatures lead to drier seasons and the accumulation of combustible material in the forests. These conditions yield an increase in uncontrolled and harmful wildfires which can be worsened by strong winds or thunderstorms. Without active and stringent measures to monitor and manage forest regions, it can be difficult to prevent and contain uncontrolled wildfires and the resulting irreversible damage. This is an especially challenging undertaking due to the vast size of typical forests, the danger posed, and the demand for highly intensive labor to fight fires or execute preventive maintenance actions.

Information collected using remote sensing methods (e.g., satellites or aircraft) can accurately monitor forestry environments while reducing labor and operational costs [3,4]. In particular, uncrewed aerial vehicles (UAVs) have gained attention in forestry research and operations (among other industrial applications) since they are easily deployable, provide a time-flexible operation, and can produce excellent spatial resolution measurements with different sensing modalities (e.g., visual, thermal, spectral or laser range) [5]. UAVs have garnered interest as a promising remote sensing modality due to their accessibility, rapid maneuverability, operational range, low operational costs, and ability to provide high-resolution data. Environmental reconstruction of sensor data collected from UAVs can help create higher fidelity virtual copies of the environment (digital twins), holding the information that may be beneficial for inventorying and mapping purposes. UAVs are also especially suitable for assisting humans when working in large dangerous areas where access is limited. These characteristics have made researchers put significant effort towards using UAV-based solutions to aid in fire fighting tasks [6,7], scout of fire detection [5,8] or fire behavior analysis and prediction [9].

A promising approach is to mitigate wildfires by reducing the accumulation of highly combustible biomass ranging from smaller biomass, such as dried pine needles, grasses, and small twigs, to larger wood pieces. Robotic forestry maintenance can play a key role in prevention tasks, especially when teaming aerial and ground vehicles. Recently, researchers have proposed and investigated a collaborative system in which a UAV will be used to survey the environment to generate a multi-layer map. This map comprises information including traversability and areas with an accumulation of combustible material. An Uncrewed ground vehicle (UGV) will receive this information and remove the highly combustible material from the forest [10].

The integration of fuel mapping plays a vital role in ensuring the quality of forest management and the conservation of local flora and fauna. By accurately identifying and categorizing flammable materials based on the United States Department of Agriculture (USDA)’s fire behavior fuel model [11] (the Anderson model), we can improve the precision of heavy-duty machines used for mulching and minimize the risk of false positives [12]. This comprehensive approach, encompassing geometry, semantics, and temporal information, allows us to create a global map that captures the different fuel models of grasslands, shrublands, timber, and slash areas. With this valuable knowledge, we can implement targeted strategies and conservation to protect our forests and promote sustainable practices.

This study seeks to expand on previous works [13,14] by developing a perception system for UAVs in wildfire prevention efforts. To enable the implementation of such a system, we have designed a custom payload that contains different sensing modalities and a framework that includes simultaneous localization and mapping (SLAM), semantic segmentation, and fuel mapping. In our work, we have extensively tested our system on two study sites and validated our SLAM implementations and semantic mapping with qualitative and quantitative results.

2. Related Work

Developments in remote sensing technology allow conditions and changes over large geographic areas to be monitored and analyzed, at a given time or period. Remote sensing has been useful for forest wildfire applications and has been extensively studied. Previous efforts have successfully applied remote sensing, in the form of satellite imagery, in all stages of forest fire efforts, such as mapping efforts for fuel type [15,16,17], burned area [18,19], vegetation recovery [20], assessments of fire risk [21,22] and fire/burn severity [23,24], and detection of fires [25,26].

In the agricultural domain, UAVs have been used to map diseases [27,28], and implement precision agriculture tasks [29]. When applied in the forestry domain, UAVs are especially appealing since they allow regular, frequent, and on-demand data acquisition, can host a diverse range of sensors according to the application, and can be used in real-time operations [30]. Since forest environments are dynamic environments, data acquisition must be accurate and up-to-date. UAVs provide a significant advantage over traditional remote sensing techniques such as satellite imagery, notably in this area.

Works in applying UAVs for forestry applications have gained traction over the years. One author Hyyppä et al. [31] explored the use of UAVs and a SLAM system to collect information using a Light Detection and Ranging (LiDAR) sensor, including stems of individual trees, stem curves, and the diameters at breast height (DBH). Another author Corte et al. [32] employed machine learning techniques to design accurate predictive models from the LiDAR data collected using the UAV to conduct forest inventories. Previous works have also used low-altitude aerial images and LiDAR data to construct a local scale model of the forest canopy surface, with information about the elevation of vegetation, through photogrammetry and structure from motion approaches [33]. Researchers have also examined using UAVs to monitor fire and its effects [34,35], identify spatial gaps [36], and assess soil disturbances [37].

Due to the large scale of forest environments, past research which used UAV data for forest inventory applications opted for using UAV data on small areas [31,33,38,39]. Results from these works have demonstrated the promising potential of using UAVs to map and estimate forest resources. Further, [40] have investigated integrating partial-coverage UAV data with field plots to estimate forest resource parameters (i.e., volume, biomass, etc.) as a means to address the issue of mapping large-scale environments. Fusing semantic information is especially important for forest fire mitigation efforts as it allows a 3D representation containing classification information such as the location of flammable vegetation and other classes for a globally-registered map of the forest environment. While semantic mapping has gained traction over the recent years and has been applied in areas such as autonomous driving [41,42] or assistive robotics [43,44], implementing semantic mapping directly into the forest domain is still a work in progress, to the best of our knowledge. Past research in this area has investigated using high-resolution RGB images acquired by UAVs to classify shrubs [45], and tree species [46] in a heterogeneous forest environment. More recently, researchers have used classification approaches to target the forest wildfire issue.

Automating survey methods that involve deploying autonomous robotic systems to navigate in the field requires the robots to have a robust Simultaneous Localization and Mapping (SLAM) system to estimate their state in 3D space. Over the years, SLAM research has matured with the development of different systems based on various combinations of sensor modalities [47]. LiDAR Odometry and Mapping (LOAM) [48] is a state-of-the-art loosely-coupled LiDAR and inertial-based SLAM algorithm. In tightly-coupled sensor fusion frameworks, sensor data from different modalities are jointly processed and combined at a lower level, usually through a filtering or optimization framework, to estimate the robot’s pose and update the map. However, tightly-coupled frameworks require more complex algorithms and computational resources to achieve joint estimation and optimization. The LiDAR odometry via smoothing and mapping (LIO-SAM) [49] is a tightly-coupled LiDAR and inertial-based SLAM algorithm which expands on the LOAM framework by including loop closure methods and feedback from absolute measurements such as the Global Positioning System (GPS) and compass heading. Works have also looked into fusing LiDAR, visual, and inertial modalities, such as [50] which featured visual-inertial odometry (VIO) system and a LiDAR odometry (LO) system. The VIO subsystem is tightly-coupled and serves as a motion model for the LiDAR mapping which dewarps LiDAR points and registers the scans to the global map. Various works have implemented SLAM systems, primarily LiDAR and inertial-based, in the forest environment, for tree detection and tree trunk estimation [51], and 3D forestry inventory mapping [52,53].

3. Materials and Methods

The design and development of the complete system included customized hardware and software components. The hardware includes a payload designed to allow easy mounting on diverse types of drones. It also contains different types of sensors that provide spectral, visual, and range information. The software stack includes synchronization of the sensor data acquisition, data logging, and processing. Details of the individual components are discussed in the following sections.

3.1. Sensing Payload

We designed and deployed a sensing payload with different types of components that include an inertial measurement unit IMU (VN-200 Rugged GNSS/INS), a three-dimensional Light Detection and Ranging scanner (LiDAR) with 32 channels (Velodyne VLP-32), a 20.2 Megapixel stereo camera with 8.5 mm focal length lenses (Allied Vision Alvium 1800U-2040c), a multispectral camera (Mapir Survey3W, Red+Green+near-infrared channels), and two onboard computers (NVIDIA Jetson Xavier Development Kit and the Intel core i7 NUC). Given the highly unstructured, dynamic, and changing forest environment, this sensing payload was designed to have robust sensing capabilities and is lightweight while also taking into account future research work. Due to the lack of perception work in forestry, the sensors chosen allow for future work and ablation studies regarding the best sensory combination for important decision-making modules such as semantic mapping, navigation, and SLAM. The LiDAR, stereo cameras, and IMU are chosen to have different sensing modalities on board with different fields of view and spectral characteristics. This helps to account for conditions that may render either of the sensors susceptible to poor performance. For instance, the stereo cameras have a horizontal and vertical field of view of 60.4° and 44.9°, respectively. The LiDAR has a horizontal and vertical field of view of 360° and 40°, respectively. We also opted for two onboard computers to divide the computing load for recording and processing the data. The Xavier board is intended to run machine learning models, while the NUC computer handles all the heavy CPU calculations (e.g., SLAM processing). Figure 1 shows a 3D rendering of the payload and the mounting of its different components.

The sensing payload was designed for easy mounting on any aerial platform and can be tilted at different viewpoints ranging from horizon to nadir. Furthermore, the components are mounted in a modular way, allowing easy integration and removal of each one, if necessary, for operational constraints (e.g., limited weight capacity). In order to minimize weight, the sensing payload is configured to be powered using the UAV’s batteries. Hence, a power distribution board was used to distribute the appropriate voltages to each component.

Table 1. Sensors and Operation Frequency.

Sensor Name	Sensor Type	Frequency (Hz)	Voltage (V)
VN-200 Rugged GNSS/INS	IMU	400	$3.2$–$5.5$
Velodyne VLP-32C	LiDAR	10	$10.5$–$18.0$
Allied Vision Alvium 1800 U-2040c	RGB camera	10	5.0
Mapir Survey 3	Multispectral camera	15	5.0

shows all the sensors in the payload and their operational frequency. The total weight of the payload is 3.832 kg, without factoring in the weight of the landing gears and the mounting plate. Altogether, the electrical and mechanical properties of the payload allowed for an operation of 18–20 min when mounted on a DJI Matrice 600 (using six TB47S batteries, each with 4500 mAh) and 30–35 min when mounted on an Alta X (using two 16,000 mAh batteries).

3.2. Simultaneous Localization and Mapping

We used the LiDAR, IMU, and Global Navigation Satellite System (GNSS) measurements to estimate the UAV’s 6 degrees of freedom pose (odometry) during the flight. Thus, the estimated state of the robot is represented as a 6-dimensional vector corresponding to the drone’s position and its orientation in space. We employed the estimated odometry to incrementally build a three-dimensional (3D) environment map in the form of a point cloud. Accurate pose estimation is crucial for a global map such as fuel mapping implemented in Section 3.3.

In this work, we used Fast LiDAR Inertial Odometry with Scan Context, FASTLIO-SC, [54,55] and LiDAR Inertial Odometry Smoothing and Mapping, LIOSAM, [49], which are two state of the art SLAM implementations that fuse the sensors measurements using a tightly coupled approach. We decided to focus on LiDAR inertial-based SLAM systems because cameras are greatly affected by lighting conditions and motion blur. Results from [14] have also reported that the visual odometry had higher RMSE compared with the LiDAR odometry. LIOSAM performs state smoothing and mapping scan matching using a factor graph that introduces relative and absolute measurements as factors [49]. Relative measurements, such as IMU and LiDAR data, are fused to optimize the calculation of the odometry, further improved by GNSS absolute positioning (when available). In particular, the IMU information allows LIOSAM to estimate an initial guess of the LiDAR’s position, velocity, and orientation by a preintegration stage, which integrates IMU measurements over the LiDAR scan rate. LIOSAM uses this LiDAR pose initial guess for de-skewing LiDAR point clouds and computing the optimized odometry. Odometry prior optimization is obtained by performing a local scan matching between consecutive LiDAR scans using corner and surface features. The GNSS measurements are included in the LiDAR optimization stage for enhancing pose estimation. Finally, the optimized pose is used as feedback to the preintegration stage for determining the IMU bias (providing the tight coupling).

Conversely, FASTLIO-SC fuses LiDAR and IMU measurements by a tightly-couple-iterated extended Kalman filter, which enables robust navigation in fast motion, noisy or occluded environments [54]. Specifically, the iterated extended Kalman filter is implemented using the Iterated Kalman Filters on Manifold’s toolkit, based on the premise that the robotic system evolves on a SO(3) manifold [56]. Thus, FASTLIO-SC proposes a new formula to compute the Kalman gain that depends on the system state instead of the dimension of the measurement. In this context, the iterative extended Kalman filter uses LiDAR and IMU measurements as input for estimating LiDAR odometry. Next, the LiDAR odometry is improved by GNSS measurements and movement restriction imposed by a scan context approach [55]. This information is fused using a factor graph strategy which outputs an optimized LiDAR odometry. In contrast to LIOSAM, the optimized LiDAR odometry is not used as feedback for estimating IMU bias. A visual representation of the architecture of these two approaches is depicted in Figure 2.

Time synchronization among the sensors and computers is a critical issue and significantly impacts the quality of the SLAM system. We designed a custom time synchronization board featuring a Teensy microcontroller that synchronized the computers, the LiDAR, the IMU, and the cameras by generating PPS signals that simulate the ones obtained from a GNSS. These signals comprise squared signals used to trigger the cameras, LiDAR, and are encoded using the National Marine Electronics Association (NMEA) strings protocol for serial communication. Having the microcontroller mimic GNSS signals is desirable in this application since forest environments can contain certain areas which would cause unreliable GNSS readings (and thus its PPS signals).

3.2.1. Photogrammetry

Photogrammetry is a technique in computer vision used to estimate the structure of a scene from a collection of images with optional GNSS information. Most modern approaches do not require any prior sensor calibration, which makes the tools robust and easy to deploy. Over the last decade, both industry and academia have developed easy-to-use software which enables wide-scale adoption of this technology to perform surveys in diverse domains such as agriculture, construction, or forestry. An important characteristic of photogrammetry is that it is solely an offline process, which requires all the images to be available before computation begins. We use photogrammetry as a benchmark to evaluate the performance of the online methods we developed in this work.

In this work, we used Agisoft Metashape [57] 2.0.2, a common commercial structure from motion software. We downsampled our images temporally so that they were at least a half second apart because they would be highly redundant otherwise. For our collection in the most representative site (see Section 4.1), this resulted in 886 images. Each image was tagged with the RTK GNSS position which most closely matched the image timestamp. For the initial step of estimating the camera parameters and computing the sparse geometry, we use the parameters which were optimized for forestry as found by [58]. Then we performed meshing using the “High” quality and “High” face count settings. The pose estimation took 10 min and the meshing process took 32 min on a cloud server with a CPU AMD EPYC-Milan Processor, 60 GB RAM, and a GRID A100X-20C GPU.

3.3. Fuel Mapping

In this work, fuel mapping involves three tightly coupled modules: semantic segmentation, LiDAR camera registration and odometry, and global map representation. To achieve accurate semantic segmentation, we employ the SegNeXt network, a transformer architecture was specifically chosen for its state-of-the-art performance on popular benchmark datasets and excellent generalization capabilities, which is particularly beneficial given our limited real-world training images [59]. This network identifies the predicted class for each pixel in the image, providing a semantically labeled image for three compressed classes, fuel, canopies, and background. We trained this Neural Network (NN) with pre-trained weights from the CityScapes dataset [60] and 40 RGB images collected in our deployments. While 40 images were used for training, the actual number of instances is higher, since a class can appear multiple times in a single image. The RGB images have a higher resolution than our multispectral and it allows for pre-weight transfer learning from other datasets. While there is current research on using GRN (Green, Red, Near Infrared) images with pre-weights from RGB datasets, due to the the lack of state-of-the-art research and the scope of this paper, we decided to use RGB images instead.

As mentioned in Section 1, we used the Anderson fuel model [11], which is comprised of 15 classes as included in

Table 2. Classes used in semantic segmentation. These are inspired by relevant classes from the Anderson fuel model [11] and also include additional classes relevant to our application.

Fine-Grained Class	Coarse-Grained Classes
Dry Grass	Fuel
Green Grass	Fuel
Dry Shrubs	Fuel
Wood Pieces	Fuel
Litterfall	Fuel
Timber Litter	Fuel
Green Shrubs	Canopy
Canopy	Canopy
Live Trunks	Canopy
Bare Earth	Background
People	Background
Sky	Background
Blurry	Background
Drone	Background
Obstacle	Background

to train our semantic segmentation model, which is publicly available (All models are available at https://github.com/Kantor-Lab/Safeforest_CMU_data_dvc/tree/public_models (accessed on 15 June 2023)). However, the classes were aggregated into three compressed classes at the post-processing steps given we did not observe all of them in the testing sites and fine-grained training has shown to improve convolutional neural network’s performance [61]. Additionally, aggregated classes allowed simplifying the validation mapping labeling for the qualitative comparison. We chose these three classes based on easier visual inspection for fuel regions, as the site’s bare earth paths and a cluster of canopies allow for clearer boundaries between the chosen classes. Moreover, the other non-forest related classes were aggregated as background as it falls outside the scope of our current proposed system.

Subsequently, to transform the semantic image into a global map, we integrated LiDAR and camera data. This step is essential to ensure that the map projection of the semantic classes is accurate for the ground robot to mulch all potential flammable materials as our main goal is to prevent harmful wildfires. Through LiDAR camera registration and odometry, we determined which LiDAR points correspond to the camera’s field of view, enabling the registration of image pixels into the global coordinate system. By leveraging the extrinsic of the LiDAR relative to the camera, we transform LiDAR measurements into the camera’s coordinate frame. Using the calibrated camera intrinsic parameters, we project each LiDAR point onto the image plane. Points falling within the camera’s field of view are assigned classification labels from the corresponding pixels in the semantic model. The resulting semantically-textured point cloud is then transformed into the inertial reference frame using the drone’s current pose estimated by our SLAM system. This integration of LiDAR, camera, and SLAM data allows us to create a comprehensive global representation that captures the fuel characteristics in the environment.

To ensure the efficient creation of a large-scale global map in real-time, it is necessary to utilize lightweight map data structures that can handle the voluminous information generated. Various methods such as Octree [62], Truncated Signed Distance Function (TSDF) [63], and openVDB [64] were considered for our research. After a careful cost-benefit analysis considering factors such as map resolution, semantic information, and computational efficiency, we identified two methods as the most suitable for our scenario: semantic OctoMap [65,66] and Unknown Free Occupied Map (UFOMap) [67]. Among them, UFOMap emerged as the preferred choice due to its significant speed advantage over OctoMap, being 26 times faster and reducing memory consumption by 38% [67]. Both methods use the Octree data structure, which offers a significant reduction in memory requirements compared to point clouds while maintaining high map quality. They convert the point cloud into voxels with a resolution of 0.1 m, while preserving valuable information about the predicted classifications. The global octree is continuously updated by incorporating newly generated semantic point clouds. By leveraging the lightweight Octree map data structure from UFOMap, we can create a comprehensive semantic global representation of the environment in real time.

4. Results

In order to evaluate our localization and mapping methods, we conducted two data collection campaigns in semi-urban and rural sites (Figure 3). The semi-urban site is located in Coimbra, Portugal ($40°{11}^{\prime }{05.9}^{″}$ N $8°{24}^{\prime }{50.5}^{″}$ W). The rural site is located in Pittsburgh, United States ($40°{27}^{\prime }{24.6}^{″}$ N $79°{47}^{\prime }{22.1}^{″}$ W). We choose these two locations in order to test the quality of the generated maps in scenarios with relatively low and high complexities see Figure 3. As the Coimbra test site is a semi-urban environment, it contains more structured-like landscapes making it a low-complexity scenario for our application. In contrast, the rural test site is unstructured and wild, posing a more challenging condition for the feature extraction and subsequent scan matching of the employed SLAM approaches.

The flight plan of the surveys performed at the Coimbra test site was controlled by an experienced UAV pilot. On the other hand, for the rural test site, which covers a larger area, we decided to use the UAV’s autopilot. For each survey the altitude, ground speed, and lateral distance were about 30 m, 3.0 ms${}^{-1}$, and 20 m, respectively.

4.1. Simultaneous Localization and Mapping (SLAM)

In order to evaluate the quality and suitability of the SLAM approaches for mapping forest regions, we separated the localization and mapping results. The system localization was evaluated by comparing the pose estimations given by LIOSAM and FASTLIO-SC with position measurements from a centimeter-level accuracy real-time kinematics (RTK) device. The main metric employed in this case is the root mean squared error (RMSE) calculated over a complete trajectory. In all the surveys, it was found that LIOSAM outputs lower RMSE than FASTLIO-SC, as

Table 3. Quantitative odometry results of LIOSAM and FASTLIO-SC approaches on forested regions. Bold values denote the lowest RMSE. The last row shows the average RMSE as a percentage of the flight distance for each survey.

	Study Sites
	Coimbra				Pittsburgh
	Survey 1		Survey 2		Survey 1		Survey 2		Survey 3
Flight distance m	175.2		424.4		1535.5		2421.1		1188.6
	LIOSAM	FASTLIO-SC	LIOSAM	FASTLIO-SC	LIOSAM	FASTLIO-SC	LIOSAM	FASTLIO-SC	LIOSAM	FASTLIO-SC
X m	0.26	1.79	0.67	42.51	1.33	2.67	0.73	0.750	0.72	1.80
Y m	0.24	3.49	0.90	27.50	0.79	5.39	1.51	5.70	0.40	6.02
Z m	0.33	4.13	0.63	7.88	1.60	4.64	3.53	3.00	1.91	5.88
Average m	0.28	3.14	0.73	25.96	1.24	4.23	1.92	3.15	1.01	4.57
%	0.16	1.79	0.30	10.70	0.08	0.28	0.08	0.13	0.08	0.38

shows.

Regarding translational RMSEs, LIOSAM outperforms FASTLIO-SC, showing errors between 0.28 m to 1.92 m. A lower RMSE indicates that the implemented algorithm can estimate the vehicle’s odometry closer to the current trajectory. Consequently, the drift between the vehicle’s current position and the estimated position by LIOSAM is lower than that achieved by FASTLIO-SC. Specifically, from Figure 4 one can observe that LIOSAM produces optimized odometry that closely aligns with the GNSS measurements. Conversely, FASTLIO-SC’s odometry significantly differs from the actual ground truth measurements at the beginning of each survey. This initial deviation is attributed to the heading initialization utilized by FASTLIO-SC, which differs from the true yaw orientation of the platform. As a result, the RMSE of FASTLIO-SC increases, leading to comparatively lower performance when compared to LIOSAM, as shown in Table 3.

Based on the before mentioned, noting that previous works have reported that in forested regions, one could anticipate RMSE values ranging from $0.32$ m to $2.90$ m [68], and considering that the RMSEs yield by LISOAM in both study sites account for less than the $0.3\%$ of the overall trajectory, we decided to assess the LIOSAM’s mapping outcomes further. Specifically, LIOSAM was tested using data collected from study site 2, a rural environment that closely resembles a forested area due to its predominant composition of low and high vegetation. At this point, we employed the mesh generated by the Agisoft photogrammetry software as the baseline, as mentioned in Section 3.2.1. In order to have a strong indication of the quality of the generated map regarding its morphological, visual, and geometry aspects, two approaches were used: (i) using a cloud-to-cloud comparison with the photogrammetry output and (ii) measuring geometric variables (i.e., distances and areas) in different maps.

The cloud-to-cloud evaluation is mainly quantitative and consisted in first converting the mesh into a point cloud (i.e., reference cloud). Subsequently, the point cloud generated with LIOSAM was translated and rotated so both of them were spatially aligned. Finally, we measured the geometric difference between them using a nearest neighbor criteria similar to the Hausdorff metric [69]. The key difference is that to improve accuracy, a local model around the closest point in the reference cloud is computed to approximate its surface and thus have a more accurate estimation of the distance. This calculation was done using the Cloud Compare software [70]. Figure 5 shows the reference cloud (top left), and the raw SLAM cloud in two formats: colorized according to its height (top right) and colorized according to its distance to the reference (bottom). In general, it can be seen that various parts of the generated map are similar to the reference under this metric (blue-colored points), but the distance increases on some of the treetops. Furthermore, the boxplot shows a median of 1.51 m, with outliers for errors above 5.54 m. These outcomes indicate that height in the maps is one of the main uncertain variables to measure in these maps. This outcome is somehow expected since the treetops in the LIOSAM point cloud are less dense and more noisy. Also, the height estimation derived from photogrammetry can be subject to large errors, which makes vertical distances in higher structures (like trees) more uncertain [71]. For this reason, we consider this evaluation qualitative.

An important thing to note in Figure 5 is the size of the maps. Since the baseline uses only the left camera images, the map is limited by their field of view. In contrast, the SLAM map spans a larger region since the LiDAR has a range of up to 200 m (in this work we limited the LiDAR range to 100 m) with a wider field of view.

Given that the previous evaluation is mostly qualitative, the second evaluation of the maps consisted of measuring two main geometric variables: the area of characteristic regions and the length between two landmarks. Since the photogrammetry map is prone to errors given by the feature matching and geolocalization (see Section 3.2.1), we considered measurements taken in Google Earth software [72] as the main baseline. Its satellite imagery provides a resolution of 0.15 m, which is accurate enough considering the total testing area covers around 3.000 m${}^2$. On the other hand, the reference and SLAM maps have a resolution of 1 cm and 30 cm, respectively. Considering that the baseline and the SLAM maps span a different area, we chose regions and landmarks that appear in both, as Figure 6 shows.

The results of this evaluation show that in general, the measurements obtained using the LIOSAM map are closer to the ground truth, as

Table 4. Quantitative evaluation of the map resulting from our SLAM implementation. All of the area regions are measured in m${}^2$ and length segments are measured in m.

Measurement Type	Google Earth	Photogrammetry	Absolute Error	LIOSAM Map	Absolute Error
Area region 1	308.81	489.24	180.43	322.00	13.19
Area region 2	3021.28	3144.47	123.19	3116.16	94.88
Area region 3	4561.49	4742.12	180.63	4591.95	30.46
Area region 4	6293.18	6216.12	77.06	6268.21	24.97
Area region 5	1156.96	1161.97	5.01	1201.53	44.57
Length Segment 1	78.09	91.22	13.13	77.29	0.8
Lenght Segment 2	31.36	37.78	6.42	32.59	1.23
Length Segment 3	78.83	92.07	13.24	79.03	0.2
Lenght Segment 4	154.97	182.03	27.06	154.96	0.01
Length Segment 5	86.49	99.66	13.17	88.84	2.35

summarizes. This outcome, along with the qualitative evaluation and the assessment of the odometry estimation, allows us to infer that the geometry and scale of the LIOSAM algorithm are suitable for generating accurate 3D models of forest regions online. In our tests, we also observed that using the GNSS signal as an additional observation in the estimation scheme of LIOSAM greatly improves its odometry estimations and, consequently the map generated. Finally, it is also noteworthy that having larger errors using the photogrammetry map does not imply that it is unsuitable for this application. Some practical strategies can be used to improve the accuracy of such maps. For example, having geo-referenced landmarks in the form of ground control points greatly helps to improve the optimization process during the feature matching and reprojection stages.

4.2. Fuel Mapping

In this section, we detail how we evaluate the critical individual modules necessary for the final map, specifically semantic segmentation and fuel mapping. We address the current drawbacks of each technique, our setting limitations, and our solutions to overcome them.

In our evaluation of the semantic segmentation model, we utilize a small annotated test set derived from the semantic mapping mission in the rural test site, which was not used for training. For this purpose, we annotated 17 images using the same strategy employed during training, but these images were not included in the training dataset. To ensure flora safety, we have used a conservative approach when labeling fuel where we would underestimate fuel to avoid future issues where the ground vehicle would mulch canopies. Similarly, for the training case, it is worth noting that despite having 17 images, the actual instances used for testing are higher. This is because some classes can appear more than once in a single image. We conducted an analysis of the semantic segmentation using precision, recall, and intersection over union (IoU) metrics to assess the fuel classification in our system. These metrics were chosen because of their importance for addressing false positive/negative results while IoU estimates the general quality of the segmentation. We decided to mainly focus on precision for our results because it avoids false positives, ideal for fuel classification to avoid mulching green vegetation and other important dynamic classes such as humans and animals.

It is worth noting that certain classes which were included in the original 15 classes, are absent from the evaluation

Table 5. Evaluation results of the SegNext network with the Anderson Fuel Model as a base for semantic segmentation in a forestry environment.

Class	IoU	Precision	Recall
Canopy	70.05	84.77	80.13
Dry Grass	79.7	93.75	84.17
Bare Earth	78.53	88.12	87.83
Green Shrubs	3.27	21.72	3.71
Green Grass	0.0	0.0	0.0
Dry Shrubs	0.0	0.0	0.0
Live Trunks	0.05	0.05	84.09

. This is because these classes were not present in the test set, as seen in Figure 7. Consequently, these classes were excluded from the results. We also took into account the lack of training instances by adding class weights to rarely occurred classes in our training model to lower dataset shift issues. Even with the aforementioned techniques used, there is a noticeable shift in performance between the common classes and the rare classes, which could stem from our lack of expert annotators to ensure consistency for each subclass, the need for more class imbalances techniques, or the addition of more training instances. However, considering a ground vehicle can do further exploring and the high precision in the known fuel classes, we can conclude that the current results are above our needed threshold to find fuel regions for further inspection.

To provide further insight into the model performance, we show qualitative predictions on the test set, as seen in Figure 8.

Finally, we present the results of the semantic mapping experiment conducted at the most representative test site (the rural location in Pittsburgh, United States). Due to the lack of ground truth information for the geometric and semantic information in the test sites, quantitative metrics for semantic mapping in large-scale forestry environments were not available. For quantitative results, a high-precision laser scanner such as the Leica NOVA MS50 or BLK360 [73] or adding ground control points for photogrammetry is necessary to quantify the results. Therefore, we devised a qualitative test to evaluate our full framework by comparing the generated UFOMap with a manually labeled semantic mesh from an orthomosaic to evaluate the accuracy of our global map.

Figure 9 showcases the semantic UFOMap results, including the orthomosaic ground truth labels for the compressed classes (fuel in red, canopy in green, and background in black), along with the RGB UFOMap representing the terrain baseline. Additionally, Figure 10 shows a close-up for both image and segmentation for a clearer comparison between semantic classes and the original rgb image.

5. Discussion

In the two test sites surveyed in this work, LIOSAM demonstrates superior performance in terms of RMSE compared to FASTLIO-SCon the study sites reported in this work. Specifically, we noticed that LIOSAM might outperform FASTLIO-SC in two key aspects: IMU heading initialization and efficient odometry updates facilitate the integration of GNSS measurements within the optimization loop. The positive impact of heading initialization and odometry updates can be observed in the quantitative and qualitative results depicted in Table 3 and Figure 4.

Although the experiments addressed in the current work show that LIOSAM achieves better odometry and mapping outcomes, it should be noted that for the Pittsburgh study site, the average RMSE is higher than the Coimbra study site. Although the flights in the two testing sites were different (autonomous in Pittsburgh, manual pilot in Coimbra), the main factor that advocates the higher errors in the Pittsburgh surveys might be related to the need for more reliable features for determining a closer LiDAR odometry to the actual trajectory. Specifically, LIOSAM extracts corner and surface features from LiDAR scans to perform scan matching between point clouds and retrieve LiDAR odometry. Thus, such features might not be reliable enough to yield accurate LiDAR odometry. Note that in the semi-urban environment, the Coimbra study site, where corner features are more abundant than in a rural area, the average RMSE is below 1 m, which is acceptable considering the size of the surveyed area and the distance flown.

While the two flights regarding FASTLIO-SC, it performs direct registration of LiDAR scans; thus, it does not need computing corner of surface features. However, the experiments performed in both study sites show that its main drawback might be related to the initial yaw orientation. In particular, the FASTLIO-SC yaw angle does not align with the actual IMU heading angle. This mistrust alignment severely affects the estimated odometry because it starts deviating from the current location of the platform; see Figure 4. For instance, one of the worst deviations occurs on the Coimbra study site, where the RMSE on the X axis rises to 42.51 m.

Based on the aforementioned, LIOSAM might be more suitable than FASTLIO-SC for performing localization and mapping tasks in semi-urban and rural areas. Nevertheless, it is suggested to continue evaluating the limitations and advantages of both SLAM approaches on rural and forest environments in future works. Further assessment of these algorithms might yield insights about a new robust SLAM strategy for these environments.

Our semantic segmentation experiments show that a small number of labeled images from a similar region can result in relatively accurate predictions for that region. Overall, the predictions in Figure 8 show fairly good agreement with the ground truth. The errors often occurred when the textures were indicative of the predicted class, suggesting that the model struggles with more global reasoning. This could be further improved with a comparison study between RGB and multispectral cameras to see which factors more in the quality of the segmentation, the channel choices or the higher resolution. This was not possible for this paper as we did not have enough annotators for both sensors. By observing both the raw classes and the classes aggregated by the rules defined in Table 2, we see that many of the prediction errors fall within the same aggregate class. Given the variability of vegetation by geographic area and different objectives of mapping, it seems that training many site-specific models may be easier than expecting one fixed model to generalize worldwide. This observation agrees with the result of our previous work [14], where it was shown that increasing the number of training samples, marginally increased the segmentation accuracy.

In the fuel mapping results, we observed that capturing the map from a higher altitude, specifically between 30 m and 40 m above the ground, with resolutions below 0.1 m did not yield a cohesive and dense map due to limitations in the LiDAR density. Furthermore, the mapping process faced challenges related to odometry dropouts as the map expanded and the computational load required for neural network operations. However, despite these challenges, a high map density was achieved by collecting data points at a frequency of approximately 4 Hz, ensuring a sufficient representation of the environment as seen in Figure 9.

Visual comparison from Figure 9b,c (between the semantic UFOMap and the ground truth map) revealed a general alignment. Nonetheless, specific issues were identified, particularly in localized areas such as thin layers of background misidentified as canopies. These issues could be attributed to factors such as pose drift, abrupt jumps resulting from the lack of loop closure updates in the odometry messages employed, and semantic mapping. Additionally, utilizing detection without tracking had a negative impact on the segmentation quality of the map. Incorporating tracking mechanisms would enhance the robustness and accuracy of the mapping process, thereby contributing to a more reliable global representation system.

Importantly, despite room for improvement, the generated global map demonstrated a conservative approach to fuel classification, prioritizing the preservation of tree cover and minimizing the risk of erroneously identifying and removing trees. This approach aligns with the objective of accurately identifying fuel regions for wildfire prevention. These observations provide valuable insights into the challenges associated with semantic mapping. However, they underscore the significance of addressing loop closure limitations, improving tracking mechanisms, and optimizing resolution parameters to achieve a cohesive and detailed global map representation.

6. Conclusions

This paper presents a comprehensive system for monitoring forests using UAVs. The system includes a custom sensor payload with LiDAR, stereo cameras, IMU, and a multispectral camera mounted on a drone to collect reliable data in challenging forest environments. The payload is designed to be lightweight and adaptable to different drones, providing flexibility. The study implements two SLAM algorithms, LIOSAM and FASTLIO-SC, for accurate localization and mapping using LiDAR, GNSS, and IMU data. Photogrammetry is used as a benchmark to evaluate the performance of the developed online methods. The integration of fuel mapping, which includes semantic segmentation, LiDAR camera registration, and odometry, is discussed, as well as the adaptation of the global map representation. The paper concludes by highlighting qualitative tests and a comparison of the generated semantic maps with manually labeled data. Overall, this system provides a comprehensive solution for forest monitoring with drones, enabling accurate data collection and mapping for effective forest management and conservation.

Future work needs to address the limitations identified in our current framework. Alternative initialization strategies should be explored to improve odometry performance from FASTLIO-SC. Improving the alignment accuracy between the SLAM map and the reference cloud, especially with respect to tree canopy, is essential for a more accurate representation of forested areas. The accuracy of photogrammetry maps can be improved by including ground control points, which will be also used to improve the evaluation of the mapping results. In this context, the collection of geometric variables like distances and areas in situ will provide ground truth for mapping evaluation. The fuel mapping experiment faced challenges related to map coherence, odometry dropouts, and semantic segmentation results. Future research should focus on improving semantic mapping loop closure, implementing tracking for the semantic voxels, adding the multispectral camera for detection, and optimizing resolution parameters to overcome these challenges. Advances in the performance of SLAM, map alignment accuracy, geometric measurements, semantic segmentation, and fuel classification are critical to developing a robust and accurate forest mapping system that facilitates effective forest management and conservation efforts.