The USDA Natural Resources Conservation Service (NRCS) commissioned the acquisition of imagery and lidar using the Leica Geosystems CountryMapper hybrid sensor. Two pilot areas of interest were flown in north-central Colorado over two different physical settings to evaluate the system’s performance. The western area of interest (AOI) included land managed by the Bureau of Land Management and the U.S. Forest Service. The eastern AOI included agricultural and urban areas. USGS scientists conducted field data collection efforts during the weeks of 9–13 September and 18–22 November 2019, using a combination of technologies to map and validate topography, vegetation, and features in the two AOIs. The work was initiated as an effort to test and evaluate the Leica Geosystems CountryMapper sensor.
The CountryMapper is currently in development and specifications about the system are not publicly available yet. The CountryMapper has the potential to collect data that satisfy both USGS NGP 3DEP and USDA NAIP requirements in a single collection. The CountryMapper is a hybrid sensor that collects imagery and lidar data simultaneously. This research will help the USGS determine if this sensor has the potential to meet current and future 3DEP topographic lidar collection requirements. The field surveys were performed to evaluate the 3D absolute and relative accuracy of the airborne Leica Geosystems CountryMapper lidar acquired for the USDA NRCS and to determine if the data meet 3DEP specifications.
High accuracy 3D point data are necessary to estimate the 3D accuracy of airborne lidar data. The survey data will be used to spatially assess the horizontal and vertical accuracy of the lidar produced by the Leica Geosystems CountryMapper sensor along with analyzing plane to plane offsets between various infrastructure roof features.
1.1. Lidar Point Cloud Accuracy
Several agencies and professional organizations have produced guidelines and standards for geospatial data, including lidar [
1,
2,
3,
4,
5]. The positional accuracy standard from the American Society for Photogrammetry and Remote Sensing (ASPRS) [
1] and the USGS Lidar Base Specification (LBS) [
2] are particularly useful references. The horizontal and vertical accuracy of the CountryMapper airborne lidar data were evaluated following the framework of the LBS. The suggested accuracy assessment has several different areas, such as horizontal and vertical, relative and absolute, and intraswath and interswath. The concepts are described here and in the methods section in detail, and then the results of the analysis are presented.
When vertical accuracy is evaluated for a lidar point cloud, the checkpoints need to be located on a flat surface so that the uncertainty of the horizontal position does not contribute to the vertical error. Lidar point clouds do not yield appropriate conjugate points that can be compared to ground truth points for assessing horizontal accuracy, unlike planimetric data. Thus, we need to rely on geometric feature-based objects to model unique points defined by those geometric objects. For instance, if there is a three-plane object, the intersection point of the three planes can be considered as a uniquely determined point. Thus, finding a conjugate intersection point from the ground truth reference data and another point, determined from airborne lidar data, allows for the comparison of the data and the evaluation of the horizontal and vertical errors from a full 3D perspective.
One area of lidar accuracy assessment is determining the relative difference and absolute error. Any lidar accuracy metric that is measured against ground-surveyed data is categorized as an absolute error, anything other than that is considered to be a relative difference. For instance, the vertical accuracy of an airborne lidar point cloud is typically measured against ground-surveyed checkpoints, thus it is an absolute error and the root mean squared error in the z direction (RMSEz) can be calculated. On the other hand, when two lidar swaths overlap and lidar point cloud features from one swath are compared to the other, it is a relative difference and the root mean squared difference (RMSD) can be determined.
Intraswath analysis and interswath analysis is another category of accuracy analysis. Intraswath analysis utilizes point cloud data from a single swath. For instance, smooth surface precision measures the consistency of the elevation along a smooth planar object from a single swath, because surface precision is determined mainly by the laser ranging uncertainty and scanner stability. On the other hand, the interswath difference analyzes two datasets from two overlapping swaths and reveals boresighting errors.
Comparing overlapping swaths from the same lidar sensor to diagnose systematic errors inherent in the instrument is a common practice in the perspective of relative error analysis [
6]. In order to evaluate absolute error, the identification of conjugate points from a reference point cloud and comparison point cloud should be made using conjugate geometric features. As the geometric feature-based accuracy assessment can properly address the lidar accuracy in a full 3D sense, the methods based on using planar features are a straightforward choice. Basically, they take advantage of using more than simple global navigation satellite system (GNSS) points to assess the accuracy of airborne data. The evaluation of the accuracy of an unmanned aerial system (UAS) point cloud was performed using planar surfaces from terrestrial lidar scanner (TLS) data [
7]. Generic plane finding techniques from lidar point clouds are available based on Hough transform [
8], region growing [
9,
10,
11], and the random sample consensus (RANSAC) segmentation algorithms [
12]. When additional high-resolution areal images are available, the fine resolution semantic information from image data can be used along with the lidar elevation information to extract planar features [
13,
14,
15,
16,
17].
The manual selection of plane points is desirable, however, to make sure that only the correct points are included in the plane modeling. From a practical perspective, roof planes are the most useful three-plane geometric features in regard to 3D accuracy assessments. In most cases, either TLS or UAS lidar data can be used as high accuracy reference data. The difficulties in obtaining permits to access the sites and time-consuming data collection will limit the number of checkpoints, thus manual selection is a practical choice. Automatically selected checkpoints will need manual inspection. This is because the accuracy assessment is not about the speed for mass production but is about the precise implementation to make sure no unwanted errors are induced from improper data sampling. In addressing this sampling issue, Kim et al. [
18] focused on the valid conditions for identifying conjugate points from geometric features. The valid conditions for a three-plane object, such as point precision, plane size, and point density, were integrated into a general external uncertainty model [
18]. The external uncertainty model was applied in assessing the 3D absolute accuracy of the NAIP/3DEP airborne lidar point cloud.
With these other methods in mind, Kim et al. [
18] focused on the valid conditions for identifying conjugate points from geometric features. The valid conditions for a three-plane object, such as point precision, plane size, and point density, were integrated into a general external uncertainty model [
18]. This research applied the external uncertainty model in assessing the 3D absolute accuracy of the NAIP/3DEP airborne lidar point cloud.
1.2. NAIP/3DEP Pilot Project
The airborne data acquisition for this pilot project was funded by the USDA NRCS National Geospatial Center of Excellence (NGCE) as an add-on to the NAIP acquisition working with the Farm Services Agency Aerial Photography Field Office (FSA-APFO). These USDA agencies collaborated to plan and execute a hybrid sensor pilot project to explore the technical viability of acquiring lidar data meeting the USGS 3DEP LBS during the statewide NAIP imagery collections. Early in the project, the USDA reached out to the USGS, U.S. Forest Service (USFS), Bureau of Land Management (BLM), National Park Services (NPS), and other federal 3DEP partners to guide and support the project with technical expertise and field work. This field work is a crucial component of the pilot project to compare ground observations with the remotely sensed imagery and lidar measurements.
The major goals for this pilot project from the USDA’s perspective were:
Determine if data meeting both NAIP imagery specifications and 3DEP LBS can be acquired in a cost-effective way on the same platform.
Determine if this approach to acquisition has any strengths or weaknesses for various purposes of the NAIP and 3DEP stakeholder communities.
Potentially increase the frequency of repeat data collection for lidar data nationally.
Encourage the innovation of more sustainable approaches to acquiring these and other national datasets as technology improves.
Explore the unique applications of co-collected imagery and lidar data.
Consider whether this technology and potential partnership between the imagery and lidar communities could support statewide lidar collections where requirements exist for both data products.
Collecting lidar data over an entire state or major watershed during the same season could offer modeling benefits due to the higher likelihood of more consistent data. Weather and surface water differences present in datasets currently collected over a span of many years might be lower in a large single-season collection. Co-collection could bring more funding stakeholders into the partnership by providing products or derivatives using both, as seen in published research on the topic.
Federal, state, and local agencies have made great strides in reducing and eliminating the duplication of effort and investments. Inter-governmental working groups, such as National Digital Orthophoto Program (NDOP), 3DEP, and the Interagency Working Group on Ocean and Coastal Mapping (IWG-OCM), have greatly succeeded in this effort by gathering requirements (for products and areas of interest) and partnering on funding on an annual and ongoing basis. If the technical data specifications can be met by this sensor and the stakeholder communities can find ways to plan areas of interest and funding, then this sensor has the potential to further reduce the duplication of flying aircraft over states to collect data. There are many scenarios in which separate acquisition flights would not be a duplication of effort, such as when the areas of interest and requirements differ substantially.
CountryMapper is a Leica hybrid system being developed with imaging and lidar operating at an altitude of 3.6 km, a speed of 180 knots, a pulse repetition frequency (PRF) of 750 KHz and with a scan rate of 112 Hz giving an imagery product of 20 cm and a pulse density 2.9 points per square meter (PPSM). Its elliptical scan pattern allows the separation of the point cloud into forward and backward scanned groups, which allows intraswath difference analysis.