Direct Measurement of Tree Height Provides Different Results on the Assessment of LiDAR Accuracy

In this study, airborne laser scanning-based and traditional field-based survey methods for tree heights estimation are assessed by using one hundred felled trees as a reference dataset. Comparisons between remote sensing and field-based methods were applied to four circular permanent plots located in the western Italian Alps and established within the Alpine Space project NewFor. Remote sensing (Airborne Laser Scanning, ALS), traditional field-based (indirect measurement, IND), and direct measurement of felled trees (DIR) methods were compared by using summary statistics, linear regression models, and variation partitioning. Our results show that tree height estimates by Airborne Laser Scanning (ALS) approximated to real heights (DIR) of felled trees. Considering the species separately, Larix decidua was the species that showed the smaller mean absolute difference (0.95 m) between remote sensing (ALS) and direct field (DIR) data, followed by Picea abies and Pinus sylvestris (1.13 m and 1.04 m, respectively). Our results cannot be generalized to ALS surveys with low pulses density (<5/m2) and with view angles far from zero (nadir). We observed that the tree heights estimation by laser scanner is closer to actual tree heights (DIR) than traditional field-based survey, and this was particularly valid for tall trees with conical shape crowns.


Introduction
Sustainable forest management needs a huge amount of tree parameters such as species distribution, timber volume, and average tree height as the basis of broad scale forest inventories [1].Among these, tree height is one of the most important variables in forest inventory, as it is often used in the estimation of forest growth, biomass, carbon stock, and site productivity [2].Traditional forest inventory field methods for forest height estimation are expensive, time consuming, and almost impossible to perform over large areas [3].When dealing with large areas, a sampling plot approach is commonly arranged in order to reduce costs, and then diameter-height relations are constructed to predict individual tree heights for those areas that were not surveyed.Using remote sensing (RS) techniques allows for the reduction of those constraints, thereby obtaining spatially continuous data over large study areas on a cost efficient basis [4].Tree height estimation from the ground is difficult due to treetop placement, which is often hidden by the branches of other trees.While with RS survey from above, the problem of hidden treetops is reduced, at least for canopy trees.Among RS techniques, the potential of Airborne Laser Scanning (ALS) in tree heights estimation has been widely demonstrated [5][6][7]; it has been shown to be useful for the estimation of other forest stand attributes such as stand volume [6] and basal area [8] as well.The impact of errors in tree height measurements has been shown to have a stronger effect on individual tree volume estimates than errors in species composition [9], but this depends on the surveyed species and the obtained height errors.For this reason, the accuracy assessment of tree height measurement derived from ALS is a very important issue in the forestry field.
The accuracy of forest attributes derived from RS data is usually assessed using traditional field surveys that can be area-based [4] or tree-based [10].Field-based surveys always take advantage of trigonometric approaches using rangefinder devices.These field-based data are commonly assumed as being the ground truth for RS data, but in actuality they are measured with errors.In spite of the huge amount of recent papers that use field-based tree height estimations as ground truth for RS data assessment [2,4], the accuracy of this indirect measurement used as reference is often not declared.
In this paper, we considered as ground truth (Ground Control Points, GCPs) one hundred felled trees to test tree heights measured with two estimation approaches: Airborne Laser Scanning (ALS) and classical field surveys.Specific goals of the paper are: (1) to evaluate the use of ALS data for tree height estimation in a comparison with traditional field-based survey method; (2) to use tree height of felled trees to assess the accuracy and bias of field-based survey method and its influence on ALS testing; (3) to study the relative influence of tree size and species on the divergence between ALS remote sensing and direct field-based estimations.

Study Area
The Cotolivier forest is about 4000 ha and is located on the right bank of the Upper Susa Valley (Piedmont, Italy), between Oulx and Bardonecchia (45 • 2 N, 6 • 46 E).It is a north facing slope ranging from the montane belt (about 1000 m a.s.l.) up to the treeline (about 2200 m a.s.l.).The mean annual temperature is 6.7 • C and the total precipitation is about 614.9 mm (Bardonecchia Pian del Sole, 1585 m a.s.l., 2004-2015 period).The topography of the area is complex with steep slopes and high roughness.Both productive and protective forests are present, with intense forest exploitation on more accessible sites, but topography limits the use of a massive mechanization.Pinus sylvestris L. stands dominate at lower elevation, with Picea abies (L.) H.Karst. and Larix decidua Mill.Prevalent at higher elevations.This mixed conifers forest is characterized by a high structural diversity: Larix stands range from wood pasture with sparse big trees on more accessible sites [11] to dense and young stands, while Pinus and Picea stands show less structural diversity.

Sampling Design and Data Collection
Four permanent circular plots with 20 m radiuses were established during a field campaign in 2013 within the "NewFor" Alpine Space project [12].Within these plots, we estimated the total height of 100 trees (28% of the surveyed trees within the plots) by using three different survey methods: (a) traditional field-based survey here defined as "indirect" measurement (IND); (b) remote sensing measurement through Airborne Laser Scanning (ALS); (c) direct measurement on felled trees, referred as "direct" measurement (DIR).

Indirect Field-Based Survey (IND)
Field-Map™ integrated system [13] was used to accomplish the traditional field-based survey that was done during June-July 2012.It includes a GPS receiver (GeoXM 2005 [14]) to obtain georeferenced data of the plot center (differential correction in laboratory with post-processing method), and a laser rangefinder (TruPulse™ 360b [15]) to measure tree heights and relative positions (distances and angles) to the plot center.A field computer (ARMOR [16]) equipped with mobile GIS software was used for real time data mapping and 3-D visualization of trees.Diameter (DBH), crown projections measure, and species have been collected for all 100 trees (DBH > 4 cm).

LiDAR Survey (ALS)
ALS data were acquired in June 2012 by a helicopter flight, which covered the whole study area.The LiDAR sensor used was an Optech™ ALTM 3100EA, that allowed a planimetric position accuracy of less than 0.30 m and an elevation accuracy (Z) of less than 0.15 m.Four echoes of each laser pulse were recorded flying at an average altitude of 400 m, resulting in an average point sampling density of 10 points/m 2 .The vendor provided the ALS data classified as ground and non-ground points using Terrascan software [17].The original data (ALS10) were thinned with the module lasthin of the LAStools software [18], obtaining two new point densities: 5 (ALS5) and 1 (ALS1) points/m 2 .This data thinning methodology is commonly adopted to test the effects of point density on forest attributes [19][20][21].Digital Terrain Model (DTM) and DSM (Digital Surface Model) of ALS10 were generated by the vendor with a grid size of 1 m using TerraScan software.The thinned ALS point clouds were used to generate raster data of the DTMs and DSMs of the area with a grid size of 1 m using the "las2dem" tool.These layers were subtracted to produce the CHM (Canopy Height Model) that was employed as a base raster for trees positions and heights detection [1,[22][23][24].The CHMs have undergone a surface smoothing using a Gaussian kernel filter [22].Then focal statistics with a local maximum detection algorithm ("Pouring" algorithm implemented in HALCON, similar to an inverted, classical watershed-algorithm [1]) were used to extract potential treetops from the smoothed CHMs.The detected trees were then analyzed through a conditional script that considers a minimum distance and height difference from the nearest trees in order to identify and delete potential false positives [24].The method is fully automated and has been implemented as a workflow of geoprocessing tools (Model Builder) within the software ESRI ArcGIS™ [25].The output is a shapefile containing tree position and height [20].We manually co-registered the remote sensing data to the field dataset by using a very high spatial resolution (20 cm) aerial image acquired a month before the ALS data in order to correctly match the trees detected remotely to each GPS-positioned tree [24,26].False negatives have been consequently detected by experienced human interpreter [22].Only correctly assigned trees (true positives) were used in this work.

Direct Field-Based Survey (DIR)
The 100 felled trees were the result of two different harvesting methods: A clearcut on an almost pure L. decidua stand (plot 1) and thinning in multilayered mixed conifer stands (plots 2, 3, 4) (Table 1).Tree length plus the stump height was measured for each felled tree to get "direct" data, which have been considered in the analysis process as ground truth.

Data Analysis
We obtained five datasets that were used to compare remote sensing ALS data with three different point densities (ALS10, ALS5, and ALS1) and field-based estimation methods (IND) by using direct measurements on felled trees (DIR) as ground control measure.We tested the statistical difference between the five measurements by using a Kruskal-Wallis chi-squared non-parametric test.We calculated standard summary statistics of tree height measures for both the whole sample and the three main species (Larix, Picea, Pinus) separately.The accuracy of tree heights as measured by ALS data was assessed by mean differences (MD) and root-mean-square errors (RMSE) between IND-ALS and DIR-ALS for the whole dataset and divided by species.We performed a regression analysis between ALS and IND data against our ground truth (DIR), again both for the entire sample and for each species separately to further investigate the observed differences.Then we regressed heights from the two field datasets (IND and DIR) against each other.We applied the hierarchical partitioning approach [27] through the statistical package hier.part[28] to calculate the independent contribution of tree attributes (real height, DBH, crown surface area and species) in determining the difference in absolute value between DIR and ALS estimates.Because of a non-uniform distribution of species along the range of tree heights, the hierarchical partitioning was applied to the whole dataset and to the species separately.Summary statistics, regression analyses, and hierarchical partitioning were performed in R software [29].

Results
L. decidua was the tallest species in our dataset followed by P. abies and P. silvestris (Figure 1), and some differences between estimation methods existed.However, the five estimation methods were not statistically different from each other (Kruskal-Wallis test, p = 0.92).Data thinning on the original ALS10 data had a strong effect on the number of the detected trees.Almost half (56) of the 100 felled trees were detected with ALS1 and 73 with ALS5 (Table 2).The more affected species was P. sylvestris (38% with ALS1) and the less influenced was P. abies (68% with ALS1).From a comparison between mean tree heights measured by ALS data and IND (Table 3), emerged that ALS10 slightly overestimated tree heights (23.51 m vs. 23.44 m) and the opposite trend was observed for Picea (19.64 m vs. 20.26m).Considering the mean tree heights of the whole dataset (All) an overestimation was obtained by using the ALS1 point cloud, this trend was particularly evident for Picea and Pinus species (Table 3).A comparison between ALS10 and real tree heights (DIR) revealed that the mean value was identical (23.51 m), but some differences emerged for Picea and Pinus species.The biggest difference between ALS10 and DIR height average measurements was observed for Pinus (0.58 m) and Picea (−0.33 m), but considering the difference between ALS1 and DIR those values increased (Pinus 1.18 m and Picea 0.55 m).Every model obtained through regression analysis showed an overestimation of small trees and an underestimation of the taller ones.The differences between the models lie in the magnitude of these over/underestimates.All the regressions were significant (p < 0.001) and every model gave very good fit if we consider the population as a whole (R 2 > 0.9).In the ALS10 vs. IND models (Figure 2) P. sylvestris achieved a poorer fit (R 2 = 0.67), while L. decidua achieved a better fit (R 2 = 0.73) and Every model obtained through regression analysis showed an overestimation of small trees and an underestimation of the taller ones.The differences between the models lie in the magnitude of these over/underestimates.All the regressions were significant (p < 0.001) and every model gave very good fit if we consider the population as a whole (R 2 > 0.9).In the ALS10 vs. IND models (Figure 2) P. sylvestris achieved a poorer fit (R 2 = 0.67), while L. decidua achieved a better fit (R 2 = 0.73) and P. abies emerged as being the best fitting species (R 2 = 0.95).All P. abies trees were underestimated by ALS as well as L. decidua taller trees (Figure 2).A similar pattern was observed in the ALS10 vs. DIR model, but with less underestimation (Figure 3a).In the ALS10 vs. DIR model (Figure 3a) among the studied species P. sylvestris achieved a poorer fit (R 2 = 0.69), while P. abies and L. decidua performed better (R 2 = 0.79 and 0.81, respectively).In the regression between DIR and IND measurements (Figure 3b) P. sylvestris and P. abies were the best fitting species (R 2 = 0.80) followed by L. decidua (R 2 = 0.71).In this model L. decidua was underestimated IND, instead P. sylvestris and P. abies showed a high correspondence between the two measurements (Figure 3b).P. abies emerged as being the best fitting species (R 2 = 0.95).All P. abies trees were underestimated by ALS as well as L. decidua taller trees (Figure 2).A similar pattern was observed in the ALS10 vs. DIR model, but with less underestimation (Figure 3a).In the ALS10 vs. DIR model (Figure 3a) among the studied species P. sylvestris achieved a poorer fit (R 2 = 0.69), while P. abies and L. decidua performed better (R 2 = 0.79 and 0.81, respectively).In the regression between DIR and IND measurements (Figure 3b) P. sylvestris and P. abies were the best fitting species (R 2 = 0.80) followed by L. decidua (R 2 = 0.71).In this model L. decidua was underestimated by IND, instead P. sylvestris and P. abies showed a high correspondence between the two measurements (Figure 3b).The error in the individual tree height estimation was higher (RMSE = 1 m) when ALS was tested against IND (Table 4) than in the case of DIR (RMSE = 0.76 m, Table 5).Overall, RMSE increased with thinned ALS1 data both by using IND and DIR data as ground truth, and this tendency was particularly evident for Picea and Pinus.The accuracy assessment between IND and DIR measurements gave an RMSE of 1.08 for All trees, 1.84 for Larix, 1.34 for Picea, and 0.94 for Pinus.The error in the individual tree height estimation was higher (RMSE = 1 m) when ALS was tested against IND (Table 4) than in the case of DIR (RMSE = 0.76 m, Table 5).Overall, RMSE increased with thinned ALS1 data both by using IND and DIR data as ground truth, and this tendency was particularly evident for Picea and Pinus.The accuracy assessment between IND and DIR measurements gave an RMSE of 1.08 for All trees, 1.84 for Larix, 1.34 for Picea, and 0.94 for Pinus.In spite of the small size of the sample, we observed that tree height estimation by laser scanner is closer to real tree heights (DIR) than traditional field-based survey.This was particularly valid for tall trees having regular crowns, and this has important implications for the use of ALS techniques in forest inventories [9,37].We are aware that our results cannot be generalized to ALS surveys with low pulses density (<5/m 2 ) and with view angles far from zero (nadir).Pulse density is a limit for singletree height estimation [32], but is not a constraint at the plot scale until pulse densities drop below 1 pulse/m 2 [38].From our experiment on thinned data we observed that pulse density had a strong influence on the number of detected trees.With a density of 1 pulse/m 2 only 56% of trees were detected and this percentage was even lower (38%) for Pinus trees.This is probably because the point cloud thinning affected shorter trees with complex crown shape.Regarding the influence of point density on the accuracy of tree height estimation we obtained higher RMSE (1.41 m) with low densities (1 pulse/m 2 ), but this value is not too far from those indicated in the literature for the same species with 10 pulses/m 2 (RMSE = 0.98 m) and 4 pulses/m 2 (RMSE = 0.63 m) [10,39].
The source of error on tree height field measurement can be related to the instrument used, to the skills and experience of surveyor, and to the tree and stand characteristics [40].There are only a few papers that compare tree height measurements obtained by handled instruments with actual height obtained by direct measurement [41], but as far as we know there are none comparing LiDAR derived height with direct measurements.Field-based surveying methods can influence the accuracy of tree height estimation and usually are affected by lack of precision [42,43].The model of handheld laser rangefinder (e.g., Impulse or Truepulse) affects the quality of compass measurements [44,45].In our study, we used a Truepulse 360B device that is affected by a length measurement error of 30 cm and azimuth error of 3 degrees [45].When using Suunto hypsometers the standard error in tree height estimation has proven to be between 0.4 and 0.8 m [46,47].Moreover, field crew judgement and precision is another important factor affecting the accuracy of field-based surveys with handheld laser rangefinders [48].The use of tacheometer, total station, and accurate GPS receiver can significantly improve field measurements, and thus the accuracy assessment of ALS data [47,49,50].Other authors have found that field methods were more accurate than LiDAR data [2,41], but point density was almost half of the present study.In any case, the authors agreed in considering that the discrepancy in height accuracy was greatly compensated by the higher cost efficiencies and wider coverage guaranteed by LiDAR.Further case study, species, instruments, and field crew should be tested in order to have a clearer idea of the issue.

Conclusions
Our study demonstrates that direct field measurements on felled trees improve the assessment of small-footprint airborne laser scanner (ALS) estimation of tree heights.Adopting direct measurements as ground truth for remote sensing tree height estimation we obtained a reduction in the magnitude of errors (RMSE).The proposed method of using felled trees as ground control points is time consuming and expensive, but highlights the fact that traditional field-based surveys methods are estimations in turn.ALS can be considered reliable for height estimation of dominant trees, but its accuracy with shorter trees having irregular canopy is lower.A solution could be to improve the number of GCPs and the number of repeated field-based measurements on dominated trees to obtain a more reliable assessment of remote sensing data.The use of felled trees as direct field measurements in forest inventory obviously cannot be adopted as a routine approach.Traditional field-based measurements should still be considered an essential survey methodology, but an increased awareness of their accuracy, especially when used for the validation of other indirect measurements such as RS methods, should be promoted.

Figure 1 .
Figure 1.Box plots of tree heights distribution as measured by three survey methods (DIR = field direct measurement of felled trees; IND = traditional field-based; ALS = Airborne Laser Scanning) of the whole dataset and considering the species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.

Figure 1 .
Figure 1.Box plots of tree heights distribution as measured by three survey methods (DIR = field direct measurement of felled trees; IND = traditional field-based; ALS = Airborne Laser Scanning) of the whole dataset and considering the species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.

Figure 2 .
Figure 2. Regression analysis between tree height derived by airborne laser scanner data (ALS = ALS10) and tree height derived from indirect field-based measurement (IND), considering the whole dataset (All) and the three species separately.Each dot corresponds to a single tree (blue square = Larix; green triangle = Picea; red circle = Pinus).

Figure 2 .
Figure 2. Regression analysis between tree height derived by airborne laser scanner data (ALS = ALS10) and tree height derived from indirect field-based measurement (IND), considering the whole dataset (All) and the three species separately.Each dot corresponds to a single tree (blue square = Larix; green triangle = Picea; red circle = Pinus).

Figure 3 .
Figure 3. Regression analysis between (a) tree heights derived by airborne laser scanner data (ALS = ALS10) and tree heights derived from direct field-based measurement (DIR); (b) tree heights derived from indirect field-based measurement (IND) and DIR, considering the whole dataset (All) and the three species separately.Each dot corresponds to a single tree (blue square Larix; green triangle = Picea; red circle = Pinus).

Figure 3 .
Figure 3. Regression analysis between (a) tree heights derived by airborne laser scanner data (ALS = ALS10) and tree heights derived from direct field-based measurement (DIR); (b) tree heights derived from indirect field-based measurement (IND) and DIR, considering the whole dataset (All) and the three species separately.Each dot corresponds to a single tree (blue square = Larix; green triangle = Picea; red circle = Pinus).

Table 1 .
Site characteristics, stand structure descriptors, and species composition of the four permanent plots as a whole and considering only the felled trees.Tree heights were obtained from DIR measurement, instead diameter at breast height (DBH), basal area (BA), bole height (Crown H), and crown area (Crown A) were obtained from IND survey method.

Table 2 .
Total number (N) and percent value of detected trees by ALS from the original dataset of felled trees.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.

Table 3 .
Average (Mean) and standard deviation (SD) values of tree heights measured by three estimation methods: field direct (DIR), indirect (IND), and Airborne Laser Scanning (ALS) for the whole dataset (All) and divided by species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.N indicates the total of detected trees by ALS from the original dataset of felled trees.

Table 3 .
Average (Mean) and standard deviation (SD) values of tree heights measured by three estimation methods: field direct (DIR), indirect (IND), and Airborne Laser Scanning (ALS) for the whole dataset (All) and divided by species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.N indicates the total of detected trees by ALS from the original dataset of felled trees.

Table 4 .
Mean difference (MD) and Root mean square error (RMSE) between field indirect (IND) and ALS data for the whole dataset (All) and divided by species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.

Table 4 .
Mean difference (MD) and Root mean square error (RMSE) between field indirect (IND) and ALS data for the whole dataset (All) and divided by species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.

Table 5 .
Mean difference (MD) and Root mean square error (RMSE) between field direct (DIR) and ALS data for the whole dataset (All) and divided by species.ALS data are presented divided by pulse densities (10, 5, 1 points/m 2 ) obtained through point cloud thinning.