1. Introduction
Development of remote sensing technologies have facilitated the ability to rapidly capture 3D data containing vegetation structural information [
1,
2]. Accurate measurement of vegetation at different spatial and temporal scales is critical in informing a range of ecological and natural resource management disciplines [
3,
4,
5]. Estimates of vegetation characteristics such as canopy height, canopy cover, diameter at breast height, and above-ground biomass have been identified as important metrics for carbon accounting, wildlife habitat diversity, precision forestry, fire behaviour modelling and understory forest dynamics for eco-hydrological monitoring [
6,
7,
8,
9,
10].
Data capture from above the canopy (satellite, aircraft or drone) is the most common approach to characterising the 3D structure of forests [
11,
12,
13,
14]. Known limitations in observing detailed below canopy structure from airborne sensors have led to investigation into terrestrial data capture [
7,
15,
16]. The two methods that have been used previously for capturing 3D information describing forests are laser scanning and image-based point clouds [
17]. Terrestrial Laser Scanning (TLS) is a ground based-remote sensing technique which utilises the time for a laser pulse emitted from the sensor to reflect off a target and return to the sensor to provide a precise range to that target [
18]. Combining this range information with the direction the laser was emitted allows for the 3D location of the target to be determined relative to the sensor [
18]. The speed of operation of modern TLS allows for multiple points to be collected rapidly and accurately [
19]. TLS point clouds have previously been used in the assessment of below-canopy forest characteristics in particular for examining stem properties; diameter at breast height, stem curve and species and allometry biomass [
18,
19,
20,
21]. More recently, fine scale vegetation characteristics have also been observed in the context of estimating biomass through allometry [
1,
16,
22,
23].
Structure from Motion (SfM) is an image based technique which observes the same features, in multiple images, from different viewpoints to build a point cloud [
24]. Prior research has utilised SfM point clouds derived from terrestrial and airborne images to estimate tree-centric characteristics such as canopy height and cover, diameter at breast height, and stem count [
14]. Preliminary research using terrestrial SfM point clouds for measuring below canopy vegetation biomass has demonstrated strong correlation to direct measurements, visual assessments, and TLS point clouds in pasture and grassy dry forest environments [
1,
17,
22,
25,
26,
27]. However, limited research has been conducted to investigate the potential of these point clouds to represent fine-scale vegetation structure between the top intercept height and ground surface.
With prior studies demonstrating potential for terrestrial point clouds for measuring below canopy vegetation, demonstration of the absolute accuracy and correlation to existing methods is seen as an important consideration for adoption on a wider-scale. Validation of point clouds have been largely focused on visual assessment and destructive harvesting to compare biomass estimates [
1,
16,
19,
22]. Whilst considered to be the most accurate form of volume assessment, manual collection and weighing is time-consuming, labour intensive and prohibits repeat-sampling [
28,
29]. Such approaches also only provide an indication of the vegetation volume across the measured area and need to be combined with other measurements to provide an indication of the vertical distribution of this volume [
30,
31]. Whilst being quick to execute, visual assessments have proven to be subjective and vary substantially between assessors in multiple environmental applications [
25,
32,
33,
34,
35].
Studies that have directly validated the representation of vegetation structure contained within terrestrial point clouds have utilised the point-intercept method [
23,
28,
36,
37,
38]. The point-intercept method involves the use of a wooden or metal dowel placed at randomly or regular spaced intervals, with the height and type of vegetation measured at that point. This method of assessment has been used extensively for the measurement of vegetation characteristics in ecological studies [
39,
40,
41,
42] and shown to be highly correlated with the estimation of plant biomass [
43]. A difficulty in using this technique is co-registering the point intercept observations to the point cloud for direct comparison. To achieve this several approaches have been trialled; Bright et al. [
38], for example, used markers placed on the ground to establish a coordinate system. Whilst this approach is suitable in areas of low vegetation, accurately locating a point intercept sample in high vegetation cover would be difficult. Loudermilk et al. [
28] and Hiers et al. [
37] used styrofoam balls to co-register TLS point clouds with manual field measurements. Whilst this may be suitable for volumetric calculations utilising top height, characterisation of the structural content would be challenging to make due to the variation in coordinate systems.
The use of laser pointers have been suggested to provide an alternative to using metal dowels to measure cover especially in studies where the phenomenon of interest is above head height [
44,
45,
46,
47]. Whilst this technique may provide a less destructive method of measuring vegetation top height due to minimal disturbance of the vegetation, obstruction from vegetation makes the use of lasers to measure vegetation impractical for measuring height of a complex surface and/or near-surface vegetation structures. The point-intercept method in combination with an appropriate sampling density has demonstrated to be an accurate form of vegetation structure assessment for measuring defined vegetation elements.
The aforementioned studies demonstrate the capability of point cloud capture to provide a representative description of biomass volume relative to destructive sampled weight. What these approaches do not provide is an indication of the quality of the representation of the overall vegetation structure in the point cloud data. Limited research has been conducted in applying methods which directly validate vegetation structural characteristics contained in point clouds.
The primary aim of this manuscript is to develop a method that allows the representation of understory vegetation structure contained within point clouds to be assessed. Two case studies were used to evaluate the performance of the method. The first case study highlighted the effect of the frame on the point cloud. The second case study highlighted how the methodology can be applied at sites that have a range of cover and height attributes. It is shown that the validation frame measurements provide a rigorous assessment for the relative performance of terrestrial point clouds generated using both TLS and SfM technology. The implications of the frame measurements to be used as an indicator for point cloud completeness at multiple stages of the point cloud processing workflow and its potential use across a wider range of environments are also discussed.
2. Materials and Method
2.1. Validation Frame Design
One of the primary obstacles in the validation of point clouds based on data primitives (such as intercepts derived from the point intercept method) is the co-registration of the two sources of data. To achieve an accurate co-registration of validation and point cloud information, a frame was designed to register point intercept locations and allow point cloud information to be transformed into the same coordinate space.
The frame consisted of a 0.5 m × 0.5 m quadrat on four variable height legs (
Figure 1). The size of the frame developed in this study was chosen to align with similar practices in other ecological studies [
48,
49,
50]. The origin of the coordinate system was the lower-left inner corner of the quadrat frame. The variable height legs allowed the frame to be levelled using a spirit level bubble approximately 1 m from the ground. A sliding rack was used to allow the collection of 8 point intercept measurements simultaneously along the
x-axis. This rack could be located at eight locations along the
y-axis, allowing a total of 64 measurement points to be measured at known locations within the coordinate system defined by the frame.
At each measurement point, an aluminum rod of 0.6 cm diameter marked at 1 cm intervals was inserted into the rack to ground level. The rack was designed with two vertical bars spaced 5 cm apart to ensure when inserted the rods remain closely aligned to the
z-axis with a maximum deviation of 2 cm. For each rod, the method outlined in [
44] for taking point intercept measurements can be followed. This involved noting the height above ground, type, and width of any vegetation elements intercepting the rod. The height of the rod at the bottom of the second vertical bar was also observed to determine the location of the ground within the frame’s coordinate system.
2.2. Point Cloud Collection
The in-field use of the validation frame has been designed to work independently from the point cloud technology (i.e., TLS or SfM). As such, point clouds can be collected in the desired form with the frame placed in the area of observation at the time of capture. Following the point cloud capture, the point intercept reference measurements were taken. As rods are removed to change rack positions, minor disturbances to the vegetation may occur. The adopted best practice was therefore to capture the reference measurements at the end of the survey.
2.3. Co-Registration of Validation Measurements and Point Clouds
This section describes how SfM and TLS point clouds were co-registered to the validation frame. These two approaches provided an indication as to how point clouds captured using emerging technologies can be co-registered to the frame. For TLS data, scale is an inherent property of the point cloud, thus, in this case, a six-parameter transformation (three translations and three rotations) of the point cloud relative to the frame was defined based on a set of common points. The common points between the two coordinate systems were defined by manual location of the outside edge of the quadrat in the TLS point cloud.
For image based point clouds, point cloud information can be transformed to the frame coordinate system using known locations on the frame as control targets. This is achieved by locating each corner of the frame on four or more images. Once added, the 3D position is estimated in the arbitrary coordinate space using the SfM solution. A seven-parameter transformation (three translations, three rotations and one scale term) is then determined and applied to the point cloud.
2.4. Voxelisation
To directly compare the reference data to the point cloud in this study, each dataset was transformed into a binary voxel model indicating vegetation presence and absence. A voxel model for these purposes is a three-dimensional grid domain with the size of the grid cells determining the resolution of the 3D grid [
51].
For the reference data, the vertical resolution of the data can be considered to be 10 mm (due to the marks on the rod), while the horizontal resolution was determined to be 20 mm to allow for any slight deviations of the rods from z-axis. The conversion of the reference information to voxel space can achieved by indicating if an intersection occurred within a voxel. For the point cloud information, a similar conversion was achieved by counting the number points that fall within the 3D space defined by each voxel, the voxel can then be considered filled if the point count is greater than a count threshold.
2.5. Metric Determination and Validation
The validation of the representation of vegetation within the point cloud can be achieved both on the raw data and on secondary metrics (such as vegetation cover and vegetation top height) based on the frame measurements. For validating the raw point cloud information, the presence or absence of voxel data can be directly compared. By determining the number of true and false positive and negatives, the completeness of the point cloud representation can be evaluated. A True Positive (TP) result indicates a point that was present in both the reference measurement and point cloud. A True Negative (TN) result highlights that a point was not present in both the point cloud and reference measurements. A False Negative (FN) indicates the presence of a reference measurement with no corresponding filled voxel. A False Positive (FP) indicates the presence of a filled voxel that does not correspond with a reference measurement. As there is likely to be a higher number of empty voxels in most environments, a Matthew Correlation Coefficient (MCC) was utilised to provide a measure of classification accuracy. This measure has been used previously in the application of point cloud analysis [
52,
53] to compare observed and predicted binary classifications where the classes are imbalanced [
54]. The properties of the intercept can also be examined to determine the type and location of missing information in the point cloud. The error in intercept height can then be summarised using measures of difference such as Root Mean Square Error (RMSE) or Mean Absolute Deviation.
The first step in extracting measurements of a vegetated scene from a point cloud is often the detection of points originating from the ground and the construction of a terrain model. The frame allows the accuracy of such ground definitions derived from point clouds to be assessed by comparing the terrain height at each measurement point to the frame measured height. Following the extraction of the ground, the point cloud can be normalised, which is achieved by subtracting the point’s raw height from the ground height at the location of the point. This provides the above-ground height of the vegetation. The accuracy of vegetation height is determined by comparing the normalised height from the point cloud, with a similar measure from the frame data. Point cloud data can also be normalised based on the definition of the ground provided by the frame measurements, allowing the influence of poor ground definition on vegetation metrics to be fully understood. Height metrics such as vegetation top height and vertical profiles can all be validated using this method. Similarly, percentage vegetation cover can be estimated based on the normalised point cloud. For instance, cover can be calculated based on the number of measurement points with an intercept or filled voxel above a height threshold.
2.6. Case Studies
Two independent case studies were conducted to assess the effectiveness of the validation frame in capturing structural information on fine scale vegetation. The first case study was designed to assess the impact of the validation frame on the resultant point cloud representation. This was done to ensure the validation approach did not bias the outcomes achievable from the point cloud technology. The second case study tested the validation frame in a variety of landscapes to assess the effectiveness of the frame at validating terrestrial point clouds in a variety of landscapes. This section first outlines the sensors used in both case studies, before providing details on plot setup, data capture and analysis, and closes with a discussion of the results undertaken within the case study.
2.6.1. Sensors
TLS Data Collection
A Trimble TX8 scanner set to capture level two quality scans (resulting in a 11.3 mm point resolution at a distance of 30 m) was utilised for both case studies. For Case Study 1, four laser scans were captured approximately 3 m from the frame in order to specifically observe the effects of the frame on point cloud re-construction. For Case Study 2, the plots were scanned using the method outlined by Schaefer et al. [
55], which involved collecting a central scan and four orthogonally located exterior scans taken approximately 10 m from the plot centre. Scans took approximately 3 min to capture and were co-registered using Trimble RealWorks 10.1, with automated alignment utilising the inbuilt cloud to cloud matching routine. Manual quality checking was conducted post-registration and adjustments were then undertaken if required.
Image Based Data Collection
Images were collected using two consumer grade mirrorless cameras: an Olympus OM-D EM-10 camera and Sony Alpha 6000. Images were captured at approximately 2 m from the reference frame centre with a horizontal spacing or offset between each image of 0.3 m. This resulted in between 30 and 40 images being taken of the area within and around the reference frame. The height of the image capture varied: 1.0 m, 1.4 m and 1.8 m. Six downward looking images over the top of the frame were also captured to complete the image acquisition process.
Images were collated and processed to form a point cloud using Agisoft Photoscan 1.4.0 (Agisoft LLC, Moscow, Russia). Images were aligned to generate a sparse point cloud using the high quality setting. The high quality setting and mild depth filtering was then applied to generate a dense point cloud. Point clouds were then clipped to represent the area of the reference frame with an approximate 1 m buffer.
2.6.2. Case Study 1: Effect of Frame on Point Cloud
The purpose of the first case study was to determine whether the validation frame had any influence on the 3D reconstruction of vegetated scene when using either TLS or SfM techniques. To achieve this, scans and images were captured of two plots with understory vegetation in a built environment. The first plot had vegetation of approximately 0.1 m in height and covered 50% of the validation frame area. The second plot represented a more complex environment with the height of the vegetation approximately 0.5 m and covered 80% of the validation frame area. To fully validate the impact of the frame on representing vegetation, images and scans were captured firstly without the frame, then with the frame present, and finally with the frame removed again. Care was taken when placing and removing the frame to reduce disturbance to the vegetation inside the plot.
Point Cloud Co-Registration
When the frame was placed in the scene, point clouds were co-registered and transformed into voxel space using the approach outlined in
Section 2.3. Without the frame in the scene, discrete features were identified in the already registered point clouds and used to provide the correct registration. To achieve this, the plots were carefully selected in built environments to provide an abundance discrete targets (brick corners, stairs and seat edges) identifiable within both the TLS and SfM point clouds.
Analysis
For assessing the impact on the presence of the validation frame on the point cloud, the area enclosed by the frame was extracted from point clouds captured both with and without the frame present. An assessment on the point cloud completeness was then conducted by comparing the point cloud with frame in place with the two point clouds without the frame. The analysis method outlined in
Section 2.5 was then followed; however, in this case, the point cloud with the frame from each technology was used as the reference source. This allowed the differences in the point clouds with and without the frame to be quantified.
2.6.3. Case Study 2: Validation
The purpose of the second case study was to implement the validation methodology developed in this paper, and to assess the effectiveness of SfM and TLS derived point clouds in detecting vegetation components at a fine scale.
The study was undertaken within Australian forest landscapes. Sites located in Australia were chosen to provide a range of cover and structural characteristics for assessment in southeastern Australia.
Site Descriptions
The validation frame was tested in multiple environments tested to examine the frame’s ability to measure vegetation structure. The High Camp study site is situated in a heath dry forest near Kilmore, Victoria, Australia, approximately 80 km north of Melbourne (
Figure 2). The overstory largely consisted of low, open eucalypt forest with an open crown cover. The understory consisted of sparse ericoid-leaved shrubs including heaths and peas. There was minimal surface litter and a range of exposed rock sections creating variable terrain.
Royal Park is an urban parkland environment located in Parkville, Vic, Australia, 3 km north of Melbourne’s CBD (
Figure 2). The overstory vegetation consisted of sparse Eucalypt trees, approximately 15 m tall. The understory consisted of a surface layer of coarse woody chips (e.g., tan bark) with some larger woody debris. Near-surface vegetation consisted of native shrubs with complex leaf arrangement.
The Silvan1 site is located in moist forest near Monbulk, Vic, Australia, approximately 40 km to the east of Melbourne (
Figure 2). This plot was located less than 50 m from Silvan Reservoir. There was also evidence of a planned burn occurring in the previous 5–10 years. The overstory largely consisted of medium to tall forests dominated by
Eucalyptus obliqua and
Eucalyptus radiata. The understory vegetation was dominated by native shrubs. The surface litter was measured at approximately 25 mm in height and consisted mostly of leaf litter with some woody debris. Near-surface vegetation was dominated by native shrubs.
The Silvan2 site is similarly located to Silvan1, in moist forest near Monbulk, VIC, Australia (
Figure 2). This plot was located at a higher elevation in comparison to the Silvan1 plot with no evidence of previous planned burn events. Increase in elevation along with a change in aspect produced varied vegetation to Silvan1. The overstory largely consisted of medium to tall forest dominated by
Eucalyptus obliqua and
Eucalyptus radiata. Understory vegetation was dense, with a high proportion of Aristia species. Surface litter was measured at approximately 25 mm and consisted mostly of leaf litter with some woody debris. Near-surface vegetation was dominated by native shrubs and aristida.
The Ridgeway site was located in dry sclerophyll forest near Hobart, Tasmania, Australia (
Figure 2). Evidence of a planned burn event (completed in 2015/2016) season was evident throughout the plot. The overstory of the plot consisted of
Eucalyptus pulchella trees of varying age and ranging in height from 4.7 to 16.2 m. The understory of the plot consisted of low to medium height (0.5–2 m) shrubs and grasses. Surface litter mainly consisted of leafs and woody debris.
Plot Set-Out
For all sites, at least one circular plot of 10 m radius was established. The centre of each plot was marked with a star picket or wooden stake. For each plot, two transects were established in opposing orthogonal directions. Along each transect, two validation frames were randomly located using random numbers chosen within 0–9 m and 11–20 m.
Point Cloud Filtering
Point clouds generated using both SfM and TLS had filtering algorithms applied to reduce noise beneath the ground in the case of SfM, and to remove spurious points in the point cloud produced from the TLS scans [
19,
26].
Point Cloud Normalisation
Ground points were identified in both TLS and SfM point clouds using the Cloth Simulation Filter [
56]. The settings for extracting the ground were optimised to obtain a set of parameters that would produce a ground surface across all plots and minimise RMSE against reference ground measurements (
Table 1). A 0.02 m DTM was generated by constructing a Triangular Irregular Network (TIN) from the identified ground points. The height of the TIN facet at the centre of each cell is attributed to the DTM. TIN interpolation was used to ensure each point intercept sample had an associated ground height estimate. Next, the point cloud was normalised based on each point’s height above the TIN. This normalisation procedure provided a representation of the point cloud in relation to the ground.
Data Analysis
To assess the performance of TLS and SfM in comparison to the reference measurements, the point clouds were assessed for their ability to represent the first intercept (highest vegetation intercept). This assessment of point cloud completeness was undertaken on non-normalised point clouds to mitigate the influence of errors introduced due to the ground height estimates. Subsequently, all intercepts measured during the validation process were compared to the respective point clouds. Vegetation height and cover were calculated from both the reference measurement and derived voxel spaces. Cover is defined as the proportion of the validation frame area that has vegetation above 10 cm. In the context of the validation frame, this is defined as the number of rods for which one or more intercept was recorded above 10 cm divided by 64. Difference in cover were summarised as a single difference value for each frame, while differences in first intercept height within each frame were summarised using RMSE calculated as follows:
where
is the first intercept (or ground) measured from the frame and
is the first intercept (or ground height) at the same location measured using the point cloud.
An assessment of the TLS and SfM point cloud’s ability to measure the ground was also made and summarised using RMSE. Assessments of cover and ground were completed in absolute terms, and on the normalised voxel space. This allowed for identification of vegetation elements (absolute) and the effect of determining a ground surface in altering the vegetation representation in the point cloud (normalised).
4. Discussion
The validation frame developed in this research provides a portable, robust device to intensively assess the accuracy of point clouds for describing vegetation structure. The method provides a rigorous means for validating the information content within point clouds. Previously, validation approaches for point clouds have not been able to directly compare the representation provided by 3D remote sensing technology without utilising a indirect measure, such as biomass, mean height or 2D cover [
1,
19,
22,
28].
The method presented here allows validation of the data through the full structure of the point cloud itself, not just the top height of vegetation. The measurements from the validation frame can be used at varying stages of the point-cloud processing workflow. The second case study presented here highlights that validation with the frame provides insight into the vegetation structure present while also allowing errors in other metrics to be quantified. For example, the effect of different ground filters can be assessed for the most accurate estimation. Previously, the effect of high frequency changes in the terrain has not been explored from these types of point clouds. The frame allows the terrain to be validated at 5 cm intervals and the effect of error in subsequent measurements or models (for example, vegetation height, litter bed depth or biomass) to be evaluated. This differentiates the approach from previous studies, which have only been able to hypothesise as to the causes of the unexplained variance when validating using biomass or visual estimates [
22,
26,
27].
The validation frame can provide insight into potential errors in ecological metric calculations. For example, missing information as seen in sites with complex vertical structure (Silvan1 and Silvan2) was demonstrated to be due to wind-driven errors in fine vegetation with small mass, and by occlusion from vegetation in the frame or surrounding vegetation. This information detected in the validation process then allows the determination of the importance of the missing information in relation to the desired metric calculation. This extra information may allow alternative processing strategies, such as the extraction of metrics that are not reliant on the ground to be used or corrections to be applied to the estimate of some metrics [
57].
The frame and method developed provides a non-destructive approach for validating point clouds. The results from Case Study 1 show that the environment being sampled has minimal change with the addition of the validation frame. This is in contrast to direct forms of assessment such as destructive sampling of biomass, which removes all vegetation and prohibits repeat sampling and change detection to occur. There is greater certainty in the alignment of the reference measurements with the point cloud can as the image capture and laser scans are conducted with the validation frame in-situ.
Current visual assessments do not provide an accurate, objective quantification of vegetation structure [
25,
32,
33]. The validation frame establishes a coordinate system allowing direct analysis of point clouds and the information content below the first intercept, including the vegetation structure and ground measurement. To achieve this, care must be taken when positioning the vertical rods to ensure minimal disturbance of the vegetation. This process was difficult in environments with fine scale vegetation (less than 0.01 m) which were susceptible to wind and also minor disturbances from the measurement rods. An alternative sampling approach using a vertically sliding frame and data collection to record presence and absence of vegetation within 10 cm × 10 cm × 10 cm voxels with biomass harvesting could be utilised to allow for some movement of vegetation through wind at the trade-off of precision [
58]. Additionally, the time taken to complete a full assessment of the frame area including the harvesting and subsequent drying of biomass is expected to be more time consuming than the approach outlined in this research.
In this study, we chose a frame size that is commensurate with recent studies utilising biomass harvesting as a source of information [
48,
49,
50]. Nevertheless, the simple frame design allows the approach to be adaptable to different scales and resolutions (or density of horizontal measurements). For larger areas, distributing these frames throughout the captured area would allow the potential uncertainty in understory vegetation measurements from point clouds to be better understood in that environment. In such examples, the number of frames and their placement would be dependent on the vegetation community being sampled, the point cloud sensor sampling approach and the area being captured. This allows the approach to be applied beyond the environments presented in this paper and adapted depending upon the purpose of validation and metric generation.