2D Photogrammetry as a Forwarder Load Measurement Technique

Abstract

Research Highlights: In this study, we present the use of time-lapse photos as a way to estimate the height of the load on the forwarders. This approach, using nonprofessional outdoor cameras, is a cheap and time-effective solution for continuous load height measurements, and it offers at least the same accuracy as a gauge measurement. This method represents another approach to the automation of time studies in forestry. Background and Objectives: Time studies require information about the load on the forwarders. Until now, this information was obtained either by using a gauge measure, sampling of the load, or averaging the load from large area datasets. More accurate methods like laser scanning are costly and fragile. During time study preparations, we suggested a robust system of measuring the load height and tested it against the commonly used gauge measuring technique. Materials and Methods: Two cameras took pictures of the load; these photos were processed for camera lens distortion and rectified into the cartesian coordinate system, and the height of the load was calculated. These values were then tested against gauge measured values using paired t-test. Results: Straight line distance calculated from the images and the gauge-measured distance did not show a significant difference (p-value 0.9354). Calculated vertical distance was, however, significantly different from the calculated straight-line distance (p-value of 0.0015), suggesting possible bias of the gauge measured distance. The root mean square error (RMSE) of the rectification process was, on average, 0.42 cm. Conclusions: The proposed method was verified to correspond with the gauge measure method; however, our research raised the question of the gauge method reliability, as the taken measurements are not perfectly vertical, and for the correct load estimation, the vertical distance is needed. We, therefore, conclude that for this photogrammetry method, the vertical, rather than straight-line, distance should be used. The presented solution can be used for long-term data collection without interrupting the whole forwarding process for taking the load measurement. The longer data processing in office enables researchers to spend less time in the field taking hand measurements.

1. Introduction

In forestry, timber production is, in most cases, the primary source of income. As such, cost optimization and creating time standards are crucial for planning and logistics [1,2]. Maximum utilization of the machines is necessary for profitable logging operations [3]. The cut-to-Length (CTL) method is frequently used, and in the Czech Republic, this method is used for approximately 29–38% of the total annual timber production [2]. In the European perspective, the use of CTL and subsequent use of harvesters and forwarders varies [4]. Forwarders can also be deployed separately from harvesters to extract timber after motor-manual felling [5].

As such, there are several studies regarding the productivity of forwarders. Either they are a detailed modelling of the forwarding cycles [3,6,7,8,9], or other, less detailed, follow-up studies [10,11], all use the volume of the loaded timber. Except for the entirely experimental forwarder cycles, the specific volume of the loaded assortments is unclear, and there are several approaches to deal with this issue in general, such as

• Woodpile-like measuring of the load with load area width, assortment length, and the height of the load [12];
• Average log estimation based on the sampling of the load and counting number of the logs in the load [13,14,15];
• Mean load calculated from the number of the forwarding cycles and total volume of the extracted timber, or mean log volumes; their count; and their assortment categories [14,16].

Each method has its downsides. The woodpile method relies on the conversion coefficient to convert the space volume to the solid wood volume, which often is an issue [17], not to mention the possible errors in gauge measuring the height. Log sampling influences the work elements; the woodpile method requires a researcher to be present at the roadside landings during the shift, which is highly time consuming [18]. The mean values of load for larger areas cannot describe specific cycles with enough accuracy, nor can they describe load-specific assortment composition [19].

Another method is the weight measurement of each load. When the measurements are to be taken on the site and not at the mill, there are two options. One is measuring forwarder axle loads or the load on the crane [20]. This method, however, depends heavily on proper wood density estimation for the loads, and multi-species loads are an issue.

Modern technologies, such as laser scanning and photogrammetry, have already found their way into forestry. Laser scanning has proved to be somewhat limited to the mill location [21,22], and the cost of the deployment of such a technology could be severely much than the use of the photogrammetry [23]. Recent studies have used ground photogrammetry methods for 3D reconstructions of the whole stand [24], tree [25,26,27], or stem parts [28], and for deriving, for example, individual tree height, diameter, and horizontal crown projection. In the timber production operations, an estimation of truckload using the unmanned aerial vehicle (UAV) has been presented [29]. Several mobile applications for roadside woodpile measuring exist.

In the preparation stage of our time study research, we explored the possibilities of other measuring methods. The primary goals were a minimal interruption to the forwarder operations and results comparable to the most widely used woodpile-like measuring, using level staff for height estimation. Given the fact that timber should be relatively faced against the protective mesh on the forwarder, we opted for cost-effective 2D photogrammetry. Without any Structure from Motion (SfM), or stereoscopic images (as the view is partially obstructed by the forwarder), the data processing was less time consuming and camera requirements were lower.

As such, we present a simple workflow using just 2D photogrammetry information for use in forestry operations and verify its viability.

2. Materials and Methods

For this study, a John Deere 1110D forwarder was used during sanitary felling on the grounds of School Forestry Enterprise administrated by the Czech University of Life Sciences Prague. Forwarded timber was mostly Norwegian spruce (Picea abies L.), with the occurrence of the scots pine (Pinus sylvestris L.).

2.1. Manual Measurements

The protective cover mesh of the load area was measured, as well as the load space profile. Because timber should be placed against the mesh, this served as a reference grid for further corrections and reference of the taken photos. Manual measurements were done at the roadside when forwarder stopped near the woodpile to unload the timber. The height of the load was measured on each side of the load near the mesh using Telefix measuring gauge with horns on both sides to get measured heights without any significant errors due to bad line of sight. This method attempted to suppress possible reader errors, as we observed such errors when using only simple levelling staff.

2.2. Cameras and Images Processing

In the forwarder, two LAMAX X10.1 cameras were installed, containing Sony IMX117 sensor with 12 Mega-pixel resolution (4000 × 3000 pixels) and sensor dimensions of 6.2 mm × 4.65 mm (1.55 µm per one pixel). Cameras were set up with time-lapse photo setting set for 3 s. We found out during the design of the experiment that use of time-lapse photos is better than a time-lapse video, as the images are sharper and therefore offer more detail. Image resolution was 0.87 mm per pixel (given the average distance from cameras to the load area being 2 m). Cameras were for this initial stage placed in the cabin, as seen in Figure 1, and powered directly by the machine power outlet. This proved to be an issue, as any interruption in the problematic 12 V car connector, which is a common problem, caused cameras to shut down. For prolonged recording with better resilience to connector disconnection, the power bank should be incorporated to serve as a buffer power supply. Because we did not use any stereographic methods, the alignment of the cameras was not crucial, and simple suction cup mounts could be used.

Camera images needed to be corrected for the lens distortion. For this purpose, several images of the calibration pattern—22 × 15 checkboard with 16.19 mm squares—were taken. Calibration and undistortion were made in GML Camera Calibration (PTC Inc.). There are many calibration techniques described; this particular method was chosen because of its ease of use for our purpose. For both cameras, re-projection errors were between 2.07 and 1.37 px. Given that the distance from cameras to the load was about 2 m, this resulted in a re-projection error below 2 mm in the plane of the measured load.

Because the protective mesh had been measured, we therefore could plot the control points in the photos against expected cartesian coordinates to scale the images up and verify successful undistortion of the image in the area of interest represented by load area so it could be used to take the measurements. This procedure was done in ESRI ArcMap (ESRI, Redlands, CA, USA) using georeferencing of the image. 4 × 4 point grid of the protective mesh was used to check for the root mean square error (RMSE) of the rectification process.

Rectification of the photos against the cartesian coordinates is shown in Figure 2. The 2nd polynomial transformation showed the best results with the lowest RMSE. The RMSE for each observation in absolute RMSE form is presented in

Table 1. Summary of the image rectification root square mean error (RMSE), measurements, and calculated distances.

Observation	RMSE of the Image Rectification	Reference-Gauge Measured Distance	Vertical Distance	Straight-Line Distance
#	cm	cm	cm	cm
1	0.33	67.2	70.4	75.0
2	0.47	155.4	154.0	154.9
3	0.41	114.7	109.2	110.0
4	0.50	147.1	142.2	145.2
5	0.39	153.9	152.3	153.2
6	0.41	170.7	163.1	163.8
7	0.53	170.9	170.1	170.3
8	0.42	151.6	151.8	152.1
9	0.25	102.4	88.9	89.0
10	0.31	160.1	166.1	166.5
11	0.23	136.2	137.9	139.6
12	0.16	125.8	126.3	127.5
13	0.29	147.1	144.2	147.3
14	0.34	177.2	173.1	177.0
15	0.41	174.9	179.0	179.5
16	1.22	160.9	163.5	163.5
p-value against the reference			0.2406	0.9354

. Equation (1) presents the calculations of RMSE described by [30], where n is the number of observations, ŷ is mean error, and y is the error of the current control point.

$$RMSE = \frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(\widehat{\mathrm{y}}-y\right)}^2$$

(1)

Mean error was calculated as mean of y values, where the y was calculated as a difference of reference control point (grid distance in the cartesian 2D system of loading place protection mesh) value and estimated (photo derived placement of the control points in loading place protection mesh) value. Except for one instance, the RMSE was below 1 cm. Such values of RMSE meant we could use the images for height estimation.

2.3. Data Processing

Once images were rectified, point shapefile was created, and points were placed on the respective side of the load indicating the top and the bottom of the load. Their X and Y coordinates were calculated in the cartesian coordinate system and exported to be later processed with R language [31], where straight-line distance and vertical distance between top and bottom points were calculated.

In total, 16 measurements of the loads were taken (eight on each side). Given the nature of the sanitary fellings, with common re-locations, branch collection by the forwarder, and waiting time for the harvester, this was the maximal achievable load count given the timeframe. Certainly, this downside will be addressed in future research, with the second version of the cameras rig testing. Datasets of the reference-gauge measured distance, vertical distance, and straight-line distance were then tested using a paired t-test with 0.95 confidence level. We tested for a difference between vertical and straight-line distance datasets with the null hypothesis of mean difference being equal to 0. The second batch consisted of testing vertical and straight-line distance datasets separately against reference-gauge measured distance with the null hypothesis of mean difference being equal to 0.

3. Results

Table 1 shows measured and calculated distances. Two types of distances were calculated: a vertical distance, which takes into account only difference along the Y-axis, and a straight-line distance, which represents the gauge measurement more accurately in case the gauge is not perfectly vertical. In several cases, the difference between vertical and straight-line distances was up to 5 cm.

First, the calculated vertical distance and calculated straight-line distance datasets were tested against each other, showing there is a significant difference between them: p-value of 0.0015. This significant difference demonstrates a need for precise vertical measurements for the load volume estimations where vertical distance is needed.

Another paired t-test, where calculated distances were tested against the gauge measured distances, showed a difference in the case of the vertical distance. It was not a significant one (p-value of 0.2406), but given the 95 percent confidence interval of the difference between −4.1727 and 1.0477, it is reasonable to suggest that vertical distance differs from the gauge measurements and is lower. This assumption was tested with the alternative hypothesis that the difference is lower than 0. With p-value 0.1203, this seems to be the case, yet not strongly significant, and it could be just due to a chance.

Comparison of the calculated straight-line distance and the gauge measured distance showed the difference is close to statistically insignificant with a p-value of 0.9354. Such a result can be interpreted as a confirmation that the camera-based measurements are up to gauge measurements of the load.

Additionally, given the substantial differences between the straight-line and vertical distances and strong evidence that straight-line distances are close to the gauge measured distances, we hypothesize that the gauge measurements can be biased and are overestimating the height of the load when the gauge measurements are not perfectly vertical. This possible error could be somewhat mitigated by using a spirit level, but we experienced issues where horns on the Telefix gauge were not long enough to grab on the top of the load due to the tapered sides of the load area. Longer horns would not be practical. In our case, difference up to 5 cm in height can be interpreted, given that assortment length was 5 m, as a possible calculated volume difference of 0.55 m³. Given the conventional conversion factor for the solid wood volume of 0.64, this is equal to the difference of 0.352 m³ u.b.

4. Discussion

Methods used in this study were not cutting-edge. Camera calibration software used together with square checkboard targets was superseded by several other techniques. This one was chosen for the ease of use for researchers who do not specialize in computer vision. Achieved re-projection errors (1.37–2.07 px) were more extensive than in other studies, which were focused on the precision of calibration, for example, 0.2705 px by Poulin-Girard et al. [32]. However, the study of Mokroš et al. [28], where two types of lenses on digital cameras were compared (a standard lens and fisheye lens), proposed the usage of fisheye lens with significantly lower error for photogrammetry purposes in forestry measurements.

Nevertheless, given the distance of about two meters from the load face, this error resulted in a possible error of 1.19–1.80 mm, which we considered sufficient for the purpose of load measurement. The one issue we were aware of was the thickness of the mesh, which could have influenced the calculated distances, as the actual load was several millimeters behind, but the improper facing of the logs created more severe distortions. For more precise estimations, the 3D stereoscopic system should be used [33,34]. It certainly is a further research goal.

To our best knowledge, current trends in using photogrammetry methods are aimed at using UAVs [29]. The nowadays main aim of ground photogrammetry is stem reconstruction for forest inventory purposes [24,28,35]. Studies aimed at the use of ground-based solutions for estimations of volume on trucks [36] or piles [37] are sparse.

Use of this technique for load measurements has its limitations. Measurements are done only in one place and cannot take into account gaps due to the different assortment lengths nor can measure the second pile of logs.

Advantages are, however, significant. By using a high vantage point and cameras, the possibility of measurement error induced by the researcher is lowered. Properly set up cameras can record the whole shift, without interfering with the work of the forwarder. The data processing time is higher than with traditional gauge measured height of the load, but there is no need for the researcher to be present at the roadside while the forwarder is ready to unload. In our study, the forwarder was using several roadside landings, and taking measurements at all of them proved to be an organizational challenge. Using this method, data can be collected without interference to the work of the forwarder, contrary to the load sampling method. This new approach also offers details on the load assembly, as the time-lapse photos can capture the whole work cycle.

Nevertheless, this study shows a potential for at least the same accuracy as the gauge measurements, which are still being used and have their downsides [17] with lower chances for researcher errors. Consumer-grade cameras are already available to many researchers and are not expensive. Timed photos can also provide basic timeframes for the work cycle during the whole shift. Together with GPS and CAN-Bus data, this method of load volume estimation can lead to a more automated approach to the forwarders time studies.

5. Conclusions

The photogrammetry can be used as a more effective method for measuring forwarder loads, mainly due to its ability to observe the top of the load. Use of photos for estimation of height of forwarder load showed inaccuracy issues in measurement by gauge measuring—mainly inaccuracy in estimation of the exact height, which is not always clearly visible from the ground. The photogrammetry method also saves time. Even though data processing is more time-consuming, researchers in the field do not need to be present at the roadside landings, or they can record other aspects of the work cycles.

We demonstrated that accuracy levels well within the limitations of the currently used method of gauge measuring could be relatively easily achieved. Furthermore, the proposed photogrammetric method enhanced with time-stamped photos can be effectively used in time studies.