3.2. Accuracy Verification Test
As the navigation line is estimated from the crop row lines, verifying the accuracy of the navigation line can be converted to verifying the accuracy of the crop rows. To obtain a more convincing verification of the accuracy of this research method, the performance of the linear fitting method in this study and of the LSM method [
16] is analysed from two perspectives: qualitative and quantitative. We extracted 20 images from the videos of samples A, B, C, and D, respectively. The two types of crop row extraction methods analysed are different from the linear fitting method, and the other parts use the same method described herein. The proposed research method uses the point-line distance optimisation based on the Huber loss function described above to fit the feature point sets, whereas the method for comparison directly uses the traditional LSM straight-line fitting method to fit the feature point sets. For the convenience of expression, the proposed research method based on point-line distance optimisation based on the Huber loss function is referred to below as HUBERP, and the method based on LSM for comparison is referred to as LSMC.
As shown in
Figure 11a–h, in the images of the different growth periods, the inter-plant relationships for crops in the same crop row and the inter-row relationships for different rows are different, and the sizes of the plants are also inconsistent. Further details are provided below.
- (1)
The leaf area of sample A is small. The single plant form can be clearly distinguished among plants with no shadows. There are almost no weeds, but the number of leaf surfaces is small.
- (2)
The leaf area of sample B is slightly larger with a small amount of shadow, and the leaf surfaces of the crops are connected between plants to a certain extent. There are small weeds.
- (3)
The leaf area of sample C is large with many shadows, and the leaf surfaces of the crops are connected between plants and rows to a certain extent. There is a large area of weeds.
- (4)
The leaf area of sample D is slightly larger and has many shadows, and the leaf surfaces of crops are connected between plants to a certain extent. There are small weeds.
- (5)
From a qualitative perspective, i.e., by comparing the crop row extraction results for the three crop growth periods and two light conditions, it can be seen that HUBERP has strong adaptability. The fitted straight lines detected in this study and shown in
Figure 11 are all distributed in the position of the crop row. The straight lines obtained by HUBERP (solid line) have little deviation from the manually marked crop row lines (white dashed line), and the results from HUBERP are basically consistent with expectations. In contrast, only part of the straight lines (solid lines) obtained by the LSMC fit well with the manually marked crop row lines (white dotted lines). Therefore, HUBERP can meet the requirements for crop row line identification in selected environments during different growth periods and different light conditions, whereas the LSMC cannot meet the requirements for the early identification of crop growth.
There are two reasons for the large deviation of the partial line fitting of the LSMC. First, as shown by the black dotted line in
Figure 11a, when extracting feature points, there is a large plant spacing between the plants in the same crop row of sample A. When traversing to a certain horizontal strip, the two crop row areas in the middle of the image have no crop features, causing the algorithm to incorporate by error the feature points (deviation points) of the adjacent crop rows into the fitting point set. The feature points of the adjacent crop rows originally do not belong to the fitting point set. Second, when the plant size is large as shown in
Figure 11e, there is a connection phenomenon (blue circled in
Figure 11e) between adjacent crop rows. This leads to the feature point extraction and the program identifying two crops in different crop rows as one. These two aspects above will introduce outliers into the set of fitted points.
When using the LSMC, outliers usually affect the fitting, because of the square of the offset. HUBERP removes the influence of outliers using an M-estimated robust regression. As shown in
Figure 11a–f, this research method can effectively suppress the influence of weeds and soil clods (marked by circles) on the recognition of crop row lines in various periods of broad-leaved plants to a certain extent. Compared with the LSMC, HUBERP can be applied to field environments with different crop growth periods; moreover, as shown in
Figure 11c,d,g,h, HUBERP can be applied to field environments with different light conditions. To sum up, HUBERP is more robust than LSMC and is adaptable to different periods and lighting conditions.
The central approach of the quantitative evaluation standard is to compare lines drawn by an expert human with fitted straight lines. We call the lines drawn by the expert human reference lines. To strictly evaluate the similarity between the two, it is necessary to evaluate both the distance and the angle. As shown in
Figure 12, the assumption is that the line
LF1 is the left-fitting line and line
LR1 is the corresponding reference line. Line
LR1 intersects with the upper and lower boundaries of the image at
T1 and
B1, respectively. Line
LF1 intersects the upper and lower boundaries of the image at
T2 and
B2, respectively. The deviation angle
θ between the fitted straight line and reference line reflects the accuracy of the angular deviation.
θL represents the deviation angle between
LF1 and
LR1.
d1 represents the distance between
T1 and
LF1, and
d2 represents the distance from
B1 to
LR1.
kF1 and
bF1 are the slope and intercept of
LF1, respectively, and
kR1 and
b R1 are the slope and intercept of
LR1, respectively. The angle
θL is calculated as follows:
θL distinguishes between positive and negative values and can represent the relative positions of
LF1 and
LR1, thereby indicating the angular deviation value;
θabsL is the absolute value of
θL, and can indicate the degree of the angular deviation. The distance
d1 from
T1 to
LF1 and distance
d2 from
B1 to
LR1 are calculated according to a distance equation from a dot to a straight line.
LR1,
LF2,
T3,
T4,
B3,
B4,
θR,
θabsR,
d3, and
d4, as shown in
Figure 12, are the relevant parameters of the right lines, and the meanings and calculation methods for the parameters above are similar to those on the left. The comprehensive evaluation indicator of the distance deviation of each image is represented by
lineComp, and is calculated from the average value of
d1,
d2,
d3, and
d4, i.e.,
lineComp = (
d1 +
d2 +
d3 +
d4)/4.
lineComp represents the central tendency of the distance deviation for each image. The maximum value of the average value of
d1, average value of
d2, average value of
d3, and average value of
d4 of all images in a single sample is represented by
dmax, i.e.,
dmax =
max [avg(
d1), avg(
d2), avg(
d3)
, avg(
d4)].
dmax reflects the maximum average distance deviation of each sample. The angular deviation comprehensive evaluation index
angComp is calculated from the mean values of
θL and
θR, i.e.,
angComp = (
θL +
θR)/2. Corresponding
angComp_abs is calculated from the average value of
θabsL and
θabsR, i.e.,
angComp = (
θabsL +
θabsR)/2.
angComp and
angComp_abs represent the angle deviation value of the overall images and the degree of deviation in the overall images, respectively.
The distance deviation analysis proceeded as follows. Two methods (HUBERP and LSMC) were used to obtain two distance deviation indicators (
lineComp and
dmax) of each sample for four sample statistics.
Figure 13a shows the average value for each image in each sample.
Figure 13b shows the statistical data of each sample’s
dmax. The two distance deviation indicators from the two methods for sample A are quite different, but those for the other samples are not very different. In sample A, the
lineComp and
dmax values of HUBERP are much smaller than those of the LSMC, indicating that the distance deviation of HUBERP in the early period of crop growth is smaller than that of the LSMC. In the samples from other periods, the difference between the two distance deviation indicators as measured by the two methods is not as evident.
The angle deviation analysis was based on two sub-indicators (
rateREC and
avgANG). The two angle indicators (
angComp,
angComp_abs) of each image in each sample were judged separately. The judging rules were as follows: if the value of the angle indicator (
angComp or
angComp_abs) was within the available range, the value was an available value; if not, it was an unavailable value. Assuming that the number of available values of an angle index set was denoted as
avNUM and the total number of elements in the set was denoted as
cNUM, then the efficiency of the angle index was the ratio of
avNUM to
cNUM, denoted as
rateREC. The available range was [−5, 5]. Without excluding the unavailable value, the average value of the angle indicator was calculated and denoted as
avgANG. The
rateREC and
avgANG values were respectively calculated for the two angle indicators. The statistical results for
rateREC and
avgANG are shown in
Table 6.
From the perspective of rateREC, it can be seen that the rateREC values of the two angle indicators of the four samples as measured by HUBERP are greater than or equal to those from the LSMC. HUBERP is generally better than LSMC from the perspective of rateREC, especially in the early crop growth period corresponding to sample A.
From the perspective of
avgANG, it can be observed from the concrete values of
avgANG that all of
avgANG values of HUBERP belong to the available range [−5, 5] referred to above. However
, the
angComp_abs of sample A of the LSMC (11.71°) does not belong to the available range. To compare the two methods more intuitively and quantitatively, we calculated the difference between the absolute values of the same angle indicator of HUBERP or LSMC in the same sample (denoted as
Df). For example, in
Table 6, |0.20| − |−2.92| = 0.20 − 2.92 = −2.72. The closer the value of
Df to 0, the smaller the absolute value of the angular deviation indicator between HUBERP and the LSMC. If
Df is less than 0, the indicator’s absolute value in HUBERP is smaller relative to that in the LSMC. The
Df of
angComp and
Df of
angComp_abs are denoted as
Df1 and
Df2, respectively.
For sample A, the values of Df1 and Df2 deviate from 0 and are less than 0. The HUBERP angle deviation is large relative to that of the LSMC, and the HUBERP angle indicator is smaller. Notably, the angComp_abs values of sample A in the LSMC are large, indicating that when using the LSMC, the angular deviation of sample A is large overall. Thus, HUBERP adapts to the environment corresponding to sample A, whereas the LSMC does not.
For sample B and sample D, both Df1 and Df2 are close to 0. Therefore, the difference between the angular deviations of the two methods is not large, and both methods are adapted to the environments of sample B and sample D.
For the C sample, although the value of Df1 is far from 0 and greater than 0, the value of Df1 (shown as 1.44°) falls within the valid range [−5, 5], and owing to the large leaves of the single broad-leaved crops, there is an error between the reference line and actual crop row centreline; thus, 1.44° is acceptable. Df2 approaches 0 and is greater than 0. Therefore, both methods are suitable for the sample C conditions.
The analysis of the two evaluation indicators (avgANG and Df) shows that HUBERP has a small angular deviation in early crop growth. In other periods, the difference in the angle deviation as measured by the two methods is not evident.
In summary, compared with the LSMC, HUBERP can adapt to longer crop growth periods. Especially in the early stages of crop growth, HUBERP has significant advantages in distance deviation and angle deviation, and can effectively improve the multi-period adaptability of the algorithm.
3.3. Timeliness Verification Test
Under the same conditions, four samples were processed five times using the above two methods (LSMC and HUBERP), and the CPU time taken by the program during each test process was recorded.
In each test of each sample, the test times for the first and last images were removed, and the average value of the remaining data was calculated. The obtained value was the average test time for each sample of each method in each test, denoted as avg_t1.
The avg_t1 data obtained from five tests were averaged, and the obtained value represented the average testing time for each sample of each method (denoted as avgt). The avgt data were used to evaluate the timeliness of each sample of each method.
The
avgt data of the four samples were averaged, and the obtained value represented the average testing time of each method (denoted as
AVGT). This was used to evaluate the timeliness of each method. The timeliness verification test results are shown in
Figure 14.
By analysing the principles of the two methods, it was found that HUBERP needs to determine the weight of the data samples according to the size of the regression residuals, whereas the LSMC does not need to determine the weight of data samples; additionally, HUBERP needs to perform two straight-line fittings. Therefore, the time complexity of HUBERP is higher than that of the LSMC, inevitably leading to lower efficiency. It can be seen from
Figure 14 that
AVGT (HUBERP) = 38.53 ms >
AVGT (LSMC) = 37.57 ms. When the size of the image meets the needs of the visual navigation industry, although the efficiency of HUBERP is inevitably reduced, the average time consumed by the two methods is not significantly different and can meet the timeliness requirements for visual navigation.
By observing the avgt values of samples A, B, and C in the two methods, it can be seen that the corresponding avgt values gradually increase with an increase in the crop growth time. This is because, with the growth of crops, the area proportion of the crops in the image increases, resulting in the increase of data processing required for preprocessing, feature point extraction, and optimisation. However, in the same sample, the maximum difference of avgt values corresponding to the two methods is less than 3 ms, indicating that HUBERP has similar timeliness to LSMC in different crop growth periods. By observing the avgt values of samples B and D in the two methods, it can be seen that in the same sample, the maximum difference in the avgt values corresponding to the two methods is less than 2 ms, indicating that HUBERP has the same timeliness as the LSMC under different light conditions.
To further validate the timeliness of the method proposed in this paper, the method proposed by Diao Zhihua et al. (denoted as Dmethod) was compared with the method proposed in this paper (HUBERP) in terms of time consumption [
30]. The evaluation index was
AVGT, and the statistical results are shown in
Figure 15.
The above data in
Figure 15 show that the time consumption of HUBERP is lower than that of Dmethod.
According to the inductive research of Yang et al., most algorithms based on using machine vision to extract navigation lines take more than 100 ms [
31]. Without considering the factors of differentiation in the image resolution and hardware, an average test time of less than 100 ms is used as the standard to evaluate whether the algorithm has high timeliness. As shown in
Figure 14, the average test time
avgt of each sample image in the proposed method (HUBERP) does not exceed 41 ms. Therefore, when the hardware used in the test algorithm is assembled into the agricultural vehicles, the proposed method can meet the requirements of high timeliness in practical applications while using the current image resolution.
In summary, under the premise of meeting the efficiency requirements of visual navigation, HUBERP has timeliness similar to that of the LSMC and can adapt to a longer crop growth period than the LSMC.