Segmentation of Rice Seedlings Using the YCrCb Color Space and an Improved Otsu Method

Abstract

Rice seedling segmentation is a fundamental process of extracting the guidance line for automated rice transplanters with a visual navigation system, which can provide crop row information to ensure the transplanter plants seedlings along the crop row without damaging seedlings. However, obtaining accurate rice seedling segmentation in paddy fields is still a challenging task. In this paper, a rice seedling segmentation method in paddy fields is proposed. The method mainly consists of two steps: image graying and threshold segmentation. In the procedure of image graying, the RGB (Red Green Blue) seedling image is first converted into the YCrCb color space and a Cg component is constructed. A color-index 2Cg-Cb-Cr is then constructed for image graying based on the excess green index (2G-R-B), which can reduce the influence of illumination variation on the equality of image graying. For the second step, an improved Otsu method is proposed to segment rice seedlings. With respect to the improved Otsu method in this research, the background variance of within class variance is weighted by a probability parameter to ensure that the method works well for both bimodal and near-unimodal histogram images, and the search range of gray levels is constrained to reduce the time to search the segmentation threshold. Experimental results indicate that the proposed method achieves better segmentation results and reduces the computational cost compared with the traditional Otsu method and other improved Otsu methods.

1. Introduction

Rice is one of the major cereal food crops in the world. The transplanting of rice seedlings is a preliminary stage for rice growing and production operations. Traditional transplanting methods include conventional approaches such as manual transplanting (which has been successfully used for many centuries) and mechanical transplanting, but neither is practical for large-scale commercial farms because they are extremely labor-intensive, costly and time-consuming. Hence, it is necessary to use an automated rice transplanter to reduce manual tasks and the cost of crop production, and to contribute to the productivity and competitiveness of farmers in providing agricultural supplies [1]. The key for an automatic rice transplanter operating in-a complex paddy field environment is to have autonomous navigation.

In recent years, along with advances in robot technology, navigation systems have been applied to intelligent agricultural machinery such as grafting robots [2], transplanting robots [3] and harvesting robots [4]. Among a number of guidance-sensing technologies investigated in the last decades, two main approaches have achieved the greatest commercial success: global positioning system (GPS) and visual navigation system. GPS is a space-based satellite navigation system with an unobstructed line of sight to four or more GPS satellites, and it has been used in agricultural robots or automated agricultural machinery to provide location and time information in all weather conditions [5]. For example, Nagasaka et al. [6] designed an automated rice transplanter with a GPS. The study indicated that the system can provide accurate absolute positioning for navigation. However, the cost of GPS is too high to be widely used in agricultural machinery and it has many practical problems, such as drops in the GPS signal because of natural obstacles, errors in satellite communication, and frequent manual intervention [7].

With the improvements in computer calculation speed and the development of visual algorithms, the visual navigation system for agricultural robots has become one of the development trends and has strong real-time and high accuracy characteristics. In the visual navigation system, how to extract the guidance line, especially in a complicated environment, is a difficult task. For the visual navigation of farm machinery, the most widely used method to extract the navigation line consists of two steps [8]: image segmentation of crop rows, and guidance line extraction in an image of the crop row. Image segmentation of crop rows is a fundamental step of extracting information on crop rows in which the essential problem to be tackled is to segment plants, by classifying image pixels into plant or background elements such as soil, stones and water. Guidance line extraction creates a reference, allowing the machinery to determine its position relative to the guidance line and to follow the crop row without damaging crops. Thus, good performance in segmenting crops is crucial for guidance line extraction.

Over the last decade, numerous crop segmentation approaches have been reported in the literature [9]. Among them, color-index-based methods are the most commonly used methods for crop segmentation, in which the RGB (Red Green Blue) space is first converted into an alternative color space to obtain a gray image with high contrast, thus highlighting the plant, i.e., with pixels brighter for plant pixels and darker for non-plant pixels, and then segmenting plants using a threshold [10]. These techniques, including ExG (excess green index) [11], VEG (vegetative index) [12], ExGR (excess green minus excess red index) [13], CIVE (color index of vegetation extraction) [14] and MExG (modified excess green index) [15], generally rely on the principle that the green channel in a plant image contains more useful information than the other two channels, and needs a threshold to classify an image pixel into one class (plant or background). Often, the color-index-based methods cannot even accurately separate plants from the background without an adaptive threshold. Furthermore, they have failed to segment plant pixels from the background in low- or bright-light conditions [16].

To obtain a good segmentation result, more learning-based approaches have been proposed such as PSO-MM (partial swarm optimization clustering and morphology modeling) [17], decision trees [18], and support vector machines [19]. These methods are carried out through supervised and unsupervised learning processes, where the training and classification phases are generally offline and online processes, respectively. Although they can obtain good crop segmentation results in a complex environment, they require sufficient training samples and additional computational costs for training and may therefore not be suitable for real applications. Compared with learning-based methods, color-index-based approaches have some advantages. Firstly, the color is an intuitionistic and easy characteristic to distinguish plants from the background. Secondly, the color-index-based method is simple and takes a relatively small amount of calculation time and achieves satisfactory segmentation results with a dynamic threshold. Additionally, to the best of our knowledge, there are very few studies aiming at handling segmentation of rice seedlings in paddy fields.

In order to accurately segment rice seedlings in a complex field environment, a color-index-based segmentation method was proposed in this paper. Firstly, the method selects the YCrCb color space in the image graying process, due to illumination invariance of Cb and Cr components. Given that the green component value of rice seedling is larger than that of the background, the Cg component is built, and 2Cg-Cb-Cr is constructed as the color-index used in this study. Subsequently, the threshold segmentation process is performed based on an improved Otsu method which weights the background variance of the traditional Otsu method. The weight parameter is adaptively adjusted with the cumulative probability of rice seedlings’ occurrence to obtain a satisfactory segmentation in this research. Additionally, the threshold is searched within a limited range to reduce the computation time. Experiments on several test rice seedling images in a paddy field were conducted to demonstrate the effectiveness of the proposed method.

The rest of the paper is organized as follows. Section 2 describes the details of materials and methods. Experimental results of our proposed method are presented in Section 3. The conclusions are shown in Section 4.

2. Materials and Methods

2.1. Experimental Fields and Rice Cultivars

The test images in our paper were collected from Agricultural Research and Demonstration Base of Anhui Agricultural University, which is located in Anhui province, China (30.57 N, 117.01 E). In the field, rice (cultivar: Fengliangyou No.4, Huiliangyou 898, Zhendao No.18 and Shangnongjing No.2) was planted. Fengliangyou No.4 and Huiliangyou 898 are two types of hybrid indica rice and Zhendao No.18 and Shangnongjing No.2 are two types of conventional japonica rice, which are widely planted in southern China. The transplanting time and model are identical with those of local custom farm practices. The inter-row space was about 20–30 cm. The plant density was approximately 27 × 10⁴–30 × 10⁴ hole/ha and 4–5 seedlings per hole. The total area of each rice cultivar was about 0.15 ha.

2.2. Image Acquistion

As shown in Figure 1A, we designed an automatic acquisition device that mainly consisted of two parts: the image acquisition device and a PC computer. Rice seedling images were obtained using a HIKVISION MV-CA050-20UM/C camera (Hangzhou Hikvision Digital Technology, Hangzhou, China) (with a USB port. The focal length of the camera lens was 25 mm. The camera mounted on a camera support at 1.8 m above the ground took pictures of seedlings (See Figure 1B). The obtained images were transmitted to the computer through a USB cable, and were saved as 24-bit color images in RGB color space with the JPG format. The resolution of the original images is 2560 × 1920. After the JPG images were obtained, it is necessary to shrink the size of images to reduce the computational load. In our experiment, the image size shrank from the original size of 2560 × 1920 to 320 × 237 pixels.

2.3. Color Space Analysis

It is well known that the green chromaticity of rice seedlings is higher than that of red and blue in the RGB color space. Moreover, the background of paddy field cannot be assumed to be soils due to water, mud, residual straws, shadow and so on, and rice seedling images are always taken under outdoor illumination. Therefore, it can be expected that an appropriate color space needs to be selected for the color-index-based segmentation method, which can reduce the influence of outdoor illumination on the segmentation quality of green rice seedlings. At present, most images taken by digital cameras are in RGB format. The RGB color space is adequate for image displaying, but not suitable for color-based segmentation because of the high correlation between the components R, G and B [20]. To achieve good segmentation results, the RGB color space is generally converted into other color spaces such as HSV (hue, saturation, value), HIS (hue, saturation, intensity), YCrCb, and L*a*b*and L*u*v*. Among these color spaces, the YCrCb color space is employed to standardize the images for television and is derived from the RGB color space using Equation (1) [21]:

$$\left[\begin{array}{l}\mathrm{Y}\\ \mathrm{Cb}\\ \mathrm{Cr}\end{array}\right]=\left[\begin{array}{l}16\\ 128\\ 128\end{array}\right]+\left[\begin{array}{ccc}0.279& 0.504& 0.098\\ -0.148& -0.291& 0.439\\ 0.439& -0.368& -0.071\end{array}\right]\left[\begin{array}{l}\mathrm{R}\\ \mathrm{G}\\ \mathrm{B}\end{array}\right],$$

(1)

$$\mathrm{Cg}=128+\left[\begin{array}{ccc}-0.318& 0.439& -0.121\end{array}\right]\left[\begin{array}{l}\mathrm{R}\\ \mathrm{G}\\ \mathrm{B}\end{array}\right],$$

(2)

where Y is a luminance component and Cr and Cb are two chrominance components. Cr and Cb essentially correspond to the color differences R-Y and B-Y, respectively. It can be seen that the luminance is decoupled from the chrominance in the YCrCb color space, and mapping an RGB color to the YCrCb space is a straightforward linear transformation that can simplify the calculation and speed up the conversion. Unlike the RGB color space, the green chrominance component is not described in the YCrCb color space, while rice seedling has a larger green chroma value than that of the background. Therefore, the selected color space should be able to describe the green chrominance component. In this regard, the Cg component is introduced to denote the color difference G-Y. It is defined as Equation (2), derived from Equation (1).

Figure 2 shows the conversion results of typical rice seedling images in different color spaces. The first column shows the original images taken under different natural illumination (sunny or cloudy conditions) and different paddy field backgrounds (water, soil) in RGB format. The corresponding G component images are shown in the second column. The contrast between rice seedling and the background in the G images is not obvious, and is affected by the variation of illumination. As shown in Figure 2(C1), rice seedlings and the background cannot even be distinguished by human vision. Unlike the G component, green chrominance information without luminance is described in the Cg component. As seen in Figure 2(A2,B2,C2), rice seedlings are clear and can be differentiated from the background with interference factors such as highlight, shadow and mud. Therefore, the Cg component is discarded mainly due to the color itself rather than the luminance component, which can enhance rice seedling information in paddy fields with illumination variations.

2.4. Image Graying Based on Color Index

Considering that the green characteristic can be used to separate rice seedlings from the background, the green chrominance of the rice seedling region should be accentuated in the image graying process based on color-index, while that of the paddy field background region will be attenuated to improve segmentation results. Figure 3 shows 50 rice seedling images taken under different paddy field environments. Afterwards, the test images in the RGB color space are converted into Cr, Cg and Cb components and 150 rice seedling and background sampling points are randomly selected from the Cr, Cg and Cb images, respectively. Values of the Cr, Cg and Cb components for the rice seedling and background sampling points are shown in Figure 4. As seen in Figure 4, pixel values of rice seedlings in the Cg image are larger than that of the background, and pixel values of rice seedlings in the Cg image are greater than pixel values of rice seedlings in the Cr and Cb images.

In order to describe the color difference between the rice seedlings and the background effectively, the differences between the Cg and Cr, and between the Cg and Cb of the sampling points are calculated, respectively. Figure 5 describes the relationship between Cg-Cr and Cg-Cb for the rice seedling and background sampling points. In Figure 5, the horizontal axis shows the value of Cg-Cr and the vertical axis shows the value of Cg-Cb From Figure 5, it can be noted that the rice seedling points and the background points converge in different bounded regions of the (Cg-Cr, Cg-Cb) plane. Therefore, one or more boundary lines can separate rice seedlings and the background. According to the definition of a straight line, the boundary line can be defined as:

$$a(Cg-Cr)+b(Cg-Cb)>T,$$

(3)

where Cg, Cr and Cb are the Cg component value, Cr component value and Cb component value, respectively; T is a threshold; a and b are the weight of Cg-Cr and Cg-Cb. Motivated by the idea of ExG(2G-R-B) defined by Equation (4) in which R, G, B denote values of the R, G and B components, we set the values of a and b mentioned in Equation (3) to 1. Thus, Equation (3) is rewritten as Equation (5):

$$\mathrm{ExG}=\{\begin{array}{ll}0& \hspace{1em} 2G-R-B<0\\ 2G-R-B& 0<2G-R-B<255\\ 255& \hspace{1em} 2G-R-B>255\hspace{1em},\end{array}$$

(4)

$$2Cg-Cr-Cb>T.$$

(5)

Figure 6 shows the distribution of 2Cg-Cr-Cb of the rice seedling and background sampling points, where the contrast between the rice seedlings and the background is clear. As a result, we are motivated to use 2Cg-Cr-Cb as the color index to extract the gray image.

2.5. Improved Otsu Method for Rice Seedling Segmentation

In this subsection, we discuss how to determine the threshold T mentioned in Equation (5) and subsequently use it to separate rice seedlings from the background. Threshold is widely used for image segmentation due to its simplicity. Until now, a number of threshold segmentation algorithms have been published in the literature. For example, Tellaeche et al. [22] applied an adaptive threshold algorithm determined by entropy of a histogram to distinguish plants and soil. Hernández-Hernández et al. [23] introduced an automatic threshold method based on Gaussian distribution functions of the soil and plant pixel intensities. A survey considering a variety of automatic threshold techniques for plant segmentation was carried out by Hamuda et al. [9], and this study showed that most threshold methods provide expected results for particular application, but it also suggested there is not a general method available for crop segmentation. The Otsu method [24] is one of the most commonly applied methods to threshold segmentation due to its simplicity and adaptability. The details of the Otsu method are described in Section 2.5.1.

2.5.1. The Otsu Method

The Otsu technique chooses the threshold by minimizing the within-class variance or maximizing the between-class variance. Suppose that the pixels in a given image be represented in L gray levels ranging from 0 to L-1. Let $n_i$ denote the number of pixels with the level i and N denote the total number of pixels. The occurrence probability of the level i is calculated as follows:

$$p_i=\frac{n_i}{N},\hspace{1em}and\hspace{1em}{p}_i\ge 0,\hspace{1em}{\displaystyle \sum _{i=0}^{L-1}{p}_i=1}.$$

(6)

The image is divided into two classes, C₁ with gray levels [0, 1, …, t] and C₂ with gray levels [t+1, …, L-1] by the threshold t. The gray level probability distributions for the two classes denoted as $P_1(t)$ and $P_2(t)$ are given as:

$$P_1(t)={\displaystyle \sum _{i=0}^t{p}_i},P_2(t)={\displaystyle \sum _{i=t+1}^{L-1}{p}_i}=1-P_1(t)$$

(7)

Let ${\mu }_1$ and ${\mu }_2$ denote the mean levels of the C₁ and C₂ classes, and let ${\sigma }_1^2(t)$ and ${\sigma }_2^2(t)$ denote the variances of the two classes, respectively. They can be computed as:

$${\mu }_1(t)=\frac{{\displaystyle {\sum }_{i=0}^{t}i{p}_i}}{P_1(t)},{\mu }_2(t)=\frac{{\displaystyle {\sum }_{i=t+1}^{L}i{p}_i}}{P_2(t)}$$

(8)

$${\sigma }_1^2(t)={\displaystyle \sum _{i=0}^t{(i-{\mu }_1(t))}^2}\frac{p_i}{P_1(t)},\hspace{1em}{\sigma }_2^2(t)={\displaystyle \sum _{i=t+1}^L{(i-{\mu }_2(t))}^2}\frac{p_i}{P_2(t)}$$

(9)

Let $\mu$ represent the total mean of the pixel values of a given image:

$$\mu ={\displaystyle \sum _{i=0}^{L-1}i{p}_i}=P_1(t){\mu }_1(t)+P_2(t){\mu }_2(t)$$

(10)

The within-class variance ${\sigma }_w^2(t)$ of C₁ and C₂ is defined as follows:

$${\sigma }_w^2(t)=P_1(t){\sigma }_1^2(t)+P_2(t){\sigma }_2^2(t).$$

(11)

The optimal threshold T is obtained by minimizing the within-class variance ${\sigma }_w^2(t)$ as follows:

$$T=\underset{0<t<L-1}{\arg \min }{\sigma }_w^2(t).$$

(12)

The Otsu method is easy to implement because it involves a simple mathematical expression when calculating the optimal threshold. As mentioned in [25], a single threshold T decided by the Otsu method is equal to the average of mean values of the two classes. Thus, the Otsu method can obtain a satisfactory segmentation only when C₁ and C₂ classes have a similar variance. In other words, the Otsu method works well for images with a clear bimodal distribution. As shown in Figure 7A, Otsu’s threshold is located at the valley between two consecutive peaks, and the two classes’ variances are in contrast to each other. However, the threshold biases toward the class with the larger variance (as shown in Figure 7B) when the histogram with a single peak is unimodal or near-unimodal. Hence, it is necessary to improve the Otsu method for selecting the desired threshold in unimodal or near-unimodal histograms of images.

2.5.2. Improvement to the Otsu Method

Aiming at automatically selecting the desired threshold for both bimodal and unimodal images, the neighborhood valley-emphasis (NVE) method [26] is a modified form of the Otsu method which selects a threshold that has a small probability of occurrence by adding a weight to the between-class variance. The weight is obtained from neighborhood information around the valley point in the gray-level histogram. Figure 8A shows a rice seedling image. The rice seedling is small compared with the size of the background. The corresponding image histogram shown in Figure 8B is a near-unimodal distribution. Unlike Otsu’s threshold, which is located towards the class with the largest variance, the NVE’s threshold is located at the right rim of the histogram, which results in almost all rice seedlings being wrongly segmented as the background. This means that the NVE method performs well when the histogram of images has enough valley information. However, in rice seedling images, most of the rice seedling area, in which the seedlings have just been transplanted by a rice transplanter, is much smaller than the background area in paddy fields. The images do not have enough valley information in this case. Therefore, the NVE method, which uses valley neighborhood information as the weight, still cannot improve the segmentation equality of rice seedlings. In this subsection, we propose an improvement to the Otsu method by adding a weight parameter to the within-class variance. The details of the improved Otsu method are described as follows.

The within-class variance of the Otsu method defined as Equation (11) contains two classes. Let the first item $P_1(t){\sigma }_1^2(t)$ denote the background variance and the second item $P_2(t){\sigma }_2^2(t)$ be the rice seedling variance. According to [25], the background class in the rice seedling image mainly determines the desired threshold. To obtain a desired threshold, the first item should contribute less to the within-class variance. In this regard, the parameter $\alpha$, ranging from 0 to 1, is introduced to weight the first item. Now, the weighted background variance is $\alpha$ times the original background variance, which influences the within-class variance. Using $\alpha$, Equation (11) can be modified as:

$${\sigma }_w^2(t)=\alpha P_1(t){\sigma }_1^2(t)+P_2(t){\sigma }_2^2(t).$$

(13)

From the analysis of the Otsu algorithm in [25], the weight $\alpha$ should be selected to be close to 0 because the rice seedling variance is close to 0 in the case of an image with small seedlings, while $\alpha$ can be set as a larger value for the rice seedling image. It is worth noting that the random or fixed weight value for $\alpha$ is not suitable for different rice seedling images, and the weight can vary with the probability of rice seedling occurrence. Therefore, to adaptively set the weight, the cumulative probability of rice seedling occurrence is considered as weight $\alpha$ in the proposed method. Under $\alpha =P_2(t)$, Equation (13) is rewritten as follows:

$${\sigma }_w^2(t)=P_2(t)P_1(t){\sigma }_1^2(t)+P_2(t){\sigma }_2^2(t).$$

(14)

The optimal threshold T is chosen by the following function:

$$\begin{array}{l}T=\underset{0<t<L-1}{\arg \min }{\sigma }_w^2(t)\\ \hspace{1em}=P_2(t)P_1(t){\sigma }_1^2(t)+P_2(t){\sigma }_2^2(t)\end{array}$$

(15)

Figure 9 shows the curves of five kinds of variance with different thresholds. In Figure 9, the green and red curves describe the rice seedling variance and the background variance, respectively. The corresponding within-class variance is shown by the purple curve. We can find that the minimum value of the within-class variance is T = 22. However, the weighted background variance (blue curve) is much smaller than the original background variance (red curve). The minimum value of the weighted within-class variance shown by the dark green curve is T = 33. According to Figure 8B, the threshold determined by the weighted Otsu method is close to the desired threshold.

Based on the above analysis, the calculation of the within-class variance traverses all gray levels in the image. Thus, the computational time increases with an increase in image resolution. To improve the efficiency of the weighted Otsu method, the optimal threshold is searched within a limited range. Using $\mu$, ${\mu }_1$ and ${\mu }_2$ as mentioned above, D is defined as the class difference of rice seedling and background:

$$D=\left|\mu -{\mu }_1\right|-\left|\mu -{\mu }_2\right|.$$

(16)

From Figure 8A, the rice seedling area is much smaller than the background area, and the gray value of seedlings is greater than that of the background. From Figure 8B, we also observe that ${\mu }_1<\mu$ and $P_1(t)>P_2(t)$. Equation (16) can therefore be expressed as:

$$D=2\mu -{\mu }_1-{\mu }_2.$$

(17)

According to Equations (7), (8) and (10), the difference, D, is rewritten as:

$$\begin{array}{l}D=2\mu -{\mu }_1-{\mu }_2\\ \hspace{1em}=2{\displaystyle \sum _{i=0}^{L-1}i{p}_i}-{\displaystyle \sum _{i=0}^{t}i{p}_i}/P_1(t)-{\displaystyle \sum _{i=t+1}^{L-1}i{p}_i}/P_2(t)\\ \hspace{1em}=\frac{2P_1(t)P_2(t){\displaystyle \sum _{i=0}^{L-1}i{p}_i}-P_2(t){\displaystyle \sum _{i=0}^{t}i{p}_i}-P_1(t){\displaystyle \sum _{i=t+1}^{L-1}i{p}_i}}{P_1(t)P_2(t)}\end{array}$$

(18)

For the case of $P_1(t)+P_2(t)=1$, Equation (18) is rewritten as:

$$\begin{array}{l}D=\frac{2P_1(t)(1-P_1(t))({\displaystyle \sum _{i=0}^{t}i{p}_i}+{\displaystyle \sum _{i=t+1}^{L-1}i{p}_i})-(1-P_1(t)){\displaystyle \sum _{i=0}^{t}i{p}_i}-P_1(t){\displaystyle \sum _{i=t+1}^{L-1}i{p}_i}}{P_1(t)(1-P_1(t))}\\ \hspace{1em}=\frac{(1-2P_1(t))(P_1(t){\displaystyle \sum _{i=0}^{t}i{p}_i}+P_1(t){\displaystyle \sum _{i=t+1}^{L-1}i{p}_i}-P_1(t){\displaystyle \sum _{i=0}^{t}i{p}_i})}{P_1(t)(1-P_1(t))}\\ \hspace{1em}=\frac{(1-2P_1(t))P_1(t)(\mu -{\mu }_1)}{P_1(t)(1-P_1(t))}\end{array}$$

(19)

Under ${\mu }_1<\mu$, $P_1(t)>P_2(t)$ and $P_1(t)+P_2(t)=1$, the difference D satisfies D < 0. Thus, the total mean $\mu$ is closer to the background class. Moreover, the histogram of rice seedling image is a near-unimodal distribution. Hence, the optimal threshold T of segment rice seedling should be within the interval $[\mu ,L-1]$. As a result, the search range is set to $[\mu ,L-1]$ in the weighted Otsu method.

2.6. Rice Seedling Segmentation Procedure

After determining the threshold T, the classification process can be carried out to classify the pixel into either background or rice seedling in the current image. In this subsection, the detailed steps of the proposed segmentation algorithm for rice seedlings are described:

• Step 1: Convert an input image into the YCrCb color space with Equation (1) and compute Cb component using Equation (2).
• Step 2: Calculate the color-index, defined as 2Cg-Cr-Cb, to obtain the gray image.
• Step 3: Compute the total mean $\mu$ of the gray image obtained from Step 2.
• Step 4: Set the initial threshold t to the total mean $\mu$.
• Step 5: Classify the gray value i according to the threshold t. If i < t, assign the pixel to background class, or assign it to rice seedling class.
• Step 6: Calculate the gray level probability $P_1(t)$, the average value ${\mu }_1$ and the variance ${\sigma }_1^2(t)$ in the background class, and perform the same operation for $P_2(t)$, ${\mu }_2$ and ${\sigma }_2^2(t)$ in the rice seedling class.
• Step 7: Compute the within-class variance ${\sigma }_w^2(t)$ according to Equation (14) and t = t + 1.
• Step 8: Loop the process from Step 5 to Step 7 and find the minimum within-class variance using Equation (15). Next, set the threshold T to t at the minimum within-class variance.
• Step 9: Classify the pixel as rice seedling if i > T, or as belonging to the background otherwise.

3. Experimental Results and Discussion

3.1. Comparison of Different Approaches in Image Gray Processing

To validate the effectiveness of image graying process introduced in our paper, three different image graying approaches were used to process the rice seedling images in paddy fields. The results are showed in Figure 10. From Figure 10(A1) and Figure 10(B1), it can be seen that the gray value of the rice seedling extracted by the G component method shows a minimal difference from that of a paddy field background in a paddy field with shadow and highlight. Compared with the G component method, the rice seedling in the image grayed by the ExG method is clear, but the gray value of some rice seedlings still shows a small difference from that of a paddy field background on a cloudy day (shown in Figure 10(A2)) and the rice seedlings mixed with their shadow are highlighted on a sunny day (shown in Figure 10(B2)). For image processing with the 2Cg-Cr-Cb method, the grayed area of rice seedling can easily be distinguished from the background because the gray value of rice seedlings shows a relatively large difference from the background gray value. Therefore, the 2Cg-Cr-Cb method outperforms other methods in image graying processing and is robust to illumination variations.

3.2. Comparison of Segmentation Performance

To evaluate the performance of the proposed algorithm, we compared it with two other threshold segmentation methods: (1) the Otsu method, which is widely used to segment crops in agricultural visual navigation [24] (referred here as Otsu); (2) a modified valley-emphasis method [26] using the neighborhood information of the valley point to select the optimal threshold (referred to here as NVE). In the experiment, we selected 50 rice seedling images taken under different paddy field environments as per the test images shown in Figure 3, and 2Cg-Cr-Cb was used to obtain gray images. Figure 11 shows segmentation results of five test images obtained via Otsu, NVE and the proposed methods. The original test images taken in different weather (sunny, cloudy) and field (water, mud) environments are shown in the first column; the second, third and fourth columns show the segmentation results by the Otsu, the NVE and the proposed method, respectively, where rice seedlings are marked with white.

As seen in the second column, the Otsu method segments many background pixels as rice seedling pixels and fails to provide reliable information on rice seedling rows. Although the NVE method performs well compared with the Otsu method, it loses many rice pixels of seedling leaf edges and brings insufficient results. Clearly, the proposed method can provide reliable information on seedling rows and does not disturb the subsequent extraction of the navigation line.

Additionally, the performance of the proposed method can be evaluated quantitatively, and one common metric is used in this study: the Misclassification Error (ME) [27]. The ME metric describes the percentage of object pixels incorrectly detected as the background, and vice versa. The ME metric is defined as:

$$\mathrm{ME}=1-\frac{\left|B_{result}\cap B_{truth}\right|-\left|F_{truth}\cap F_{result}\right|}{\left|B_{truth}\right|+\left|F_{truth}\right|},$$

(20)

where $B_{result}$ and $F_{result}$ are the background and object pixels in the segmentation result, $B_{truth}$ and $F_{truth}$ are the background and object pixels in the ground truth labeled manually and $|•|$ denotes the cardinality of a set. The lower value of ME leads to a higher accuracy of segmentation. The ME results of the proposed method and of the other two methods are computed for the 50 test images which are presented in Figure 3, and the ME curves are plotted in Figure 12. It can be seen from Figure 12 that the ME results of the Otsu and the NVE methods are greatly affected by the paddy field environment because the Otsu and the NVE methods usually bring over-segmentation and under-segmentation of results, respectively, under complex paddy field environments. However, the segmentation performance of the proposed method improves on that of the other two methods and the proposed approach offers the lowest ME in the same image, which can provide reliable seedling row information for visual navigation system of automated rice tranplanters.

3.3. Processing Time Comparison

To obtain a systematic evaluation of the proposed method, the processing time of threshold searching was investigated. The experiment was implemented on a Lenovo T470 PC with an Intel Core-i7 processor and 8 GB RAM. In the experiment, the original image in size of 2560 × 1920 shown in Figure 7A was selected as the test image and the 2Cg-Cr-Cb was used to obtain gray images for the Otsu, NVE (the neighborhood valley-emphasis) and proposed methods. To compare the processing time in the images with different resolutions, we resized the test image to 320 × 237, 720 × 576 and 1327 × 1000.

Table 1. Processing time comparison (Unit: Second).

Resolution	Otsu	NVE	Proposed
320 × 237	0.005	0.027	0.003
720 × 576	0.014	0.122	0.009
1327 × 1000	0.047	3.851	0.031
2560 × 1920	0.204	13.385	0.119

shows the processing time of threshold determination. From Table 1, we can see that an increase in the image resolution leads to an increase in processing time. For an image of size of 1280 × 960, our method finds the threshold in around 0.031 s, compared with 0.119 s for an image of size 2560 × 1920. However, in the same case, the processing time of the Otsu method is more than double that of ours. The NVE method is the most time consuming due to the use of neighborhood information. Therefore, the proposed method with a constrained search range reduces the time consumption while providing better rice seedling segmentation performance compared with the Otsu and NVE methods. In addition, an automated rice transplanter usually moves at speeds of 1–2 m/s. In this regard, each image must be processed with time below 1.8 s and the segmentation consumes below the 20% of total time. For images of size 320 × 237, 720 × 576, 1327 × 1000 and 2560 × 1920, the processing time of obtaining 2Cg-Cb-Cr images is 0.010851 s, 0.3875 s, 0.104278 s and 0.391722 s, respectively. To reduce the computational load, the image size shrank from the original size of 2560 × 1920 to 320 × 237 pixels in our experiment. According to Table 1, the proposed method can be applied to real-time applications.

4. Conclusions

This paper supplies an improved Otsu method for rice seedling segmentation in the YCrCb color space. We built the color-index 2Cg-Cr-Cb based on the YCrCb color space rather than the RGB color space in the image graying process, which reduces the influence of illumination variation on image graying. We then used the cumulative probability of rice seedling occurrence as an adaptive weight on the background class variance of within-class variance. Comparing the weighted method with the traditional Otsu method and the NVE method, the threshold determined by the weighted method is close to the desired threshold. The Otsu method’s threshold biases towards the background class, which misclassifies background pixels as seedling pixels, while the NVE method’s threshold causes over-segmentation. We constrained the search range to improve the efficiency of the weighted method. Experimental results demonstrate that the proposed method can achieve better segmentation performance than the other two algorithms while reducing the cost of calculating the threshold which can provide reliable seedling row information for visual navigation system of automated rice tranplanters. Moreover, accuracy navigation line can ensure the transplanter plants seedlings along the seedling row without damaging seedlings and improve the working quality of rice transplanter, which are beneficial to increasing rice yield.