Automatic Brix Measurement for Watermelon Breeding

Abstract

Sweetness or sugar content, represented by soluble solids contents (SSC), is a vital quality trait in watermelon breeding which can be assessed by the refractive index method. However, sampling watermelon juice out of the pulp is a process that is both labor-intensive and error-prone. In this study, we developed an automatic SSC measurement system for watermelon breeding to improve efficiency and decrease costs. First, we built an automatic cutting system to cut watermelons into precise halves, in which a laser rangefinder is used to measure the distance from the upper surface of the watermelon to itself, and thus, the diameter is estimated. The experiments showed a high correlation between the estimated diameters and the ground truths, with $R^2 = 0.9608$ and $\mathit{RMSE} = 4.321$. Then, we built an automatic Brix measurement system to obtain the Brix data from a central point on the watermelon’s section, where an image analysis procedure is applied to locate the testing point. This is then transformed to the camera coordination system, and a refractometer is driven by a 3-axis robotic arm to reach the testing point. Brix measurement experiments were conducted using three vertical gaps and four lateral gaps between the probe of the refractometer and the pulp. The result showed that the best parameters were a vertical gap of 4 mm and a lateral gap of 2 mm. The average accuracy reached 98.74%, which indicates that this study has the potential to support watermelon breeding research.

1. Introduction

Watermelons (Citrullus lanatus) are important and highly cultivated fruit crops in many countries for their large amount of edible sweet flesh. Their total production worldwide was reached 101.6 million tons in 2020 [1]. Watermelon breeding, enhancing their productivity, flavor, and resistance to stresses from the environment, has become a hot topic in many countries [2,3,4]. China, for example, commercially released 2103 watermelon cultivars between 2017 to 2022 [5]. Among the various phenotyping traits that can be accessed using different technologies, sweetness or sugar content, which is represented by soluble solids contents (SSC), is a vital quality trait of watermelons. It can be assessed via non-destructive and destructive methods. Non-destructive SSC measurement methods include electrical methods [6,7], acoustic response analysis [8,9,10], infrared spectroscopy [11,12,13], and hyperspectral imaging [14]. The author in [6] used a low-cost electronic instrument to estimate the SSC of watermelons from density and mass by multiple regression analysis. The author in [7] used an open-ended coaxial-line probe and an impedance material analyzer to study the relationships between the frequency of the signal and the dielectric characteristics of the melon sample. He et al. [15] provided a pendulum hitting device to study the curves of the sound waveform of watermelons using a fast Fourier Transform method, which reached a correlation between the peak frequency of power spectral density and the SSC of 0.83, with a 5% confidence level. The researchers in [16] developed a watermelon quality inspection system based on acoustic technology, where the correlation between the transmission velocity and SSC of watermelons reached 0.81–0.95 for the different striking points of various growth statuses of watermelons. Due to the development of chemometrics, infrared spectroscopy, especially near-infrared spectroscopy, has been widely used in many nondestructive determination tasks, including determination of the SSC of watermelons. Dull et al. [17] used a near-infrared transmission method to assess the SSC of honeydew melons, which resulted in a correlation coefficient of 0.85. The authors in [18] developed a method to nondestructively determine the SSC of watermelons by applying multiple regression with wavelengths of 902 nm and 872 nm, which resulted in a root mean square error of calibration of 0.87. Tao et al. [19] developed a method to determine their SSC by combining near-infrared reflectance spectroscopy and dielectric properties of the watermelons, which resulted in a correlation coefficient of calibration of 0.84. The authors in [20] used a field type near-infrared spectroscopy instrument with a wavelength of 700-1050 nm to measure the sweetness of golden melons, which resulted in a correlation coefficient of prediction of 0.85%. Another technology that can be used in non-destructive SSC measurement of watermelons is hyperspectral imaging, which obtains the spectrum for each pixel in the image of an object [21]. The authors in [22] used the hyperspectral imaging method to non-destructively measure the SSC of grapes, achieving a correlation coefficient of 0.77. Weng et al. [23] used features extracted from a hyperspectral reflectance imaging system of 400–1000 nm to non-destructively assess the SSC of strawberries, which resulted in a determination of prediction of 0.937. Li et al. [24] used a near-infrared hyperspectral imaging system of a wavelength of 874–1734 nm to estimate the SSC of harvested cherries, which resulted in a standard deviation of prediction error of 2.7.

Due to the high efficiency and low-cost features of these techniques, non-destructive SSC measurement methods are applied extensively in the fruit industry. However, the low accuracy and robustness of these methods mean that they are not suitable to be used for the accuracy-sensitive breeding task, because it is difficult to compare various cultivars using an inaccurate tool. The destructive SSC measurement methods are usually performed using the refractive index method, which is accurate and stable enough for breeding tasks [25]. However, sampling watermelon juice out of the pulp is a time-consuming process, which includes manually cutting the watermelon into halves, crushing the pulp to extrude juices, and sampling juice via pipettes. It is labor-intensive and error-prone when handling a huge number of samples. In this study, we developed an automatic SSC measurement system for watermelon breeding in order to improve its efficiency and decrease costs.

2. Materials and Methods

The watermelon variety used for the study was ZaoJia 8424, which is bred and released in China. The watermelons were grown in an open field located at a longitude of 28.18$°$ and a latitude of 113.07$°$, and were picked on 11 July 2022, 119 days after sowing. A total of 25 watermelons were sampled from a single plot for the experiments. On the day they were picked, as shown in

Table 1. Information about watermelon samples.

No.	Weight/kg	Vertical Diameter/mm	Diameter Along 45°/mm	Horizontal Diameter/mm
1	0.521	103.0	103.1	101.9
2	0.689	112.1	110.5	110.7
3	0.838	121.5	120.0	120.0
4	0.875	115.7	126.5	123.1
5	1.088	126.6	128.6	130.0
6	1.135	128.7	127.5	127.5
7	1.204	134.5	132.7	131.5
8	1.231	131.1	131.5	134.0
9	1.365	139.0	138.0	139.0
10	1.581	145.3	146.0	142.2
11	1.970	155.5	156.6	156.5
12	2.046	152.0	153.5	156.5
13	2.218	162.8	161.5	160.1
14	2.226	166.7	162.8	160.1
15	2.237	159.3	160.5	162.0
16	2.256	165.8	165.2	162.0
17	2.350	161.3	160.9	164.4
18	2.371	159.3	164.2	167.5
19	2.388	169.8	163.0	161.0
20	2.399	163.1	164.2	165.3
21	2.463	159.5	160.0	171.0
22	2.832	168.9	166.5	171.3
23	2.836	175.7	176.3	173.7
24	3.039	181.5	182.6	177.0
25	3.336	182.7	179.1	173.3

, the vertical diameter, horizontal diameter, and diameter along 45° of the watermelons were measured using a lathe slide with an accuracy of 0.05 mm; the weights were measured using an electronic balance with an error within 0.1 g.

2.1. Automatic Brix Measurement System

The automatic Brix measurement system includes a laser rangefinder, 2 Red-Green-Blue (RGB) cameras, a refractometer, a button-pushing servo motor, a 3-axis robotic arm, and a cutting device. As shown in Figure 1, the working procedures of this system are as follows: (i) the diameter of the watermelon is acquired using the laser rangefinder; (ii) the watermelon is cut into precise halves, then the upper half is removed by a sucker; (iii) the position of the testing point is calculated after a set of image analysis processes and the refractometer is driven into the testing point; (iv) the START button on the refractometer is pushed using a servo motor, then the digital tube is recognized using the image analysis method.

The laser rangefinder, produced by the Ruxing company in China, has a measurement capacity of 0.05~80 m, an accuracy of 1.0 mm, and a laser band between 620~690 nm. The RGB cameras, manufactured by the Weixinshijie company in China, have a resolution of $1920 \times 1080$. The refractometer is PEN-PRO, which is manufactured by the Atago company from Japan, and has a measurement range of Brix 0.0 to 85.0%, an accuracy of ±0.2%, and a temperature compensation range of 10 to 100 °C. The 3-axis robotic arm is constructed with linear stage actuator, where the refractometer is mounted on its end. The cutting device consists of a driven cylinder, an air compressor, two slideways, a blade-supporting shelf, a blade driven by the cylinder, and a blade position adjustment module driven by a step motor linear actuator.

2.2. Automatic Watermelon Cutting

In breeding research, samples are cut into halves along the texture on the surface to obtain section information [26]. In order to perform this task automatically, we approximate the watermelons as ellipsoids for analysis, using a V-groove for fixing, as shown in Figure 2. Therefore, the radius of the watermelon $R$ is calculated as follows:

$$R = \frac{H - z - a}{\frac{1}{\sin \left(\theta \right)} + 1}$$

(1)

where $H$ is the distance from the laser rangefinder to the base surface, $z$ is the distance from the laser rangefinder to the top point of the watermelon, $a$ is the distance from the bottom of the V-shaped groove to the base surface, and $\theta$. is the angle between one side of the V-groove and the vertical plane.

Thus, to cut the watermelon into equal halves, the height of the cutting blade from the base surface is $a + b$, where $b$ is the distance from the bottom of the V-shaped groove to the imaginary center of the watermelon.

$$b = \frac{R}{\sin \left(\theta \right)}$$

(2)

2.3. Automatic Watermelon Brix Measurement Method

2.3.1. Testing Point Localization Based on Image Analysis

In order to obtain the pixel coordinates of the center measurement point, as shown in Figure 3, we use the following equation to extract the pulp mask $M$:

$$M = \mathit{Otsu}\left(R - 1.25G\right)$$

(3)

where $R$ is the red channel of the image, $G$ is the green channel, and $\mathit{Otsu}$ is the Otsu method for binarization calculation [27].

Then, the smallest circumscribed rectangle for the extracted pulp area is generated, and the center of the rectangle and the position, which is offset by 0.425 times the side length from the center of the rectangle along the short side direction, are set as the measurement points, as shown in Figure 3.

In order to drive the Brix measurement sensor using a 3-axis robotic arm, the pixel coordinate of the measurement points is converted into a camera coordinate. In the camera coordinate $\mathit{OXYZ}$, as shown in Figure 1, since the laser rangefinder is perpendicular to the watermelon section, the $z$ value of any point on the watermelon section can be easily obtained. The coordinates of any point on the watermelon section can be obtained through the pinhole imaging model [28].

$$x = \frac{x_{\mathit{pixel}} - C_x}{f_x} · z$$

(4)

$$y = \frac{y_{\mathit{pixel}} - C_y}{f_y} · z$$

(5)

where $x$ and $y$ are the coordinate values, mm; $x_{\mathit{pixel}}$ and $y_{\mathit{pixel}}$ are the image coordinate values, pixel; $C_x$ and $C_y,$ which are generated by the camera calibration, are the camera optical center coordinates, pixel; $f_x$ and $f_y,$ which are also generated by the camera calibration, are the two components of the camera focal length, pixel; and $z$ indicates the distance from each point in the image to the camera along the $Z$ axis, namely the distance from the watermelon section obtained by the laser rangefinder to the camera.

2.3.2. Refractometer Manipulation Strategy

The process to conduct a Brix measurement using PEN-PRO is to dip the probe into the sample and press the START key. Since it was designed based on the principle of refraction, which means it works with liquid samples such as fruit juices, the measurement would fail when dipping it into a solid or hemi-solid sample such as watermelon pulp, as the emitted light would be blocked. Therefore, it is necessary to maintain a certain gap between the sensor and the pulp to accommodate the juice. As shown in Figure 4, a 3-axis robotic arm is used to drive the sensor controlling the lateral and vertical gaps.

2.3.3. Digital Tube Recognition Using Image Analysis

Since the PEN-PRO refractometer has no data interface, it is designed only to be read through a digital tube screen. To achieve automatic measurement, we use an RGB camera to capture the images of the digital tube, where the camera and the refractometer are installed in relatively fixed positions. As shown in Figure 5, the recognition procedure is as follows: (i) converting RGB images into gray as Input; (ii) dividing each digital area in the grayscale image into 7 ROIs according to the positions of the digital tubes; (iii) creating a new image, namely Output, and setting its values with the following equation

$$\mathrm{Output}\left(\mathrm{x},\mathrm{y}\right) = \left\{\begin{array}{c}255 \mathrm{while} \mathrm{Input}\left(\mathrm{x}, \mathrm{y}\right) > 260 - \mathrm{Mean}\\ 0 \end{array}\right.$$

(6)

where x and y are the image indices, and the Mean represents the average gray value of each ROI. By calculating the number of non-zero pixels contained in the ROIs representing the position of the digital tube in the generated image, it can be easily determined whether the digital tube is activated according to some specific patterns.

3. Results

3.1. Watermelon Diameter Measurement Performance

To evaluate the measurement performance of the distance sensor when facing the surface material of the watermelon, we obtained the diameter of 25 watermelon samples using the direct distance detection method as follows: (i) fixing the distance sensor perpendicular to a plane; (ii) after measuring the distance of the plane, placing the watermelon and measuring the distance between the watermelon and the sensor. The difference between the two values is the measurement result of the watermelon diameter.

We compared the measurement results using an RGB-D camera, RealSense D455, manufactured by Intel, as well as the laser rangefinder which was previously mentioned. As shown in

Table 2. The correlation results of watermelon diameter measurement using different sensors.

Sensor Type	${\mathit{R}}^2$	RMSE
RealSense D455	0.9975	1.136
Laser rangefinder	0.9902	2.163

, RealSense D455 showed $\mathit{RMSE}$ of 1.136, which is lower than that of the laser rangefinder. However, since RealSense D455 has a relatively higher cost than the laser rangefinder, we chose the latter for further experiments.

We placed every watermelon sample on the V-groove, then measured the diameters along the three directions in Table 1 using the laser rangefinder. The result showed that, as shown in Figure 6, there was a high correlation between the measurements and the ground truths, with $R^2 = 0.9608$ and $\mathit{RMSE} = 4.321$.

3.2. Manipulation Parameters of the Refractometer

In order to obtain feasible measurement parameters, we conducted a two-factor experiment, in which the first factor was the lateral gap, with four levels of 2 mm, 3 mm, 4 mm, and 5 mm, and the second factor was the vertical gap, with three levels of 2 mm, 3 mm, and 4 mm. There were 12 parameter settings in total. For every setting, we chose three new testing points on a watermelon’s section, and for each testing point, we recorded the results 10 times to generate an average. The ground truths were obtained using PAL-1, manufactured by the Atago company and with Brix accuracy of $\pm 0.2\%$, by manually dropping the sampled watermelon juice into its sensor via a pipette with three repeats for an average result. The experiments were finished within 5 days after watermelon samples were picked.

As shown in Figure 7, the accuracies were turbulent, from 59.88% to 99.12%, when the lateral gap was 3 mm and 4 mm, and when the vertical gap was 2 mm, respectively. This is likely because the squeezed watermelon pulp rebounds slightly after release, when the vertical gap is small enough, the measurement light beam may be blocked. When the vertical gap was 3 mm and 4 mm, we found that the smaller the lateral gap, the higher the accuracy. This is because when the lateral gap is small, the watermelon juice is not easily lost through infiltration, so enough liquid can be maintained in the hole, keeping PEN-PRO functioning well. Therefore, we obtained the best Brix measurement parameters, namely 2 mm for the lateral gap and 4 mm for the vertical gap, with an average accuracy of 98.74%.

4. Discussion

The development of breeding technology plays a vital role in improving agricultural crops. Currently, research on breeding improvement techniques is a hot topic, and has been widely applied in the breeding of various grain crops and fruits. However, whether it is haplotype-based speed breeding, transgenic breeding, or gene editing breeding, they all face the same problem, namely high cost, due to factors such as the high equipment and labor costs required when conducting large-scale cross-processing experiments [29]. The technologies for phenotyping crops have significantly improved the efficiency of information acquisition of crop traits during the breeding process.

In this study, we proposed a solution for automatic large-scale Brix measurement of watermelon for breeding purposes. We also conducted a set of experiments to obtain the important parameters of the manipulation, providing certain technical support for the watermelon breeding research field. However, some deficiencies that appeared in this study need to be further investigated by researchers. First, when squeezing the pulp of a watermelon with a pen-type sensor, which, in this study, was PEN-PRO, there may be juice overflow or tipping moment, possibly causing errors in the measurement. The robustness and efficiency of this kind of method can be improved by optimizing the structure of the sensor’s probe, reducing the squeezing force, and avoiding the outflow of juice. Moreover, the method of cutting the watermelons in half in this study was achieved by the squeezing force of the blade on the watermelon. This means when the rind is tough, the cutting resistance will change abruptly, which is not conducive to forming a complete cutting surface. The cutting device can be enhanced by changing the angle of the blade appropriately, so that there is a shear force between it and the watermelon during the cutting process. Third, the sampling speed of this machine is around 1 sample per minute, which is close to the speed of manual sampling. The sampling speed can be increased by applying high-speed step motors, decreasing the inertia of movable parts, and using sharper blades. Since the process of this machine is automated, the labor cost of using it would simply include placing the sample onto it and recording the results, but this process will be replaced by an additional automatic feeder and recorder.

5. Conclusions

In this study, we proposed a device to automatically measure the sugar content of watermelons for breeding, including an automatic half-cutting system and a Brix measurement system. In the automatic half-cutting system, the measured values of the watermelon diameters showed a high correlation with the real values, with $R^2 = 0.9608$ and $\mathit{RMSE} = 4.321$; in the Brix automatic measurement system, the optimal Brix measurement parameters were a lateral gap of 2 mm and a vertical gap of 4 mm, with an average accuracy of 98.74%. This research can provide support for the breeding of elite germplasm of watermelon.