1. Introduction
The integration of contemporary information technology, engineering technology, and other high-tech applications to agricultural output is known as intelligent agriculture and precision agriculture. Modern agricultural technology: As a representative agricultural product, the total annual output of edible fungi in China accounts for more than 85% of the total annual output of edible fungi in the world. The edible fungi industry has become an indispensable role in ensuring farmers’ “vegetable basket”, promoting regional industrial economic development and export exchange [
1]. In recent years, whether from the perspective of nutrition or health care, domestic people have a higher and higher awareness of the edible fungi represented by mushrooms, which makes the consumption of this kind of edible fungi increase rapidly. With the continuous penetration and deepening of industrialization, factories and intelligent cultivation modes, mushroom production enterprises are also developing and growing. Under the new production mode, most of the production processes in the mushroom greenhouse have been mechanized and automated. However, in the process of mushroom picking statistics, weighing operation and inventory management, the existing production mode still adopts traditional manual basket handling to weigh the weight of mushrooms, and manually records the mushroom grade, inventory and other information, which leads to the problems of high labor intensity, low efficiency and ineffective docking of weight information with the information management system, making the production information easy to tamper with, and thus, it is difficult to achieve efficient, accurate and transparent inventory management.
This study has great research value in shiitake grading measurement. It is mainly used in the grading process and weighing measurement process, which can reduce labor costs, reduce errors and improve production efficiency. Under the background of the existing production mode of the shiitake industry, exploring intelligent agricultural machinery and equipment to promote the intelligent development of the shiitake industry is an urgent problem to be solved in the shiitake industry. Therefore, according to the actual operation needs of enterprises, combined with the shiitake greenhouse scene, this study developed a special robot for shiitake grading measurement that integrates an intelligent navigation decision-making service and information collection function, so as to realize the real-time, intelligent, accurate and efficient acquisition of shiitake grading measurement information, promoting the intelligent development of agricultural machinery and equipment, accelerating the internationalization of shiitake standards and promoting the intelligent development of the shiitake industry [
2].
With the continuous penetration and deepening of industrialization and intelligent cultivation modes, mushroom production enterprises are also developing and growing. At present, the degree of automation in
Lentinula edodes grading in China is not high. Ge Liang [
3] designed a real-time grading production line for dried
Lentinula edodes, integrating feeding, identification, transportation and grading functions. However, the turnover mechanism cannot meet the requirements of fresh
Lentinula edodes turnover, and it is difficult to realize the grading of air-blown fresh
Lentinula edodes. The grading and weighing process of shiitake mushrooms in shiitake mushroom factories still adopts the method of manually recording the grade and weight of shiitake mushrooms. The shape and size of shiitake mushrooms mainly depend on the judgment of human eyes, through which it is difficult to meet the accuracy requirements of grading and weighing. Heinemann et al. [
4] built a mushroom detection and grading system. The external quality characteristics (color, shape, stem incision and umbrella opening degree) of white double-holding mushrooms were quantitatively analyzed by the image analysis method, and the automatic detection and grading of mushrooms were realized. The average error rate of the grading system for mushroom grading was 20%, while the inconsistency rate of manual detection was 14∼36%. As a result, the mushroom factory encountered difficulties in efficient, accurate and transparent inventory management.
With the development of image detection technology extended to the field of edible fungi, detection technology based on image processing has obvious advantages. The integration of artificial intelligence technology and image recognition technology makes computer image recognition more intelligent, gives full play to the advantages of intelligent image recognition technology and plays an irreplaceable role in some fields [
5]. Hwang [
6] combined machine vision and artificial neural networks to develop a grading image processing algorithm for dried mushrooms, and developed two automatic grading systems and grading software for dried mushrooms. The system is composed of a feeding device, a turnover device and a computer vision system with a lighting chamber, a microcontroller and a pneumatic actuator. Experiments show that the performance of the two-step grading and sorting system is about 0.3 s, and the classification accuracy of each mushroom is more than 95%. Using computer vision and image processing technology, the image of
Lentinula edodes was collected and preprocessed, and the characteristics of cap diameter, thickness and color were extracted from the image of
Lentinula edodes [
7]. Xia Qing [
8] used image processing technology to classify dried mushrooms, but the image processing recognition process was complicated and the model was thick. In this study, the defect detection and appearance quality classification of
Lentinula edodes were completed according to the classification standard of fresh
Lentinula edodes. Aiming at the problem of irregular arrangement of mushrooms, it is necessary to determine the position of the mushrooms and extract the boundary of the mushrooms. After a large number of test sample images, it was found that only using the OpenCV function library cannot maintain stability, and the accuracy rate was also greatly reduced for the more complex images. Chen et al. [
9] removed heterochromatic mushrooms by image analysis, calculated the ratio of mushroom cap area to perimeter to detect whether mushrooms were damaged, and counted the number of image pixels to determine the size of mushrooms. The developed grading system used PLC as a grading controller to receive the grading instructions of the host computer and perform grading actions. The grading accuracy rate reached 97.6%, but the grading time was long and the efficiency was low. In order to effectively solve the problem, the mushroom grading method based on Yolov5 + OpenCV is optimized to make it more suitable for irregularly arranged mushroom recognition and to improve the detection performance of small targets. The mushroom metering system based on a resistance strain gauge sensor provides technical support for actual production and promotes the intelligent development of the mushroom industry.
The edible fungus production statistics robot, which integrates an intelligent navigation decision-making service and information collection function, realizes the whole industrial chain coverage and seamless connection of the industrial chain information of the edible fungus factory. The yield, quality and other information of edible fungi products can be traced, promoting the deep integration of information engineering technology and the edible fungi industry, and building a national modern agricultural industrial park with distinctive edible fungi characteristics, reasonable industrial layout, efficient resource utilization and significant radiation demonstration effects.
The organization of the thesis is as follows: The second partis the related work of the research;
Section 3 introduces the experimental methods of this study;
Section 4 verifies the effectiveness of this study through experiments;
Section 5 discusses the feasibility of the study according to the experimental results of each group;
Section 6 contains the summary.
2. Related Work
This section describes the professional robot measurement of mushroom grading. In order to realize the effective grading measurement of mushrooms, it is compared with other grading measurement methods. In order to effectively combine robot technology with intelligent agriculture, a special robot for the grading and metering of mushrooms is proposed. In order to improve the flexibility and intelligence of the robot, a self-propelled mobile chassis and automatic path planning algorithm are developed.
2.1. Classified Metering
With the continuous development of computer vision recognition technology, edge recognition algorithm is gradually applied to the field of agricultural product recognition. Using the recognition algorithm to classify agricultural products is more efficient in the classification of agricultural products. For example, Bhargava [
10] proposed an automated algorithm based on machine learning to detect the types and quality grading of different vegetables and different varieties of fruits. Bandyopadhyaya [
11]’s microcontroller-based robot system is used for tactile sensing to classify vegetables. Usha [
12] introduced a vegetable quality assessment based on computer vision for eggplant and strawberry. Unay [
13] proposed a multi-spectral machine vision system based on apple fruit grading, which has multiple gradings. The results show that the cascade structure is more useful than the single structure. Biswas and Hossain [
14] proposed a real-time broccoli vegetable recognition system, which uses hue, saturation and value color histograms to classify vegetables. Rong’s [
15] window-based segmentation is used to detect defects on the surface of oranges, and citrus recognition based on color index is performed by acquiring RGB images. However, the recognition accuracy of the above recognition algorithm for objects with uneven shape and small volume needs to be improved. In this paper, a recognition algorithm of yolov5 + opencv is proposed to identify objects with uneven shape and small volume.
2.2. Agricultural Robot
After the development of industrial robots, robots have been gradually applied in the field of agriculture. With the continuous development of robotics, robots with different functions have emerged [
16,
17,
18]. In terms of agricultural information acquisition robots, Bao et al. developed a mobile crop phenotype analysis robot in the field, which realized the high-throughput measurement of crop stem scale strength and geometry [
19]. Kayacan et al. from the University of Illinois developed a light and small robot, ‘TerraSentia’, applied to the canopy of a corn field. The robot uses a machine vision algorithm to automatically drive through the field to collect crop data. Using a deep learning algorithm, it can also monitor early plant growth vigor, identify diseases and estimate crop yield [
20]. Mohd Saiful et al. developed a spraying robot, which uses intelligent equipment such as nozzles and medicine tanks to use the robot ’s autonomous navigation in the real environment, and then realizes the function of spraying pesticides according to the set action [
21]. Majeed et al. designed a vineyard pruning robot, used RGB-D machine vision to detect the structure of fruit trees, optimized the decision-making operation points, and realized the autonomous pruning operation on the move [
22].
At present, agricultural information acquisition and agricultural operation robots have carried out research and applications such as transplanting and environmental control. However, the development of agricultural robots still needs to focus on breaking through the problems of applicable cost, operation efficiency, use reliability and category diversification, as well as further improving the breadth and scale of technical demonstration. In the research process, it is necessary to analyze the characteristics of the operation object, combine the agricultural operation robot with the production management process and further improve the practicability of the robot research scheme. Because there are few studies on the robot in the mobile grading measurement of mushrooms, this paper develops a special robot for mushroom grading measurement in mushroom picking and the weighing of mushrooms in mushroom factories.
2.3. Robot Path Planning
Li Ning uses the global path planning algorithm to search for the best path in the known path. However, if the surrounding environment of the robot changes, the best path obtained by the global path planning originally used in the static map is no longer used [
23]. Aiming at the problems of slow path planning speed and too many turns in the conventional genetic algorithm, the turning angle is added to the adaptive function as a control factor, and finally, the size of the adaptive function is constrained by the turning angle and the path length. Zhang Hongyang used MATLAB to verify that the path planned by the improved algorithm has fewer turns and is smoother [
24]. In addition, Korkmaz used the path planning obtained by the improved artificial potential field algorithm to inspire the ant colony algorithm, and finally improved the shortcomings of the ant colony algorithm that is easy to fall into the local optimal solution [
25]. In the aspect the of dynamic window algorithm, Mohamed proposed an improved dynamic window algorithm based on adaptive parameters to ensure that the robot can move quickly while maintaining a safe distance from the obstacle [
26].
The path planning technology of mobile robots has become more and more mature. However, due to the complexity and variability of its application scenarios, there are still many practical problems to be solved in the path planning technology, and there is still a lot of room for development. Although the existing algorithms have certain limitations and are mostly applied to simple planning scenarios, the basic algorithms still have their applicable scenarios, and these algorithms are an important basis for practical applications. Therefore, it is still of great significance to optimize the limitations of different path planning algorithms. More mathematical theories can be applied to optimize path planning algorithms, such as improving the cost function of the A * algorithm and optimizing the fitness function of the genetic algorithm.
3. Methods
This part takes lentinus edodes as the research object, and introduces the hardware system implementation, mushroom grading algorithm, weighing system and automatic planning path implementation method of mushroom grading, as well as a professional metering robot. The robot object and simulation experiment photos are displayed, and the realization of each function is described in detail.
3.1. Hardware Establishment
In order to improve the intelligent level of the edible fungi industry, meet the digital needs of the whole industrial chain of edible fungi and realize the collection of heterogeneous data information such as the grade and weight of mushroom products, according to the digital demand of the whole industrial chain of mushrooms, combined with the basic conditions of mushroom factories and the operation needs of mushroom picking and sorting, warehousing weighing, inventory management and other links, a special robot for mushroom measurement is designed and developed.
The main control unit of the information acquisition system uses MSP430F5438 A single chip microcomputer, equipped with ADAM Advantech acquisition card, and connects various types of sensors such as current type, voltage type and 485 protocol. Weighing sensor, RFID module, voice module, barcode scanning module, face recognition module through the data acquisition card to RS-485 protocol and MSP4305438A master control unit achieve data information transmission and reception; the speech recognition module, speech synthesis module and display control module interact with the main control unit through the TTL interface. The main control unit of the information acquisition system communicates with the wireless data transmission terminal VTU through TCP. The TCP/IP protocol is used to realize long-distance data transmission and reception, and the information collected in the mushroom shed is sent to the server for storage.
The robot system structure diagram is shown in
Figure 1:
The specific functions are as follows:
Mushroom grading: Based on the intelligent robot, the mushroom grading algorithm based on Yolov5 + OpenCV is used to identify the image and measure the size of the mushroom cap. According to the established mushroom grading measurement standard, the mushroom is graded accordingly.
Mushroom measurement: Mushroom measurement system is based on resistance strain gauge sensor. When the sensor undergoes elastic deformation, the resistance value of the resistance strain gauge changes, and the strain of the mushroom weight through the equivalent resistance is converted into an analog signal (1) and (2). The weight of different grades of mushrooms is converted into voltage signal output according to the linear relationship. The analog signal is processed by the detection circuit, and the amplified analog signal is converted into the weight data of mushrooms through the AD conversion sub-module.
Analog signal output formula:
Linear relationship conversion formula:
- 3.
Autonomous obstacle avoidance function: The multi-sensor fusion SLAM algorithm suitable for ROS environment is studied. The global exploration is used for path planning and autonomous navigation, and the dynamic window method is used for local obstacle avoidance. Among them, the red grid points in
Figure 2 represent dynamic obstacles moving up and down, and
Figure 3 represents additional unknown obstacles. The blue grid point represents the starting point; the green grid point represents the target point; the black dotted line is the global path planned before the obstacle is added; the blue solid line is the actual trajectory of the robot. The function simulation experiment is shown in
Figure 2 and
Figure 3.
The navigation path of the robot is constantly updated with the change in the target point or obstacle so that the robot can move autonomously and avoid obstacles in the mushroom shed. The automatic obstacle avoidance simulation experiment is shown in
Figure 4 and
Figure 5.
- 4.
The automatic navigation and robot mobile chassis are shown in
Figure 6 and
Figure 7.
3.2. Mushroom Grading Measurement Standard
In this experiment, by synthesizing the ‘mushroom quality grad’ standards formulated throughout the country, the mushroom grading measurement standards are summarized as follows:
Grade 1 mushroom: mushroom shape round, all open umbrella or slightly curled, more than 6 centimeters in diameter.
Grade 2 mushroom: the mushroom shape is round, the cap is fully open and flat, with a diameter of more than 4 cm.
Grade 3 mushroom: mushroom shape is not neat, the diameter of the cap is less than 4 cm and more than 2 cm.
3.3. YOLOV5
YOLOV5 is divided into four different network structures, namely YOLV5s, YOLOV5m, YOLO5L, YOLOv5x. According to the requirements of mushroom grading, the recognition algorithm should have certain real-time and high efficiency. YOLOv5 is a pre-trained object detection architecture and model series on COCO data set. It is an extension of YOLO series. The data set adopts COCO. Its network structure is divided into four modules: input, backbone, neck and head. Compared with YOLOv4, YOLOv5 has greatly improved performance in speed and accuracy, thanks to operations such as Mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling. In the network training stage, YOLOv5 uses Mosaic data augmentation operation to improve the training speed of the model and the accuracy of the network. However, when using the Yolov5 algorithm for image recognition, the size of the mushroom cap cannot be identified and marked, resulting in the size of the mushroom not being able to be identified. Therefore, this paper improves the Yolov5 algorithm to improve the practicability and recognition accuracy of its practical application.
Figure 8 is the network structure of yolov5.
3.4. Algorithm Optimization
In this study, the method of mushroom size detection based on OpenCV pixel/metric ratio algorithm was used. The core of the algorithm is to establish the pixels_per_metric ratio index of the image by using the reference object in the image (which satisfies the real size that is known and easy to obtain, called the reference object). It is used to measure the number of pixels for each given target, that is, the ratio of the pixel width of the object to the true width of the object, pixels_per_metric = object_width/know_width. The size of the mushroom in the image is calculated by this index. At the same time, the OpenCV method is integrated into the Yolov5 recognition algorithm to improve the Yolov5 algorithm.
In this algorithm, the storage path of the mushroom image to be tested and the size of the reference object are used as two input parameters, and the Python package named imutils package is used. At the same time, the midpoint function for calculating the midpoint of two coordinates is defined. First, the image is grayed and smoothed by a Gaussian filter. On this basis, the edge of the mushroom image is detected, and the gap between the edges is eliminated by expansion and corrosion, and the edge of the mushroom is extracted. Then, the cv2.cv.BoxPoints function of OpenCV 2.4 and the cv2 ḃoxPoints method of OpenCV 3 are used to calculate the rotation bounding box of the image. The coordinates of the bounding box are arranged in the order of upper left, upper right, lower right and upper left. The midpoint function is used to calculate the midpoint of the four edges of the bounding box and connect them accordingly. Finally, the pixels_per_metric variable is initialized by the Euclidean distance between the midpoint sets, and the actual size of the mushroom is calculated by combining this variable with the Euclidean distance between the midpoints of the side lengths of the mushroom bounding box.
In this study, circular and elliptical regions were compared with the smallest quadrilateral outside the contour. It was found that the use of circular and elliptical regions under the irregular umbrella cover of Lentinula edodes exceeded the identification anchor frame or insufficiently identified the boundary of the anchor frame, resulting in inaccurate measurement of the diameter of the shiitake umbrella cover, which in turn affected the classification accuracy of Lentinula edodes.
3.5. Mushroom Metering System Composition
The mushroom metering system takes the resistance strain gauge weighing sensor as the main body, including the detection circuit and the sensor protection device. The resistance strain gauge sensor is linked to the acquisition master microcontroller through the detection circuit. The detection circuit includes an amplifier and an AD converter, thereby realizing the amplification and AD conversion of the weighing sensor signal. The structure diagram of the resistance strain gauge sensor is shown in
Figure 11, and the equivalent circuit diagram is shown in
Figure 12. It is composed of resistance strain gauge 1, weighing tray 2, elastic element 3, weighing chassis 4, bracket 5 and 6. When the resistance strain gauge is subjected to elastic deformation, the resistance value of the resistance strain gauge 1 will also change.
When the mushroom is weighed, the resistance value of the resistance strain gauge changes when the sensor is elastically deformed. The weight of the mushroom is converted into an analog signal through the strain of the equivalent resistance. The weight of different grades of mushrooms is converted into a voltage signal output according to the linear relationship. The analog signal is processed by the detection circuit. The detection circuit diagram is shown in
Figure 13,the operational amplifier sub-module includes an operational amplifier U6 (model AD8656). The + INA and + INB pins of the operational amplifier U6 are connected to the positive and negative s a and s b of the analog signal UOUT, respectively. The-INA and-INB pins of the operational amplifier U6 are connected by resistance R a, the-INA and OUTA pins of the operational amplifier U6 are connected by resistance R b, and the-INB and OUTB pins of the operational amplifier U6 are connected by resistance R c. The OUTA pin of the operational amplifier U6 outputs a positive signal S a that amplifies the analog signal, and the OUTB pin of the operational amplifier U6 outputs a negative signal S b that amplifies the analog signal. And the amplified analog signal is converted into the weight data of the mushroom through the AD conversion sub-module.