Next Article in Journal
Blind Watermarking of Color Medical Images Using Hadamard Transform and Fractional-Order Moments
Previous Article in Journal
Stainless-Steel Antenna on Conductive Substrate for an SHM Sensor System with High Power Demand
Communication

Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach

1
College of Mechanical and Electronic Engineering, Shandong Agricultural University, Tai’an 271018, China
2
College of Information Science and Engineering, Shandong Agricultural University, Tai’an 271018, China
*
Author to whom correspondence should be addressed.
Academic Editor: Anastasios Doulamis
Sensors 2021, 21(23), 7842; https://doi.org/10.3390/s21237842
Received: 11 October 2021 / Revised: 22 November 2021 / Accepted: 23 November 2021 / Published: 25 November 2021
(This article belongs to the Section Electronic Sensors)
Since the mature green tomatoes have color similar to branches and leaves, some are shaded by branches and leaves, and overlapped by other tomatoes, the accurate detection and location of these tomatoes is rather difficult. This paper proposes to use the Mask R-CNN algorithm for the detection and segmentation of mature green tomatoes. A mobile robot is designed to collect images round-the-clock and with different conditions in the whole greenhouse, thus, to make sure the captured dataset are not only objects with the interest of users. After the training process, RestNet50-FPN is selected as the backbone network. Then, the feature map is trained through the region proposal network to generate the region of interest (ROI), and the ROIAlign bilinear interpolation is used to calculate the target region, such that the corresponding region in the feature map is pooled to a fixed size based on the position coordinates of the preselection box. Finally, the detection and segmentation of mature green tomatoes is realized by the parallel actions of ROI target categories, bounding box regression and mask. When the Intersection over Union is equal to 0.5, the performance of the trained model is the best. The experimental results show that the F1-Score of bounding box and mask region all achieve 92.0%. The image acquisition processes are fully unobservable, without any user preselection, which are a highly heterogenic mix, the selected Mask R-CNN algorithm could also accurately detect mature green tomatoes. The performance of this proposed model in a real greenhouse harvesting environment is also evaluated, thus facilitating the direct application in a tomato harvesting robot. View Full-Text
Keywords: Mask R-CNN; detection and segmentation; mature green tomato; mobile robot Mask R-CNN; detection and segmentation; mature green tomato; mobile robot
Show Figures

Figure 1

MDPI and ACS Style

Zu, L.; Zhao, Y.; Liu, J.; Su, F.; Zhang, Y.; Liu, P. Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach. Sensors 2021, 21, 7842. https://doi.org/10.3390/s21237842

AMA Style

Zu L, Zhao Y, Liu J, Su F, Zhang Y, Liu P. Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach. Sensors. 2021; 21(23):7842. https://doi.org/10.3390/s21237842

Chicago/Turabian Style

Zu, Linlu, Yanping Zhao, Jiuqin Liu, Fei Su, Yan Zhang, and Pingzeng Liu. 2021. "Detection and Segmentation of Mature Green Tomatoes Based on Mask R-CNN with Automatic Image Acquisition Approach" Sensors 21, no. 23: 7842. https://doi.org/10.3390/s21237842

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop