1. Introduction
Due to the shortage and aging of rural labor in recent years, agricultural production must rely on mechanization to improve production efficiency, but in order to improve labor productivity and to reduce production costs, automation and technicalization are the inevitable trend of agricultural development after agricultural mechanization [
1]. The world is moving towards the fourth industrial revolution, and the latest technologies of IoT and AI, and Cloud Computing are becoming the mainstream [
2]. Image processing plays a very important role in industrial production, which can visualize the anatomical structure of the product, can check and judge the advantages and disadvantages of the product in real-time, and reduce unnecessary losses to a certain extent [
3]. Therefore, it is necessary for operators to avoid errors easily arising from the traditional artificial judgment method and to accelerate the harvest of agricultural products. With in-depth learning and the rapid development of image recognition technology, how can innovative high-tech agriculture be developed from traditional agriculture through computer technology and science and technology to solve thorny problems such as shortage of labor and R&D from the source, and improve the productivity and international competitiveness of agricultural fruits [
1,
4]. Advanced robotics, AI, and machine learning technology enable providers to offer their services with greater productivity, efficacy, and efficiency [
5]. Service robots and AI promise to increase productivity and reduce costs, prompting substantial growth in sales of service robots [
6]. At present, developed countries in the world are affected by an aging population and declining birth rate, which leads to the continuous decline of the working population and a serious shortage of labor productivity. How to solve the gap of labor demand is the goal that various countries are eager to solve at present [
7].
The rise of robotics has been widespread in all industries. High-tech robots, coupled with AI and machine learning, can provide higher productivity and product quality and constant increase of service efficiency so that customer satisfaction is improved [
6]. A survey of business leaders in the United States shows that 24% of companies are already using AI, and 60% expect to use AI before 2022 [
8]. Sales of service robots continue to grow at a rate of more than 30% per year. The International Federation of Robotics [
9] predicted that in the next decade, the range of professional and personal use of service robots would be further expanded and introduced into various industries for reform to help solve production/service problems.
AI pertains specifically to ‘machines that exhibit aspects of human intelligence’ [
10]. Focused on the technological capacity to perform tasks, rather than physical skills, AI initially was envisioned as a way to combine perception, reasoning, and improvement. Over time though, AI development has focused more on algorithms, while robotics has addressed mechanical functioning [
11,
12]. From a service management perspective, the value of AI stems not from its virtual or unrecognized use but rather on the technology’s ability to engage with customers at a social level [
13]. With the rapid development of AI technology, intelligent robots have been widely used in various fields, such as agricultural management and water resources management [
2,
14,
15,
16]. In the past, machine operators were isolated from machinery for safety reasons, but Industry 4.0 is designed to support future factories and reduce the tiredness of operators. Robots are integrated into assembly lines to work with humans to produce high-quality products and complete tasks so as to improve efficiency and satisfaction [
17,
18].
With the rapid development of the machine learning industry in recent years, deep learning is more obvious. The application fields of deep learning are very wide, such as bio-medicine, machinery, and information. Traditional condition-oriented design is replaced through deep learning. Among which image recognition can improve the accuracy of recognition and computing efficiency through deep learning, which is an indispensable technology in the automatic drive industry. In order to improve system efficiency and recognition of the deep neural network model, object cutting and object tracking technology will be developed. Object cutting technology is a condition-oriented and basic image processing technology, it can quickly cut the large range of images that may contain objects under detection to increase efficiency [
19]. Image recognition technology has been continuously improved and has been successfully applied in many industries, such as digital rights management (DRM) [
20], robotic arm path planning [
21], studies on the determination of fruit ripeness [
1,
4], lighting control technology [
22], and diabetic retinopathy diagnosis [
23,
24,
25,
26]. With the investment of related resources and the evolution of technology, more innovative applications will be developed in the future.
Husqvarna of Sweden, irobot of the United States, and HONDA of Japan have the highest market shares or patent numbers of robot lawn mowers in the world [
27], each of which has its advantages. In Taiwan, most farmers use traditional hand-held lawn mowers, which are convenient to use and operate in the lawn mowing area, but have many disadvantages, including (1) Heavy engine and heavy body burden; (2) Gasoline-powered, exhaust gas causes harm to the environment, and (3) The noise produced harms human health. As a result, more and more farmers buy electric hand-held lawn mowers, which retain the advantages of hand-held lawn mowers, improve exhaust gas and noise problems, and are lighter in weight. The other kind is a driving lawn mower, which is suitable for operation in large areas of lawn and operated by users riding on the mower. Compared with the traditional mower, it is much easier to use. However, as the blade is installed under the mower, it cannot mow the grass clean when it meets the boundary or corner. Based on the above disadvantages, combined with the advantages of traditional mowers and driving mowers, and by adding an automation concept, image recognition, and remote control technology, a personified automatic lawn mower with image recognition function is developed, and it is hoped that the labor of farmers in mowing can be reduced and that it can contribute to the development of agricultural automation.
The purpose of this research is to develop a lawn mower robot in a way that reduces costs to help farmers solve the problem of cutting grass. The design, manufacture, and development are all concerns in the cost for producing and supplying the product to be used by farmers. Therefore, this case study is an empirical discussion for solving the problems faced by farmers. The market for lawn mower robots is very plentiful with their functions often not coming cheaply and the operations sometimes dangerous. This case study expects to build a safe, labor-saving, low-cost robot for farmers that will solve the lack of labor and be easy to use. As a result, it will not have an advanced academic theory nor a complex structure.
Agriculture is very important and indispensable to human beings, people must rely on agriculture to produce food, so there is a need for continuous supply and production to meet the needs of the growing population [
2]. This study has designed a set of advanced image recognition technology to complete remote monitoring. The lawn mower can achieve the effect of personification by means of capturing the pictures shot by video cameras. Then, data are sent back to the system for subsequent processing, which is pioneering in Productivity 4.0 Sci-Tech Agriculture, and a contribution of this study. This study will replace human resources, save time and develop high-tech agriculture, develop a combination of lawn mower and crawler tracks, and introduce an automatic control system for mowing, then use Raspberry Pi as image recognition to simulate human eyes, avoid obstacles, and input the length of both sides of a rectangular area to achieve automatic mowing. This can reduce the demand for lawn mowing manpower.
4. Program Development and Process Introduction
The program development of the robotic lawn mower was designed by using CCS to write C language, a TMS320F2808 chip communicated with the Raspberry Pi through the RS-232 communication protocol, and a human-machine interface was added at the Raspberry Pi end. When the length and width of the rectangular range to be mowed were inputted, the Raspberry Pi transmitted the packet to the TMS320F2808 chip, corresponding actions were implemented through the peripheral interrupt instruction and custom flags, with each system process being introduced as follows one by one.
4.1. Process Architecture of DSP System Program
In order to realize the functions required by the user, each function of the DSP chip had a corresponding module and peripheral program, such as the GPIO module and EPWM module used in this study. Since each peripheral program will make one or more peripheral interrupt instruction(s) when a triggering event occurs, the CPU cannot process too many external interrupts in real-time, therefore, the Peripheral Interrupt Expansion (PIE) module was required.
The multiplexing interrupts diagram of the PIE module is shown in
Figure 9. The PIE module of the TMS320F2808 chip could support up to 96 peripheral interrupt instructions, which could be divided into 8 groups (INTx.1~INTx.8), then further divided into 12 sources (INT1~INT12), and Global was set overall to enable the switch of the peripheral interrupt instruction. When the peripheral interrupt flag was enabled, the interrupt service program was called, after completion of the operation, the program counter was reset, Source and Group flags were eliminated, and acknowledgment was made, when an event was triggered, the program would run again.
When the program started to run, the system would eliminate the Source and Group interrupt flags in all PIE modules and vector tables to avoid misoperation. Then make the input and output arrangement of the GPIO, after the completion, make Global Enable enter the main program. The main program includes the background program for processing non-real-time events and the interrupt service program for processing real-time events. After the completion of the main program implementation, the reset will enable again.
4.2. Control Process of Crawler Tracks
When the TMS320F2808 chip received the packet sent from the Raspberry Pi end, the data in the packet was processed; the distance inputted was converted to the number of moving steps, then the crawler tracks move. To make a comparison of the number of steps to be moved and the feedback signal of the proximity switch through the subprogram of the proximity switch, when the two were the same, the crawler tracks immediately stopped, which indicated that the action had been completed, and it would continue to wait for the next packet. The flow chart of the moving subprogram of the crawler tracks is shown in
Figure 10.
4.3. Control Process of the Mowing System
The main body of the mowing system was the mowing device. The optocoupler switch was used to receive the signal and the DSP TMS320F2808 chip was sent back to control the negative rotation of the motor. When the DC motor of the auxiliary steering started, the motor executed the right rotation, and when the left optocoupler switch was interrupted, the motor executed the left rotation. Similarly, when the right optocoupler switch was interrupted, the motor executed the right rotation until the instruction was completed. The reserve protection process was designed so that when the optocoupler switch does not send back the signal within six seconds, it was considered a hardware failure to stop the blade and steering motor. The mowing system flow chart is shown in
Figure 11.
4.4. Communication Process
The communication protocol adopted in this study was RS-232. In the RS-232 standard, characters were transmitted one by one through a sequence of bit strings. The advantages were fewer transmission lines, simple wiring, and relatively long transmission distance, which was suitable for data transmission between the TMS320F2808 chip and the Raspberry Pi. The system first initialized and entered the waiting line, the TMS320F2808 chip then sends the initial code (0xFC), determines whether the Raspberry Pi was connected successfully, and if the Raspberry Pi sends back the same value, it meant that the line connection was successful, re-sending a receiving code (0xFE) meant that it allowed receipt of the packet. The Raspberry Pi sends a packet, the TMS320F2808 chip would have a packet with the size of six entering the stage of receiving, receiving the start code (0xFF) meant that the packet was placed in the first position of the array at the beginning of the packet. The fifth code of the packet was the addition of the 2nd, 3rd, and 4th codes. If an error occurred, the error would be displayed, and the waiting stage was entered. If the packet had no error, the end code (0xFD) would be received. The flow chart of the RS-232 communication protocol is shown in
Figure 12.
4.5. Image Recognition Process
In this study, a webcam was used to capture images, and a Raspberry Pi was used for image recognition. If no obstacle was recognized, the packet would not be returned. If there was an obstacle, the captured image would be fuzzy processed, the picture would be converted into gray-scale, and then the frame reduction method would be used to identify the front object. If the identified object area exceeded the set value, the object in front is judged as an obstacle. The image recognition flow chart is shown in
Figure 13.
4.6. Control Process of the Automatic Robotic Lawn Mower
After the start-up of the power supply, the robotic lawn mower needs to wait for the connection between the Raspberry Pi and the human-machine interface, then the human-machine interface sends the communication packet, when the success of the communication was confirmed, modes could be chosen. Both side lengths of the rectangle were input after the blade had started for a few seconds. The crawler tracks and lawn mower operated and cooperated with the choice of the mode to complete the instruction. After the motor started, the PIR sensor module continuously detected for a warm body. If a warm body was detected within 2.4 m of the straight line, then all motors stopped action, the mode would need to be re-selected. The control process of the automatic robotic lawn mower is shown in
Figure 14.
4.7. Warm Body Sensor
The circuit diagram applied to the warm body sensor in this study is shown in
Figure 15. The module itself had automatic induction. When someone entered the induction range, a high potential was inputted; when someone left the induction range, it automatically delayed to shut down high potential and output low potential. LED could be used to tell that a human body had been detected in the front. After testing, the output high-potential voltage was about 3.3–3.6 V and the low-potential voltage was 0 V. The test waveform is shown in
Figure 16.
4.8. Human-Machine Interface Process
In this study, the human-machine interface was written by using Tkinter, an accessory program of Python, four buttons were made respectively, including Mode 1, 2, 3, and the emergency stop button. Mode 1 and Mode 2 were used to input the mowing area and to complete instructions in a bow path. Mode 1 was slow mode: The crawler tracks moved forward, and the mowing pole would swing to the left and right until the action was completed. Mode 2 was the inching mode: The crawler tracks would advance for a distance of half a blade, then stop, the mowing pole would swing once, and repeat the operation until the instruction was completed. Mode 3 was the obstacle avoidance mode: The crawler tracks moved forward, meanwhile the mowing pole would swing to the left and right. If the webcam identified obstacles on the way, it would send back the signal to the Raspberry Pi and then transmit the packet to the TMS320F2808 chip to command the crawler tracks to perform the obstacle avoidance function. In case of hardware and software failure, the emergency stop button could be used to stop the crawler tracks and the lawn mower motor. The human-machine interface displayed on the mobile phone is shown in
Figure 17.
Actual Measurement of Image Recognition
The method of the Raspberry Pi combined with the frame subtract was used to identify whether there was an obstacle in front, which was used to imitate human eyes. For example, people would avoid the obstacle in front when moving. If the object identified exceeds the set value, the object in front would be regarded as an obstacle and marked on the window as shown in
Figure 18.
4.9. Experimental Data of Robotic Lawn Mower
With other robots that use ultrasonic sensors to achieve the effect of avoiding obstacles, when the weeds on the ground are high, it is easy to regard the tall grass as an obstacle. This study used image recognition where it was less easy to misjudge this and utilized the method of frame subtract to identify whether there were obstacles ahead or not. This study designed and manufactured this robotic lawn mower from scratch. It has provided empirical study results. The “human-machine interface” and “image recognition” in this study were built on the Raspberry Pi. Therefore, this study used the more popular connection method VNC, to connect to the Raspberry Pi for remote control. Image recognition was the result of many experiments. During the study process, the situation of being unable to identify an obstacle was very rare, but image recognition is indeed the part that needs to be strengthened. Suggestions for a future study are when the light source is weak, a light source system could be installed on the car which can be turned on with the GPIO control of the Raspberry Pi to allow the webcam to identify any obstacles.
The crawler tracks in the study, could not only carry loads forward and climb the slope, but also do a 360° rotation, identify obstacles through webcam, achieve an obstacle avoidance function, combine the mowing structure, cooperate with the human-machine interface operation, and complete the remote control. The mowing blade could cut a distance of 55 cm for each swinging action. In the slow mode, an average mowing distance of 15 m could be completed per minute; in the inching mode, an average mowing area of 100 cm × 100 cm could be completed per minute, and the preferred mode could be selected based on different mowing needs. The finished product of the automatic robotic lawn mower is shown in
Figure 19.
5. Discussion
Companies like Honda and Husqvarna have launched a number of robotic lawn mowers in recent years, at prices ranging from USD 1000 to 2500. Currently, robotic lawn mowers are more popular in the European market, but sales in the United States are not ideal. Some robotic lawn mowers of Swedish Husqvarna are sold at USD 5100/unit, which can handle difficult terrain with a slope of 45° and high weather resistance. With GPS-aided navigation and being driven by Automower Connect, lawn mowers can be controlled anywhere. Robotic lawn mowers launched by iRobot of the United States can memorize the positions and automatically generate a digital map of the area where the grass needs to be mowed. They can also be programmed to automatically mow regularly and control the height required for mowing. Users of the lawn mowers launched by Honda can let go of both hands without pushing them for mowing, and they can also warn against theft for $2500/unit. Robotic lawn mowers are quite varied and have different functions. Consumers can buy suitable models according to their own needs.
When robotic lawn mowers manufactured by Qiu, et al. [
34] plan obstacle avoidance paths, ultrasonic waves and infrared rays are used, but this study found that if this method is used to avoid obstacles, misjudgment may occur due to weed interference. Therefore, this study does not use ultrasound but an image recognition function to deal with the obstacle avoidance path. Yu [
35] points out that robotic lawn mowers face six problems: (1) When the working environment is complex, the information conveyed by sensors is not accurate enough; (2) The scope of work is difficult to be covered completely, and repeated work results in energy waste and reduced efficiency; (3) The communication function between the robot and the outside world is not strong, the human-machine interaction ability needs to be improved; (4) The positioning system is not accurate enough, which easily leads to the deviation of the working range; (5) The energy system is not good enough, and the ability to work continuously is weak; (6) The ability to learn is not strong enough to avoid repeated mistakes. In this study, robotic lawn mowers are developed from an academic point of view. In consideration of the cost problem, they cannot be compared with commercial lawn mowers on the market, but their functions and mowing performance have their advantages. It is hoped that when robotic lawn mowers are developed again, they can overcome the above problems as much as possible, and machinery with full functions, at an affordable price, and with excellent performance can be manufactured to meet the needs of consumers.
6. Conclusions
In this study, the Raspberry Pi is used as the human-machine interface platform, Python is the programming language, and its auxiliary program Tkinter is used for design; because lawn mowers can be dangerous, the human body infrared sensor is installed on the high platform of the crawler tracks to avoid accidents. When the human body infrared sensor acts, the warning message is displayed immediately on the human-machine interface and the lawn mower is stopped. The human-machine interface includes the obstacle avoidance mode using image recognition, the image processing function library used is OpenCV, obstacles are filtered with averaging fuzzy and thresholding procedures, and the obstacle avoidance action is further implemented. In this study, advanced image recognition technology is used to complete remote monitoring. Pictures taken by the image lens make the lawn mower achieve the effect of personification, and data are sent back to the system for subsequent processing, which is pioneering in productivity 4.0 sci-tech agriculture and is a contribution of this study.
The study direction of this study takes agricultural automation as the goal, considering new and old type lawn mowers, and referring to domestic and foreign robotic lawn mowers, it is decided that the conception of personification is added to complete the design of the robotic lawn mower with image recognition. The design of swinging to the left and right not only improves the range of mowing but also saves a lot of time and greatly increases work efficiency. The overall cost of the robotic lawn mower is not high. Farm commercialization can be developed in view of this price advantage. In the development of farm commercialization, users can input the required range through the easily-operated human-machine interface to complete mowing operation in a large area, which brings convenience for farmers, and also creates new opportunities for agricultural automation.
Suggestions for the Future
Due to the limitation of cost and time, there are still many ideas that cannot be realized in this study. The following are summarized in view of future research directions and suggestions:
Remote control mode can be added to the human-machine interface, buttons can be used to control various directions and angles of the crawler tracks and start or stop of the mowing motor. This mode can control the moving range of the robotic lawn mower more accurately and can display the remaining electrical charge of the robotic lawn mower on the screen to tell the user whether to charge it or not.
The fixing device of the mowing device system uses 3D printing equipment. To avoid wire wear, a stronger iron sheet or aluminum material can be used instead, the position of the optocoupler sensor can be adjusted, and the baffle plate structure can be redesigned to reduce the error range.
In terms of image processing, it is hoped to strengthen the identification of obstacles, increase the accuracy of identification of obstacles and return coordinates, identify the specific size, area, and circumference of objects, and then take obstacle avoidance actions.