Autonomous Vehicles Management in Agriculture with Bluetooth Low Energy (BLE) and Passive Radio Frequency Identification (RFID) for Obstacle Avoidance

Abstract

Obstacle avoidance is a key aspect for any autonomous vehicles, and their usage in agriculture must overcome additional challenges such as handling interactions with agricultural workers and other tractors in order to avoid severe accidents. The simultaneous presence of autonomous vehicles and workers on foot definitely calls for safer designs, vehicle management systems and major developments in personal protective equipment (PPE). To cope with these present and future challenges, the “SMARTGRID” project described in this paper deploys an integrated wireless safety network infrastructure based on the integration of Bluetooth Low Energy (BLE) devices and passive radio frequency identification (RFID) tags designed to identify obstacles, workers, nearby vehicles and check if the right PPE is in use. With the aim of detecting workers at risk by scanning for passive RFID-integrated into PPE in danger areas, transmitting alerts to workers who wear them, tracking of near-misses and activating emergency stops, a deep analysis of the safety requirements of the obstacle detection system is shown in this study. Test programs have also been carried out on an experimental farm with detection ranging from 8 to 12 meters, proving that the system might represent a good solution for collision avoidance between autonomous vehicles and workers on foot.

1. Introduction

Agriculture and forestry are hazardous work sectors due to the high-use of machinery and the need to perform high-risk operations with dedicated equipment, which carry their own specific hazards [1]. Smallholding farmers often work in solitary conditions and, therefore, face additional issues when accidents happen and rescue is needed; on the other hand, mechanization is vital for larger farms and the dangers derived from poor interaction between workers on foot and vehicles eventually arise. Difficulty in handling protection devices can also be a major concern [2] as it adds to an existing negative attitude towards safety [3]. Furthermore, the coming of digital agriculture has highlighted how such hazards need to be mitigated or eliminated, given the increasing usage of remotely controlled vehicles and automated equipment; previous research has, in fact, already shown that safety plays a key role even at the design stage [4] and its management needs to be embedded with quality in every agricultural practice [5]. The first attempts to develop a remote-control system for an agricultural tractor appeared in the 1980s; they were based on radio-frequency modulation and hardware components such as solenoid valves operated by a controller, but despite such technical limitations, the system already included safety and emergency procedures [6]. Another approach which exploits the benefits of using wireless communication in agriculture can be found in attempts to sense any kind of machine anomalies and intervene before the occurrence of an accident, using low-cost electronic devices to integrate them even in small equipment [7]. Satellites can also provide great help in locating vehicles especially if combined with Wi-Fi networks to increase their reliability [8], while the estimating of distances between workers and obstacles can also be performed by 2D laser scanners [9], exploiting this established technology from the automotive sector in a simpler context. Given the high number of fatalities involving agricultural machinery during the last decade in Italy [10,11] the lack of compliance to health and safety legislation [12] and the need to protect bystanders or children nearby, who are consistently involved in farm-related injuries [13,14], the purpose of this project is to provide the National Institute for Insurance against Accidents at Work (INAIL), which founded the research activities, a solution to reduce everyday risks related to the presence of autonomous remote-controlled vehicles, workers on foot and various obstacles that can be found in agricultural and forestry working environments. To achieve this, an integrated network made of passive radio frequency identification (RFID) tags placed on obstacles and a set of antennas placed on vehicles to scan for other vehicles and RFIDs has been studied and developed.

2. Materials and Methods

The SMARTGRID project has a multidisciplinary dimension which required wide know-how and precise task definitions. One of the goals of the project is to provide a deep analysis of the working environment and its hazards, so a special focus has been placed on the risks of run-over and impact against obstacles or between remote-controlled machines. A specific process has been followed to define the domain and part of the requirements for the wireless infrastructure. The steps consist of:

-

Bibliographic research of previous attempts to tackle safety issues in agriculture with the use of smart/wireless sensors and applications;

-

Hazard identification in some well-known agricultural processes with hierarchical task analysis (HTA) [15], a tool that has been increasingly used recently for error prediction, performance assessment and system design [16];

-

Scan for single points of failure (SPOFs), a well-known procedure in aviation safety, and analyze them by performing a failure mode and effects analysis (FMEA). This analysis method can be flexible enough to be adapted to processes, designs or services and help to eliminate sources of accidents [17,18].

2.1. State of Art

The first step focused on bibliographic research in the Scopus Database. Specific criteria were selected in order to include as many relevant papers on the subject as possible and to avoid activities that were not of interest for the project.

As a result of the previously described Scopus search in Table 1, a set of 15 papers was selected to analyze what has been attempted in previous years. Technology has obviously improved every safety system design, and this becomes clear when discussing how a new wireless network protocol played a key role for the success of every project.

Table 1. Scopus search criteria.

Criteria.	Content	Field(s)
Wireless communication protocols and infrastructures	Mobile devices, remote, wireless, BLE, Bluetooth, RFID, internet of things	Title, abstract or keyword
Selected sectors	Agriculture, tractors	Keyword
Excluded sectors: markets and traceability	Markets, traceability, supply chain, water purification, breeding, products	Title, abstract or keyword
Excluded sectors: transportation	Road safety, maritime safety, plane crash, automotive, public works, traffic control, road accidents
Excluded sectors: disasters and emergencies	Asbestos, terrorism, suicide, acid rain, forest fire, radiation
Excluded sectors: others	Medical applications, mining, spectroscopy, environmental education, occupational health, testbeds

A thorough analysis of the papers listed in Table 2 was carried out to understand the strengths and weaknesses of every solution that had been tried before. As a result, it was found that a reliable and fault-tolerant safety system has always been considered one of the core processes for all the papers, while major differences were seen between network protocols and infrastructures:

Table 2. Scopus search results for wireless and smart solutions in agricultural safety.

Title	Year	Citations
The radio-controlled tractor from Kubota [6]	1987	0
Remote controls on an agricultural tractor for performing asae/sae field upset tests [19]	2000	2
An obstacle identification algorithm for a laser range finder-based obstacle detector [20]	2005	35
Child-safety driver-assistant system and its acceptance [21]	2009	1
Method for an electronic-controlled platooning system of agricultural vehicles [22]	2009	4
Safedriving: A mobile application for tractor-rollover detection and emergency reporting [23]	2013	21
A Wii-controlled safety device for electric chainsaws [7]	2013	0
Application of the Zigbee wireless communication technology on the endless rope continuous tractor derailment monitoring system [24]	2013	0
Field tests of a tractor rollover detection and emergency notification system [25]	2015	6
Smart machines, remote sensing, precision farming, processes, mechatronic, materials and policies for safety and health aspects [26]	2018	9
Agricultural security monitoring and safety alert system: implementation of wireless video on the farmstead [27]	2019	0
Data collection from outdoor IoT 802.15.4 sensor networks using unmanned aerial systems [28]	2019	0
Developing and testing an autonomous all-terrain vehicle to experimentally test rollover incidents [29]	2019	0
Developing and testing a GPS-based steering control system for an autonomous all-terrain vehicle [30]	2020	0
Multi-camera-based person recognition system for autonomous tractors [31]	2020	0

-

The usage of mobile phone applications can be of great help in sending automatic alerts in case of accident, while Bluetooth specific devices can communicate additional info to nearby devices. This can be carried out even without internet connection, so represents an important feature of the network protocol;

-

Cameras are already largely exploited for obstacle detection and pattern recognition, but agricultural working environments could generate several maintenance issues and additional mission-critical failure modes from dust effects and vibrations;

-

ZigBee networks, since they allow every device to share information with each other without the need of communicating through an access point, can increase the feasibility of wireless networks in harsh conditions, but this also comes with a lower communication speed and an inferior number of available channels;

-

Unmanned aerial systems can play a key role in improving connectivity between machines working on large areas and in areas with a lack of good GSM mobile connection, but still require an active communication network with vehicles and workers on foot to perform safety functions;

-

System robustness proved to be, for almost all the papers listed in Table 2, a fundamental requirement to adapt the solutions to a large variety of agricultural working conditions and environments;

-

Low-energy and passive modules, in association with a reliable wireless infrastructure for longer ranges, can prove to be robust enough to enhance detection capabilities and safety alert systems. Lack of electric sources can pose a serious threat to many devices, and this explains the need for energy-saving network designs.

2.2. Hazard Identification in Agricultural Processes with Hierarchical Task Analysis (HTA)

The SMARTGRID project is focused on particular risks that are part of everyday practices in agriculture and this consequently requires hazards to be identified. The analysis was carried out accordingly for three kinds of hazards:

-

Running over and impact with obstacles;

-

Contact between workers on foot and the tractor’s equipment;

-

Contact between workers on foot and trailers or towed equipment.

Further analysis was then provided by the description of the activity’s workflows for some of the agricultural processes linking accidents to the previously identified hazards. Such work has been carried out by performing a hierarchical task analysis (HTA) oriented to understanding the dependencies and to identify which steps might play a relevant role in triggering the accidents that were found in the statistical research.

For this step of the analysis, three agricultural activities were considered in order to identify the most safety-related concerns for possible accidents. The activity selection was based on the presence of the relevant number of workers who are needed to work along with an agricultural machine in order to complete the task, and on the involvement of equipment or any kind of active machinery component that might represent an additional source of hazard for nearby workers. The identified activities which almost always include such aspects were:

-

Mechanical harvest of shell fruits, as shown in Figure 1; a considerably high number of workers on foot operate equipment from machines or self-propelled vehicles which need to be properly configured before activation, such as suction pipes. If workers skip part of the setup or happen to work too close to the machine, hazardous scenarios may arise;

Figure 1. Basic HTA for mechanical harvest of shell fruit.

-

Mechanical fertilization with manure spreaders, as shown in Figure 2; manure spreaders do not require workers on foot but still need to be properly configured in order to adjust on-board deflectors and eventually reload the machine when needed;

Figure 2. Basic HTA for mechanical fertilization with manure spreaders.

-

Mechanical pruning, for example, in vineyards, as shown in Figure 3; pruning consists of a mix of manual and mechanical operations and since pre- and post-pruning operations are necessary, vehicles need to be preceded by workers on foot and followed by others. Pruning machines are often made of bars supporting a set of rotating blades, which could remain powered by inertia for a very short period of time, even if the driver deactivated them.

Figure 3. Basic HTA for mechanical pruning in vineyards.

The results of this analysis provided a good basis for the identification of the single points of failure (SPOFs); while the term naturally belongs to analysis at design level to remove flaws which may result in severe accidents, the goal for the project was to identify any flaws in everyday practice that may lead to an accident even if the machine’s design would not normally let it happen. This can be due to a major difference between the way work has been imagined and the way work is actually carried out, but could also be due to contingencies that have not been taken into account in accident prevention analysis.

2.3. Scan for Single Points of Failure (SPOFs) with Failure Mode and Effects Analysis (FMEA)

Given the importance of the human factor in agriculture and forestry activities, a deep analysis of accident causes is paramount to any progress in designing the system itself. The domain of the analysis was, however, restricted to the most relevant activities as previously described and to the failure modes which lead to risks that the SMARTGRID project aims to reduce. In order to do so, a modified FMEA was carried out and the result of this brand-new approach allowed us to understand how a safety system should perform to overcome the SPOFs which have been found.

The FMEA analysis in Table 3 shows that the activities involving workers on foot and vehicles together at the same time are almost always affected by hazards which can lead to fatal consequences. Contingencies are, therefore, mandatory to mitigate such risks and to make sure that work procedures are carried out as intended; drivers would need additional devices to identify nearby workers and, at the same time, workers on foot should receive any warnings as soon as they get dangerously too close to vehicles.

Table 3. Failure mode and effects analysis for activities of major concern.

Activity	Requirements	Single Points of Failure	Failure Mode	Severity
Mechanical harvest of shell fruits	Equipment setup and margin definition	Workers on foot getting run over by machines	Wrong distances or wrong maneuvring	Severe injury or death
Mixed mechanical and manual harvest	Machines must keep a fixed parallel distance from workers	Physical contact with active mechanical parts of the equipment	Distance too close to active parts of the equipment	Severe injury or death
Use of manure spreaders with vertical axes rotors	Maintenance and cleaning	Activation of equipment during maintenance	Power still present on the equipment	Severe injury or death
Setup of manure spreading equipment	Need to be done before use on field	Physical contact with moving parts such as conveyor belt carpets while adjusting its speed	Requirements’ check ignored	Severe injury or death
			Requirements’ check carried out while using equipment	Severe injury or death
Mechanical pruning, blades as cutting equipment	Manual pruning to be executed, margins to be set	Physical contact with blades while performing manual activities	Missing alerts for danger zones near the equipment	Death
			Wrong maneuvering, impossibility of checking for the presence of workers	Severe injury or death
			Failure of blade supports	Injury and material losses
		Physical contact with workers, both on foot and on board, assigned to manual finishing of missed branches	Absence of barriers	Severe injury or death
			Inadequate distance from danger areas
			Increase in working speed for mechanical pruning that workers cannot cope with
Mechanical pruning, disks as cutting equipment	Manual pruning to be executed, margins to be set	Physical contact with cutting disks while performing manual activities	Absence of barriers	Severe injury or death
			Wrong maneuvering, impossibility of checking for the presence of nearby workers
			Failure of disk supports
Mechanical pruning or cutting with telescopic arms	Verify safety distances and working conditions	Side overturn	Stability check incomplete or not correctly performed	Severe injury or death
		Impact with obstacles or workers on foot	Wrong maneuvering, impossibility of checking for the presence of workers on foot

2.4. System Layout and Antennas

In order to fulfil the essential requirements that have been discussed and to achieve a reasonably high level of safety, a network infrastructure based on passive RFID tags positioned on obstacles and on PPE and Bluetooth Low Energy (BLE) devices was designed. These types of devices and networks were selected according to considerations made in previous research in paragraph 2.1; low-cost and maintenance-free batteryless devices such passive RFIDs seemed to be the most suitable for agricultural working environments and are already used for object location in their environment [32,33], while BLE is the most common network infrastructure to deliver push notifications to workers on foot even during emergencies [34,35], detect occupancy patterns [36] or even pass pre-processed data to machine learning models for further location analysis [37]. Another advantage of such systems is their reliance just on workers’ mobile phones as receiver devices being able to provide alerts or emergency notifications even when they have no GSM signal.

By merging RFID identification capabilities with BLE push notification, the SMARTGRID project achieved its goal of finding a robust wireless infrastructure for obstacle detection and safety purposes. Hence, the following elements were identified as key system components:

-

Passive RFID tags attached to individual protective equipment. To avoid reliability issues related to reading only one single tag, several tags were attached to PPE and other equipment as a contingency;

-

Every vehicle was equipped with:

○

sensors to identify other vehicles and vice-versa;

○

antennas to identify nearby workers, by scanning for RFIDs on their PPE;

○

actuators for emergency stops and signals;

○

BLE devices to deliver push notifications to workers’ mobile phones;

-

control systems to establish a connection with an on-site server and update the real-time notification.

A specific layout was designed to install such antennas on a test tractor, namely an old FIAT 55-66 which provided pre-existing supports for bolts; having many anchorage points available along the structure, no modifications were carried out on the tractor’s cab. Power for antennas was provided by a 24V-2A battery located inside the tractor; the same battery powered a Wi-Fi router for data transmission as well, also positioned in the cab. The antenna model for passive RFID identification was the Impinj beam-scanning xArray gateway; given the size of the antennas (approximately 40 cm per side square) their layout deserved additional care in order not to create an obstacle to the driver’s sight, as shown in Figure 4.

Figure 4. Three anchorages for antennas. Blue (front), black (middle) and red (rear).

Having to place only two antennas per side, one anchorage point had to be discarded. Since a minimum distance between the devices was required, only front and rear anchorages were selected for this study. The anchorages were created with off-the-shelf supports and fixed on a VESA 200 plate, as shown in Figure 5.

Figure 5. Anchorage details: (a) VESA 200 support for the xArray antenna; (b) anchorage bolts already available on tractor’s cab, plus pan-tilt plate for further adjustments.

3. Results

The integrated network infrastructure is described in Figure 6, based on the system’s layout and the system’s safety requirements.

Figure 6. SMARTGRID’s flowchart for the safety system.

The safety algorithm is intended to define three areas based on alert and danger distance thresholds, generating different responses to the workers, to the backend server which keeps track of every dangerous event and to vehicles for emergency procedures, if necessary. Distance thresholds can be defined for each activity accordingly to their risk analysis, and dedicated gates at the entrance of the working area scan for the RFIDs placed on PPE to check if they are in use; a backend server completes the system’s layout, receiving information from the machine about any near-miss events.

Preliminary tests were carried out to determine the maximum range of beaconing and the maximum detection for passive RFIDs in agricultural and forestry working environments. Such tests simulated the presence of a tractor with the required antennas which scanned for the presence of workers on foot and other machines, as shown in Figure 7.

Figure 7. Tractor performing field tests. Antennas are visible on tractor’s cab and passive RFIDs have been placed on obstacles and poles all around the area.

A pair of RFID antenna arrays with embedded RFID readers were placed on one of tractor’s cabin sides with the aim of tracking nearby RFID tags. The two RFID antenna arrays scanned the nearby area by extracting the angle of arrival (AoA) of the RFID tags through the acquisition of signal strength, available from the RSSI (received signal strength indicator) output parameter. AoA measurements were been combined to estimate the RFID tag coordinates. These processes can also be deployed in the same way for detecting the coordinates of radio-controllers from a remotely controlled vehicle, providing additional information regarding the operator’s maximum safety distance from the machine. AoA measurements are advantageous over methods that attempt to estimate the distance between antenna and tag from received power; in fact, although reflections from the ground and nearby objects are often considered by every distance estimation method, the measurements rarely match the theoretical models in complex operating environments when unpredictable terrain conditions arise. This phenomenon is shown in Figure 8, where some differences between measured and expected RSSI data can be observed through field tests, but it still remains negligible.

Figure 8. Field test data for signal RSSI over distance in meters.

From field tests, the system showed an accurate detection capability of 8–12 m, a value in line with current maximum read distances of passive RFID systems. The range could also be increased to 25 m, if required, for any long-range radio-control tracking by using semi-passive, i.e., battery-assisted, RFID tags.

Location by triangulation, on the other hand, allows the continuous tracking of the operator and obstacles in view. The system was validated through a field acquisition campaign, for which a case-test is depicted Figure 9. The actual trajectory of the tractor relative to the position of an operator equipped with 12 passive RFID tags was measured using an UWB (ultra-wideband) system and plotted in blue. The output of the RFID system is shown in red, and the RFID arrays on the test tractor are shown as green markers. A good matching between the actual and estimated paths is visible.

Figure 9. Actual trajectory relative to the tractor of an operator equipped with passive RFID tags measured with UWB (blue) and RFID system (red). RFID antenna arrays on the tractor chassis are the green markers.

4. Discussion

The system’s capability for detecting nearby obstacles in an open field confirmed the project’s expectations for maximum detection ranges. The RFIDs could have been hidden by the worker’s posture and affected whether the worker could be located; as a result, the presence of multiple RFIDs on all sides of the PPE ensured that posture seldom prevented the antennas from detecting all RFIDs and that an acceptable number of RFIDs would always be identifiable by the system. The system’s latency was also tested; it was possible to receive position updates in less than 300 milliseconds and given the speed of agricultural vehicles, the latency of the system does not affect the definition of the hazard threshold distances.

Natural obstacles did not particularly affect the system’s locating accuracy, while terrain conditions sometimes both negatively and positively affected the maximum detection range. The tractor’s structure was also tested to check whether it could generate any interference, but no metallic or other parts affected signal transmission.

The next step of the SMARTGRID project will involve final tests on the experimental farm at Tuscia University to check the system in place as a whole (antennas, BLE beaconing and back-end server for near-miss tracking) and to test more representative conditions with the presence of workers on foot. It is worth considering that similar capabilities have been achieved in previous research concerning collision avoidance for excavators on building sites [38], but that system relied on onboard cameras. Another interesting aspect may be represented by wireless devices with energy-harvesting capabilities [39], which can gather energy from vibrations and increase the detection range of SMARTGRID system.

5. Conclusions

Although wearable wireless devices have already been considered as an interesting solution for safety solutions [40,41], the usage of passive RFID tags on PPE has not been unexplored before. The system in this research has shown good potential in estimating workers’ positions near to agricultural machinery. Such results seem to be promising and confirm that wearable smart solutions and low0cost and low-energy wireless devices can greatly enhance safety in agriculture and forestry if remote controlled or autonomous vehicles are used [42] and, in addition, improve the detection of misuse or absence of PPE. It is worth mentioning that the system’s layout has been specifically developed for agricultural working environments and, as a result, it cannot be deployed as it is in a different context, such as road traffic, construction, or indoor vehicle management. The main limitation of the system was identified as being hard to deploy in those machines that can extend equipment within 2–4 meters beyond the vehicle’s perimeter, such as telescopic arms or other extendable tools; such equipment would require additional smart devices such as active RFIDs, RFID readers with integrated antennas or similar solutions.