Determining the Precise Work Area of Agriculture Machinery Using Internet of Things and Artificial Intelligence

Abstract

Precisely measuring the work area of agriculture farm machinery is important for performing the authentication of machinery usage, better allocation of resources, measuring the effect of machinery usage on the yield, usage billing and driver’s behaviour. The manual measurement, which is a common practice is an error-prone and time-consuming process. The irregular fields make it even more difficult to calculate the work area. An automatic solution that uses smart technology and algorithms to precisely calculate the work area is crucial for the advancement of agriculture. In this work, we have developed a smart system that utilizes the Internet of Things (IoT), Global Positioning System (GPS) and Artificial Intelligence (AI) that records the movement of agriculture machinery and use it to measure the precise work area of its usage. The system couples the nearest neighbourhood algorithms with Contact-based mechanisms to find the precise work area for different shaped fields and activities. The system was able to record the movement of machinery and calculate its work area, regardless of how many times the machinery runs through a particular field. Our evaluation shows that the system was able to precisely find the work boundaries and calculate the area with a maximum of 9% error for irregular shapes.

1. Introduction

As the world is progressing towards the fourth Industrial Revolution [1], the use of state-of-the-art technologies including Internet of Things (IoT) [2], Artificial Intelligence (AI) [3] and Cloud Computing [4] are becoming mainstream. The Agriculture yearly contributes a major chunk to the world economy [5]. Modernizing agriculture can drastically increase the production of farms and can cause overall growth in the world economy. Agriculture machinery is used to plant, cultivate and harvest the crops.

By measuring the work area of agriculture machinery, we can verify the machinery usage and estimate the cost of operation. Conventionally, a measurement tape is used to manually measure the area of farmland. This is a time-consuming process and very inefficient. The inefficiency increases even more when we try to measure the area of a field that is irregular in shape [6].

Typically, people have been using Global Positioning System (GPS) in navigation services and tracking [7]. GPS is a satellite-based navigation system, which uses the geographic coordinates system to give us the accurate location of a GPS module. The GPS coordinates include the latitudes and longitudes. The GPS can also be applied towards the agriculture domain to find the work area of agriculture machinery. Automatically finding the work is interesting, but poses several challenges. First, the machinery is often used on the fields that are very irregular in shape. Second, the field may have been covered sparsely. Third, the machinery could be used around the perimeters only to trick the system. Finally, the GPS data itself could have outliers.

To overcome these issues and precisely calculate the machinery work area, we have developed a mechanism in this paper. We use a smart IoT device to record the movement of agricultural machinery. After filtering the data for outliers, we apply different convent-hull and concave-hull Artificial Intelligence (AI) schemes on the data. The AI algorithms detect the boundary points of work area. On the boundary points, algorithms are applied to calculate the precise work area of agriculture machinery.

The AI algorithms are good at calculating area of regular and irregular shapes when the work is done uniformly across the field. However, its accuracy reduces when the field is not uniformly covered or the machinery is used around the perimeters only. To overcome this issue, we use the AI in combination with a Contact-based mechanism. The Contact-based scheme uses the distance covered by machinery and width of the machinery to calculate the area, thus helping detect anomalies in the field work.

The rest of the paper is structured as follows. Section 2 describes the related work in the agriculture domain that utilizes IoT and other technologies to boost agriculture production. Section 3 describes the use of GPS and AI in agriculture. Section 4 discusses the IoT-cloud platform for data collection, storage and processing. Section 5 formulates a methodology to detect the boundary of the work area of machinery. Determining the boundary leads to the shape of the field and work area estimation. Section 6 outlines the methods used for area calculation. Section 7 shows the accuracy and performance evaluations of the schemes for different field shapes. Concluding remarks and future work is presented in Section 8.

2. Related Work

The Internet of Things (IoT) and data analytics (DA) are playing a big role in increasing the operational efficiency and productivity of agriculture [8,9,10]. Zhao, J.-c., et al. have mentioned several sectors in agriculture that can be monitored with IoT [11]. Elijah, O., et al. has found that for making agricultural decisions, important factors to be known are the shape and work area of farmland [12]. Since the agricultural fields are not always of regular shape, most of the agricultural fields are polygon or irregular structure. By knowing the coordinates of the vertices of a complex polygon, its area can be calculated. In a research article, Li, L. explained that cloud-based Global Positioning System (GPS) can be used to track the location and other information [13].

The field size, shape and working speed can be calculated and reported with the help of AI algorithms. Three main methods are used in the context of agricultural statistics: respondent self-reported land area, compass and rope. Carletto, C., et al. mentioned that conventionally, a measurement tape is used to manually measure the area of a farmland [14]. Ning, X., et al. developed a farmland measuring device for the measurement of the arbitrary shape of farmland area [15]. Evaluation of testing shows the average relative error of measurement remains within 5%. However, it is found to be a time-consuming process and inefficient.

The inefficiency increases even more when we try to measure the area of a field that is irregular in shape. The shape of the field has a great impact on the calculation of the machinery work area. For example, the area calculation of simple rectangular field is easy compared to a very irregular shaped field. Due to frequent outliers in GPS data that could make the machinery work area calculation very inaccurate. Kudo, M., et al. concluded that for the shape extraction framework, in general, most of the convex points exist in the boundary; therefore it was proposed to detect the outliers on alpha shape basis [16]. Park, J.-S., et al. mentioned that Euclidean space is one of the central concepts in computational geometry [17].

Yahya, Z., et al., research shows that convex and concave hulls are useful concepts in area calculations [18]. By considering the results of both concave hull performs better than the convex hull, and to formulate it few algorithms were suggested. Moreira, A., et al. concluded that concave hull based algorithms; Delaunay Triangulation and K-Nearest Neighbour are the closest methods to be reconnecting the vertices for geometric shape reconstruction [19]. The Delaunay Triangulation works same as the convex hull in the case of 3-dimensional, and it does not retain digging points within Delaunay Triangulation. Abdullayeva, A., et al. proposed a k-nearest neighbours approach, where the value of k is the only algorithm parameter used to control the “smoothness” of the final solution [20].

3. GPS and AI in Agriculture

GPS is a satellite-based navigation system, which uses a geographic coordinate system to give us the accurate location of a GPS module [6]. The geographic coordinates system consists of the longitudes and the latitudes. Every location on Earth can be specified using a set of numbers. The longitudes specify the horizontal position (east-west) and the latitudes specify the vertical position (north-south). Both combined pinpoint the exact location on the Earth. GPS is utilized to accurately measure the location of agriculture machinery working in the field. With the GPS coordinates at every instant of work done, we can keep track of the movement of the agriculture machinery. The most feasible way to record the GPS data these days is using smart IoT devices including modern smartphones. Modern smartphones contain a built-in GPS module that could be utilized for data collection purpose.

AI has some very convenient use-cases that are revolutionizing the world [3]. It became a very powerful tool, which can be helpful to companies standout in the competition with their rivals, accomplishing tasks with ease and speed. The use of AI techniques surely reduces the chances of making errors as well as a great help in making critical decisions [21]. AI is making machines smart enough that they could mimic the cognitive functions of the human brain which includes decision making, learning and problem-solving.

In agriculture, besides seeds, fertilizers and water, agricultural productivity is also dependent on on-farm labour productivity, which in turn is dependent on whether the farmers have access to tools, equipment and machinery to carry out farm operations efficiently. Mechanization, therefore, plays an important part in improving agriculture productivity. Farm mechanization has an enormous potential to leverage the efforts of farmers and boost crop productivity. By finding the precise work area of agriculture machinery, we can efficiently allocate our resources to increase productivity even further.

In this paper, we discuss a use case where we detect the boundary points of the GPS data that we collect to record the movements of agriculture machinery. By getting the boundary points, we can determine the work area of machinery using AI algorithms.

4. Data Collection and Storage

The system that we have developed consists of an IoT module, a smartphone application and a cloud module as shown in Figure 1. The cloud module consists of event-handlers, data storage units and AI modules. Details of each of the components are given below.

4.1. IoT Module and Mobile Application

The IoT module consists of a smartphone and a GPS. The GPS helps record the movement of the machinery in the field. A mobile application has been developed to record GPS data. The mobile application gives control to the machinery driver to start and stop the recording of the data. While carrying out the study the smartphone is kept with the driver to record the machinery movement. Initially, the data is stored inside the smartphones cache memory and as soon as the smartphone is connected to the internet, the data are sent over to the cloud platform.

4.2. Cloud Data Store

The data from IoT module are stored on the cloud for further processing. The cloud has two storages: a simple storage and relational database [22]. The simple storage stores the data-points and the relational records the output of AI algorithms [23].

4.3. AI Module

The AI module, which is hosted on the cloud, consists of several algorithms. The algorithms run on the GPS data collected using the IoT module. The AI algorithms mainly perform two tasks: boundary detection and area calculation. More details of each algorithm are given in Section 5.

5. Field Boundary Detection

The boundary of work area is calculated from the collected data. The GPS coordinates from machinery usage may include all the points showing agriculture machinery movement. To get the field shape, we need to identify the boundary points only. The work area field could be regular or irregular in shape. The boundary detection for the regular shaped field is easy to calculate, while the irregularly shaped field is relatively difficult. After studying several methods, including Delaunay Triangulation and concave hull, we were able to find the best algorithm for boundary detection. An overview of each algorithm is presented here. The detailed comparison of algorithms is presented in evaluation (Section 7) of the paper.

5.1. Convex Hull with Delaunay Triangulation

Delaunay Triangulation is a geometric technique, that could be used to determine the convex hull of a set of points [24]. It performs well for regularly shaped fields, but its accuracy suffers when it comes to the irregularly shaped field.

5.2. Concave Hull Using KNN

The K-Nearest Neighbour (KNN) Algorithm is a machine learning technique used for classification and regression. We use KNN to determine the concave hull of the data points [19]. This gives us the boundary points of the field. This is the best performing algorithm as it works for both the regular and irregular shaped fields. The problem with this method is that it is computationally complex for larger data sets. Therefore, a solution for it was thought out to achieve the boundary points at a faster rate.

The Fast Library for Approximate Nearest Neighbours (FLANN) is a library that contains a collection of algorithms optimized for fast nearest neighbour search in large datasets [25]. The concave hull using KNN with FLANN was the best solution to detect the boundary of the GPS dataset, small or large in size. The comparison of concave hull using KNN with FLANN and concave hull using KNN without FLANN is presented in the evaluation section.

6. Field Area Calculation

In this section, we discuss the work area calculation algorithms. We discuss the Delaunay, KNN and Contact-based algorithms along with its merits and demerits. A comparison of these algorithms is shown in the Evaluation section.

6.1. Delaunay Triangulation

The Delaunay Triangulation shown in Algorithm 1 uses a convex-based mechanism to find the boundary points of the work activity [24]. The area is then calculated based on the GPS points. Since the algorithm uses a convex hull, it may include the region where activity was not performed. As a result, the calculated area will be large as compared to the actual activity. The algorithm works well for regular shapes and the activities where dense activity was performed but fails to calculate the area of irregular shapes.

Algorithm 1: Boundary Detection using Delaunay Triangulation

6.2. Concave Hull Using KNN without FLANN

The KNN mechanism shown in Algorithm 2 uses a concave hull for boundary points detection and is good at calculating the area of fields of different shapes [20]. The algorithm works well for different shapes; however, the calculations can be inaccurate if the field activity was performed around the perimeter or certain points in the field. More points help the algorithm in finding a better approximation of the area, however, as the number of points increases the execution time of algorithm increases exponentially.

Algorithm 2: Boundary Detection using concave hull using KNN

6.3. Concave Hull Using KNN with FLANN

Since the KNN algorithm suffers from larger execution times, it is combined with a FLANN mechanism [25]. The FLANN removes the unwanted points from the calculation of neighbourhood detection, which results in lower execution times. However, as with KNN, this algorithm suffers from incorrect results if the activity was performed around the perimeter.

Contact-Based

To overcome the shortcoming of Delaunay and KNN, we have developed a Contact-based mechanism. The Contact-based mechanism shown in Algorithm 3 uses the GPS latitudes and longitudes to calculate the distance travelled by the machine during the activity. The area is calculated by multiplying the distance travelled with the width of the machinery. This can help us find the precise amount of work performed and will also tell us about the driver’s behaviour. The Contact-based method will show a very different area calculation if the driver skips parts of the field.

Algorithm 3: Distance Based Mechanism

7. Evaluation

In this section, we compare the different boundary detection and area calculation algorithms. The algorithms are mainly compared in terms of shape detection, area calculation and execution performance.

7.1. Experimental Setup

The results were collected on different farm machinery (Cultivator, Rotavator and Laser-leveler) in five different regions of Punjab (Bahawalpur, Layyah, Okara, Mandi Bahauddin and Lahore) of Pakistan. The field shapes were regular and irregular ranging in area from 0.28 to 4.8 hectares. All the studies were performed in dry conditions. The IoT modules were mounted on the Farm Implements collecting 10 samples per second of vibration, gyro, humidity, pressure and light sensors. The sensors data was sent to the android smartphone (Samsung Galaxy running android version 8 or higher) using Bluetooth. The smartphone was used in screen-on mode. The smartphone app accumulates the sensors data and add GPS latitudes and longitudes to it. The accumulated data from the android app was sent to the AWS (Amazon Web Services) cloud using Wi-Fi, where the AI algorithms were applied to them. The AI algorithms were run on a t2.micro EC2 Linux instance (Elastic Computer Cloud) of AWS. EC2 is a compute virtual server and AWS is the cloud web service of Amazon. The outcome of AI modules was stored in RDS (relational data store) and were forwarded to android app as well.

7.2. Comparison of Boundary Detection

The algorithms are compared in terms of detecting the boundary of the work area. Figure 2 and Figure 3 shows a comparison of concave hull (that uses KNN) with convex hull (that uses Delaunay Triangulation). It could be observed that both algorithms performed well for a regular shaped work area.

For irregular shape, the concave hull algorithms (KNN) have performed well determining the boundary points. While Delaunay Triangulation fails to identify the edges in detail as shown in Figure 4. Delaunay Triangulation has completely failed to identify the cave like shape, while concave hull using KNN performs well.

In case of the very irregular field shown in Figure 5, Delaunay Triangulation again failed but concave hull using KNN was able to detect most of the shape. It is also important to note the shape formed by concave is also very close to the actual shape.

The Delaunay Triangulation is good only for regular shaped fields, while concave hull using KNN performs well, but as the irregularity increases its accuracy is also affected. Thus, the irregularity of shape has a direct effect on the accuracy of boundary detection.

7.3. Comparison of Area Estimation

The KNN and Delaunay schemes are compared in terms of area estimation.

Table 1. Regular Shape 1.

Serial Number		Area (m2)	Execution Time (sec)	Area Estimation Error
1	Actual Area	1240	0	0
2	Delaunay	1324.47	0.0008	6.8
3	KNN without FLANN	1239.5	0.114	0.04
4	KNN with FLANN	1239.75	0.0013	0.02

and

Table 2. Regular Shape 2.

Serial Number		Area (m2)	Execution Time (sec)	Area Estimation Error
1	Actual Area	1470	0	0
2	Delaunay	1622.5	0.0014	10.34
3	KNN without FLANN	1500.2	4.216	2.04
4	KNN with FLANN	1500.75	0.0287	2.09

show that KNN with and without FLANN show good performance when calculating the area of regular shapes. The error is less than 3% for all the regular shapes. The Delaunay scheme shows slightly larger errors, in the range of 6–11%.

For irregular shapes, shown in

Table 3. Irregular Shape 1.

Serial Number		Area (m2)	Execution Time (sec)	Area Estimation Error
1	Actual Area	660	0	0
2	Delaunay	752.69	0.00144	13.9
3	KNN without FLANN	622.3	0.998	5.75
4	KNN with FLANN	683.75	0.0104	3.5

and

Table 4. Irregular Shape 2.

Number		Area (m2)	Execution Time (sec)	Area Estimation Error
1	Actual Area	1500	0	0
2	Delaunay	1970.5	0.00104	31
3	KNN without FLANN	1619.5	0.389	7.9
4	KNN with FLANN	1623	0.0134	8.2

, the KNN without FLANN shows a maximum of 8% error. The KNN with FLANN shows a slightly larger error (up to 10%). The increase in error is due to the fact that FLANN removes some points which results in a lower granularity.

Table 4 also shows that for irregular shapes, the error of Delaunay increases up to 31%. The increase in error is largely due to the fact that the convex mechanism is used for approximation.

7.4. Comparison of Performance

Concave hull using KNN algorithm shows the best performance when detecting the boundaries and calculating the work area. However, it suffers from larger execution times as number of GPS points are increased as shown in

Table 5. Execution Time comparison; concave hull using K-Nearest Neighbours (KNN) without Fast Library for Approximate Nearest Neighbours (FLANN) vs. concave hull using KNN with FLANN.

Serial Number	Number of Data Points to Be Processed	Concave Hull Using KNN without FLANN (sec)	Concave Hull Using KNN with FLANN (sec)
1	100	0.153	0.003
2	500	0.932	0.028
3	1000	1.204	0.041
4	1500	1.246	0.053
5	2000	3.795	0.067
6	2500	2.754	0.093
7	3000	1.429	0.112
8	3500	2.792	0.135
9	4000	7.772	0.092
10	4500	6.272	0.1
11	5000	6.128	0.148

. To overcome, this, we are using KNN with FLANN. FLANN using neighbourhood approximation and reduces the number of comparisons. As a result, the execution times of FLANN-based KNN is may folds less than the KNN mechanism, as elaborated in Table 5.

7.5. Number of Field Runs

Sometimes, we are interested in knowing not only the area of field activity but also in the number of times the field runs were performed. The KNN and Contact-based algorithms can be combined to find the number of field-runs performed. The KNN will find the area covered, and the Contact-based algorithm will find the amount of cumulative work. Combining both (Contact Area/KNN Area) will give us the number of runs. This information can be useful in field outcomes and billing. The machinery usage time and number of field runs represent as with the equations,

$$Usage\_time=Endtime-Starttime$$

(1)

$$Numberoffieldruns=\frac{distancebasedarea}{area\_concave\_hull}$$

(2)

7.6. Authentication of Work

An important use of the algorithms developed in this paper is to authenticate the work performed by the agriculture machinery. To perform the authentication, the expected work area (manually done) is compared against the estimated work area (calculated by the algorithms) as shown in Algorithm 4. If the error is less than the threshold (10% in this case), the work is authenticated as being done; otherwise it is not as shown in Algorithm 4. The area was calculated in hectares.

Algorithm 4: Work Authentication

8. Conclusions and Future Work

The irregularity of the field, driver’s behaviour and number of field-runs by machinery has a direct impact on the work area calculation. By getting the machinery work area we can perform the authentication of machinery usage, allocate resources efficiently, calculate the yield per unit area, measure the cost of machinery operation and study the driver’s behaviour. The GPS and AI are good at finding the precise work area of agriculture farm machinery. The convex shape algorithms like Delaunay were good at finding areas of regular shapes but gives inaccurate numbers when dealing with irregular shapes.

The concave hull algorithms like KNN was good in detection of both regular and irregular shapes, however it was computationally expensive for larger data sets. KNN when combined with FLANN showed good accuracy and was computationally cheaper compared to other algorithms. The KNN is a very good solution, but it could show incorrect results when the study is performed around the perimeter only. The functionality of KNN was further extended to irregular and sparse work using Contact-based algorithm. The combination also helped find the number of field-runs.

The Contact-based area calculation uses a distance mechanism and was helpful finding the area and number of field-runs. The mechanism expects that the field was covered in good proportion in all regions. If the work was performed around the boundaries (as is the case in Figure 2 and Figure 4), the Contact-based mechanism will give incorrect results. Therefore, KNN, Delaunay, KNN with FLANN and Contact-based mechanism are useful only when used in conjunction.

As part of future work, the work can be extended to show a relationship between number of GPS samples of data points and precision of work area calculation. The boundary detection for a small number of points is difficult to achieve, a technique could be derived to deal with this problem. We can integrate the system in this paper, with a billing system to estimate the cost of machinery operation using the calculated work area. By measuring the cost of machinery and the net yield produced, we can effectively allocate the resources to increase the production of farms. Studying the driver behaviour, we can highlight the practices that cause low yield and effectively introduce the best agriculture machinery operation practices.