Next Article in Journal
A Two-Step Phenotypic Parameter Measurement Strategy for Overlapped Grapes under Different Light Conditions
Next Article in Special Issue
Identification of Two Commercial Pesticides by a Nanoparticle Gas-Sensing Array
Previous Article in Journal
Study of a Current and Voltage Polarization Sensor Network
Previous Article in Special Issue
Review of Weed Detection Methods Based on Computer Vision
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Sensor Fusion with NARX Neural Network to Predict the Mass Flow in a Sugarcane Harvester

by
Jeovano de Jesus Alves de Lima
1,2,
Leonardo Felipe Maldaner
1 and
José Paulo Molin
1,*
1
Precision Agriculture Laboratory, Biosystems Engineering Department, ‘Luiz de Queiroz’ College of Agriculture, University of São Paulo, Av. Pádua Dias 11, Piracicaba 13418-900, São Paulo, Brazil
2
AGCO Corporation, Av. Guilherme Schell, 10260, Canoas 92420-000, Rio Grande do Sul, Brazil
*
Author to whom correspondence should be addressed.
Sensors 2021, 21(13), 4530; https://doi.org/10.3390/s21134530
Submission received: 16 April 2021 / Revised: 14 June 2021 / Accepted: 15 June 2021 / Published: 1 July 2021
(This article belongs to the Special Issue Smart Agriculture Sensors)

Abstract

:
Measuring the mass flow of sugarcane in real-time is essential for harvester automation and crop monitoring. Data integration from multiple sensors should be an alternative to receive more reliable, accurate, and valuable predictions than data delivered by a single sensor. In this sense, the objective was to evaluate if the fusion of different sensors installed in a sugarcane harvester improves the mass flow prediction accuracy. A harvester was experimentally instrumented, and neural network models integrated sensor data along the harvester to perform the self-calibration of these sensors and estimate the mass flow. Nonlinear autoregressive networks with exogenous input (NARX) and multiple linear regression (MLR) models were compared to predict the mass flow. The prediction with the NARX showed a significant superiority over MLR. MLR decreases the estimated mass flow variability in the harvester. NARX with multi-sensor data has an RMSE of 0.3 kg s−1, representing a MAPE of 0.7%. The fusion of sensor signals improves prediction accuracy, with higher performance than studies with approaches that used a single sensor. The mass flow approach with multiple sensors is a potential approach to replace conventional yield monitors. The system generates accurate data with high sample density within sugarcane rows.

1. Introduction

Mass flow is an essential parameter for the automation of the control processes performed by the harvester. These data support engine rotation control, harvester speed, clean system rotation, chopper blade rotation, and feeder roll speed. The automation of these systems can bring benefits such as a decrease in the damage to the billets [1], uniform billet lengths [2], quality of cleaning, and reduction of losses of the harvested product [3,4]. The mass flow predicted in the harvester provides data necessary for the localized management of sugarcane plantations. Through the mass flow predicted, it is possible to calculate the yield in a given area (e.g., ton ha−1), and together with the coordinates, yield maps can be generated. Yield maps are the key to managing sugarcane spatial variability [5,6]. This is the starting point for implementing Precision Agriculture (PA) practices, which aim to assist decision-making and interventions in high spatial definition in the agricultural fields.
Different sugarcane yield monitors have been taken to predict the sugarcane mass being harvested. The first approach to measure the mass flow for mapping sugarcane yield was performed by [7,8]. From then on, yield monitors using load cells [9], optical fiber [10], deflection plates [11], and 3D sensors [12] have already been tested on harvesters to measure the amount of sugarcane processed in real-time. However, according to [13], few are being used commercially due to their accuracy, the continual need for calibration, and the fact that most were developed for research purposes.
In a recent study, the prediction of sugarcane yield using four variables of the harvester engine shows promising results, with fewer minor prediction errors than studies with a single sensor approach [14]. In this sense, we hypothesize that the fusion of data generated by sensors installed in the recollection may be an alternative to reduce the error in mass flow prediction. Sensor fusions have the advantage of making information more significant and precise than a single sensor, and can reduce sensor errors [15]. In agriculture, the integration of the different sensor data is used to improve the autonomous navigation of mobile robots [16], and increase the accuracy of the guide-line of an autonomous robot for the weeding operation [17]. In the study by [18], data coming from heterogeneous but complementary sensors were combined to generate a multi-layer map of the environment and the supporting ground.
Among the models used for sensor data fusion, artificial neural networks (ANN) provide a new approach to the problem of parameter estimation of linear and non-linear models [19]. The learning ability of ANN allows them to adjust to changing, dynamic environments, and is a more flexible predictive tool than traditional statistical models [20,21]. The nonlinear autoregressive networks with exogenous input (NARX) differ from the other ANN in that the outputs of the model are feedback as an input for future predictions. NARX has the advantage of not requiring many different input variables, and the integration of multiple variable inputs and autoregressive inputs benefits the modeling process to provide more accurate estimates [22]. NARX was used to predict and compensate the inertial navigation system position errors when global positioning system (GPS) is unavailable [23,24], diagnose faults for pneumatic control valves [25], predict marine engine performance parameters [26], underwater passive target state estimation [27], undergo multiple data fusion for rainfall estimation [28,29], and perform long-term machine state forecasting [21].
On the other hand, there are still no reports on the use of the NARX to process signals from sugarcane harvester sensors. Thus, this study makes the first approach to investigate the use of the NARX on harvested sugarcane. The NARX can incorporate the dynamics of signals from sensors [25], such as those installed in sugarcane harvesters that monitor the harvester’s operational processes. In this way, the use of the data generated by the sensors already installed in the harvester and harvester instrumentation to measure sugarcane parameters during harvest would be an essential advancement in harvester automation and agricultural operations [30].
The use of the fusion techniques with the sensors in the sugarcane harvester to obtain a single output for the flow mass monitor can be an option to improve the accuracy of prediction. Embedded sensors in the sugarcane harvester can collect high resolution data to guide site-specific management within each sugarcane row with high resolution [30]. However, there is still no knowledge about the use of a combination of hydraulic pressure, strain gauge, and potentiometric displacement sensors on a single harvester for sugarcane mass flow prediction. Our goal was to test machine learning based on a neural network model to integrate sensors installed at various points in a sugarcane harvester. Moreover, the objective was to evaluate if the fusion of different sensors installed in a sugarcane harvester improves the mass flow prediction accuracy.

2. Materials and Methods

2.1. Mass Flow Prediction

The approach to predicting mass flow in this study is based on sensor signals as input data for models generated through machine learning (ML) (Figure 1). First, the calibration curve was created for the n sensor installed on the harvester. The model generated in the initial calibration was used to perform the first preliminary prediction (PPn) of the mass flow. Due to the characteristics of the sensors and the harvester mechanical factors, it is necessary to recalibrate the sensors over time. Thus, an ML model was used in the n sensor to perform the self-calibration, maintaining the accuracy in predicting the mass flow (PPCn) over the harvester’s operation time. Finally, the fusion of the PPCn from the n sensor was performed using a supervised model to predict the mass flow in the harvester in real-time. Data processing and model training was performed offline for later use in the harvester.

2.1.1. Sensor Calibration

The calibration curve was obtained by fitting a linear response function (Equation (1)) to a set of experimental data consisting of the sensor measured signal relative to a known ground truth [31].
y = β0 + β1 x
where y is the ground-truth flow mass value, x represents measured sensor signal, and β0, and gain β1 are the calibration coefficients. The constant β0 is the offset that arises when the measured value differs from the true value, and it can be determined by measuring the sensed value when the ground-truth value is zero. Gain β1 refers to the rate or the amount of change of the measured value to the change in the underlying ground-truth value [32].

2.1.2. Sensor Self-Calibration Approach

To ensure a good precision of a measurement made by the sensors, it is necessary to recalibrate the sensors installed in the harvester. For each sensor, the calibration curve is applied (Equation (1)), generating a preliminary prediction (PPn). After the PP for each n sensor, the mean of the values (Mppn—Equation (2)) and the standard deviation (SDppn—Equation (3)) are calculated. In this way, it is possible to calculate the prior error of each sensor. If the calculated error (Equation (4)) of sensor n is greater than 2%, the data is discarded, and not used for yield prediction. If the sensor keeps registering errors greater than 2% over 10 cycles (10 data collected by the sensor), a warning message is reported. In this way, it is possible to detect a sensor with problems and perform maintenance. In this study, several sensor configurations were tested so that if one of the sensors is defective it is possible to use a prediction model without this sensor, not affecting the yield prediction.
M p p n = 1 n i = 1 n P P n
S D p p n = 1 n i = 1 n ( P P n M p p n ) 2
E r r o r n   ( % ) = [ P P n ( M p p n   ± S D p p n ) ] × 100
where Mppn is the mean and SDppn is the standard deviation of the preliminary prediction (PPn) of the n sensor.

2.1.3. Sensor Fusion

We combine the PPn from the n sensor into a common MSf predicted in the stage. To merge the multiples PPn, machine-learning approaches were used to achieve one fused output. The MLR was used because all commercial sensors for sugarcane flow measure use a linear approach, and NARX was used due to the algorithm having good adaptation with multivariable and nonlinear data. The accuracy of predictions using data fusion methods was compared with those of individual sensors, and among these data fusion methods.

NARX Neural Network

The nonlinear autoregressive network with exogenous input (NARX) [33] was chosen due to the excellent ability of this neural network to have good adaptation with multivariable and nonlinear data [34]. A typical NARX regression neural network consists of three layers (input, hidden, and output layer), input and output delay, and the number of hidden layer neurons before application [35] (Figure 2).
The NARX neural network model is a nonlinear autoregressive model that has exogenous inputs. That is, the model relates the current PPn value of each sensor used in the model, and the value predicted by the model using previous PPn data. In addition, the model contains an “error” term that relates to the fact that knowledge of other terms will not allow the present value of mass flow to be accurately predicted. The NARX was developed using the Neural Network Toolbox of MATLAB R2018b (MathWorks, Natick, MA, USA) and was calculated as:
y(t) = f(y(t − 1), y(t − 2), …, y(tny), u(t − 1), u(t − 2),…, u(tnu))
where the next value of the dependent output signal y(t) is regressed on previous values (y) of the output signal and previous values of an independent input (u) [36]. The function f in Equation (5) was trained by the network, which depends on the number of neurons in the hidden layer. In this first study, we used only a hidden layer in the NARX. To find a set of the model’s parameters to produce the best performance, four different numbers of neurons (m = 10, 15, 20, and 25) and three order delays (d = 3, 5, and 10) were tested. The absolute error between the NARX output and the mass flow value observed was used as the performance measure. The optimal parameters for the NARX were m = 15 and d = 3. These values obtained the best performance of the model in predicting the sugarcane mass flow.

Multiple Linear Regression

The Multiple Linear Regression (MLR) is a parametric model and a supervised learning algorithm that uses a linear approach between the variables. It is the most widely used method to establish relationships between a single dependent and small set of independent variables (Equation (6)).
yf = β0 + β1xi1 + β2xi2 + … + βnxin + ε
where yf is the dependent variable (mass flow, kg s−1), xi is exploratory variables, β0 is the y-intercept (constant term), βn is the slope coefficients for each exploratory variable deduced from the training dataset, and ε is the model’s error term. The exploratory variables considered were PPCn from the n sensor.

2.2. Harvester Instrumentation

A sugarcane harvester, model Valtra BE1035 (AGCO Corporation, Mogi das Cruzes, SP, Brazil), was instrumented with three sensors with different operating principles to predict the mass flow (Figure 3). The sensors were installed in eight different mechanisms throughout the sugarcane flow in the harvester. These mechanisms carry out the processing and transport of sugarcane or are related in some way to the sugarcane flow. Previous researchers on sugarcane yield prediction have already studied the sensors used individually in this study [8,37]. Until this moment, only the scale installed at the end of the elevator can measure the mass flow as a direct response; however, the scale system presents problems such as tilt compensation, vibration, and tare change caused by mineral material, sediment, and debris. [10] use optical sensors to predict sugarcane mass flow in a harvest. Another approach was the installation of instrumentation to monitor the opening of the feeder rollers made by [38,39]. The measurement errors obtained in the reported tests are still about 100% higher than those obtained on grain mass flow used on combines and forage harvesters. In addition, there are reports of only short tests, usually with a calibration adjusted for a full over-flow load. There are no reports of continuous measurement so that the actual behavior of the sensors can be assessed. In this way, we start from the premise that the sensors and the place where they were installed have the potential to predict the mass flow in the harvester.

2.2.1. Pressure Sensor

The pressure sensor, model SCP01-400-14 (Parker Hannifin Corporation, Cleveland, OH, USA), was used to measure the hydraulic pressure in the chopper motor (CEP) and elevator conveyor motor (EPS). This sensor has a reading range from 0 to 400 bar with a response time of ≤1.0 ms. Its non-linearity and hysteresis error was specified as ±0.1% at full scale. This study assumes that the differential hydraulic pressure (bar) in the elevator hydraulic motor is proportional to the amount of sugarcane transported on the elevator conveyor. Also, we consider that differential hydraulic pressure (bar) in the chopper motor is proportional to the amount of sugarcane being chopped.

2.2.2. Strain Gauges Sensor

Unidirectional strain gauges, model P1L11-5/350 (Exel Sensores, Taboao da Serra, SP, Brazil), were used to measure the deformation of elements in the elevator. The strain gauge grid had an area of 1.0 mm2, electrical resistance of 350 Ω, and a gauge factor of 2.18. Strain gauges are commonly used to measure mechanical strain and are applied to load cells, torque, and pressure sensors. A finite element model was used to define the strain gauge installation points. Based on this model, it was possible to determine which regions in the elevator would be most sensitive to small sugarcane flow mass variations. The strain gauges were installed in four points in the elevator: left base (LBSG), right base (RBSG), right hydraulic cylinder (LCSG), and left hydraulic cylinder (RCSG). At each point, the strain gauges were configured as a full Wheatstone bridge.

2.2.3. Displacement Sensor

A potentiometric displacement sensor, model WPS-1000-MK46 (Micro Epsilon America, Raleigh, NC, USA), was used to measure the gap between two feed rollers. This sensor has a reading range from 0 mm to 350 mm. Its nonlinearity and hysteresis error was specified as ±0.1% at full scale. This study assumes that the separation of the feed rollers is proportional to the volumetric flow rate. In the feed rolls, the sugarcane bundle has already spread over the entire width of the rollers, allowing for the uniform processing of the sugarcane in the chopper. The sugarcane flow speed in the last set of feed rollers is constant, as the chopper rollers pull the bundle at a uniform rate to obtain a constant stalk length.

2.3. Data Collection

The study field is located in Santa Lucia, SP, Brazil (21°41′13.2″ S 48°05′29.3″ W). The variety is RB985879, spaced 1.5 m between the rows. The harvest was performed at an average speed of 1.5 m s−1. The data was collected using a harvester that processes only a single row of sugarcane. A data logger with analog inputs was used to connect the sensors. A script was written in LabVIEW software to control the start and end of data acquisition and records the sensor signal data in a text file. The data collection frequency of the pressure sensor was 10 Hz; strain gauge and potentiometric sensors were 400 Hz.
An external sensor was used to measure the mass flow in real-time to train the predictive models and compare the predicted values. This data was used as a dataset to train and test the NARX. Load cells in the infield wagon were installed to measure the amount of sugarcane mass dumped into the infield wagon by the harvester. The load cells in the infield wagon have an independent system for mass acquisition at an acquisition frequency of 10 Hz. Mass data was collected using a timing trigger to evaluate the input time until the mass flow output time. Sensor data and infield wagon data were collected during the harvest of 9 sugarcane rows with 600 m length, giving the data set necessary to train and compare the prediction models proposed in this study (Table 1).
To validate the best models developed, data was collected on a commercial sugarcane field of 49 ha. The instrumented harvester was used to predict the sugarcane yield and the infield wagon was instrumented with load cells to generate the observed data. The data generated by the harvester and the instrumented infield wagon were georeferenced using a Global Navigation Satellite System (GNSS).

2.4. Pre-Processing and Training Models

In the first process, null values and global outliers were excluded. Values out of the measurement scale of each sensor were excluded. The global filtering method for identifying outliers was the Interquartile Range (IQR) [40]. After filtering processes, a time offset was applied for each sensor to correct the time lag difference. Therefore, the sensor signals, located at different points in the harvester, were synchronized, by the time, with mass flow observed (measured by a load cell in the infield wagon).
The random sampling method was used to split the dataset into train and test data with a 70:30 ratio. The same ratio was used to carry out the training of each model used in this study. Nine combinations between the different sensors were evaluated (Table 2). The first combination consists of all eight points with the sensors on the harvester. One or more sensors were removed from the data set, and the new models were tested.

2.5. Analysis

The prediction error was calculated for each sampled point in the test data set, which is the difference between the predicted mass flow by the MLR and NARXs and observed mass flow. The metrics Root Mean Square Error (RMSE, Equation (7)) and Mean Absolute Percent Error (MAPE, Equation (8)) were used to evaluate and compare the models.
RMSE = 1 n i = 1 n ( y i x i ) 2
MAPE = [ 1 n i = 1 n | y i x i | x i ] × 100
where n is the number of the sample, yi is the predicted variable response, and xi is the observed variable response.

3. Results and Discussion

In the mean comparison analysis, there was no significant difference in the mean of the values predicted by the different models tested (Figure 4). However, the predictions using the NARX had a larger range of values than the predictions using MLR. That is, the MLR reduced the variability of the predicted data. This is a first indication that the NARX is each able to identify the mass flow variation within the harvester It is common for mass flow variation peaks to occur over short periods, since sugarcane has high yield variability over short distances within the row. In this way, the MLR may not be identifying these variations, which leads to a concentration of data close to the mean, as shown in the boxplot (Figure 4). The MLR with only the potentiometric displacement sensor in the feed roller (Fusion 5), and model with only a strain gauge sensor (Fusion 7), has the lower amplitude of the estimated data. On the other hand, there was no reduction in the amplitude of this data using the NARX.
All fusion models using NARX had a high correlation between the predicted and observed mass flow (Table 3). The best model was with all variables with input to the NARX, with a correlation of 0.99. The model with only the strain gauge sensors has a correlation of 0.17 with observed values, a lower value than other combinations of variables using the NARX. Using the MLR, the best correlation was found (r = 0.63), using all sensors as input variables of the model (Table 2). The removal of the input variables RBSG and LBSG did not affect the estimation of the mass flow by the MLR, presenting a moderate correlation with the observed mass flow (r = 0.62). There was no correlation of predicted values with that observed (r = 0.17) using only data from the four locations with strain gauge sensors as input variables in the MLR. A similar result was obtained using only the displacement of the feeder rollers 1 and 2 as MLR input (r = 0.32).
Mass flow prediction with the NARX showed a significant superiority over MLR (Table 3). NARX with all input variables had an RMSE of 0.3 kg s−1, representing an MAPE of 0.7%. The RMSE of the MLR was 240% higher than the NARX. In previous studies using parameters of the harvester engine [14], the ANN model performed better (MAPE = 5.6%) than the MLR (MAPE = 7.8%) to predict sugarcane mass over the collection time. The error of the prediction by NARX shows that this approach can be an alternative for predicting sugarcane yield in a harvester. This approach, using a new multi-sensor model, has similar or even superior performance to the approach using a single sensor or system for measuring sugarcane yield, such as using fiber optics [10], load cells [9], or a deflector plate [11]. However, although the NARX model with multi-sensors had a smaller error than the other approaches. A disadvantage with the other approaches for yield prediction is the use of multi-sensors, which can increase the cost of implementing this system.
Other studies comparing prediction models have also shown higher prediction accuracy using an NARX compared to an MLR [41,42]. According to some studies, the NARX was the best performing model compared with the MLR due to its ability to dynamically adapt its trajectory based on previous errors [43,44]. Using the values previously predicted by the NARX minimizes the current prediction error, while MLR does not have the dynamic ability to learn from the error values of previous predictions. The ability of a model to self-calibrate is important when it comes to sensing crops with high yield variability, such as sugarcane [45]. This high variability over small distances directly influences the feeding rate of the harvester during the sugarcane harvest and, consequently, there will be high mass flow variation over a short time interval.
The removal of strain gauge sensors from the elevator base (RBSG and LBSG) did not influence mass flow prediction. The NARX had an RMSE of 0.4 kg s−1 without these sensors, a tiny difference (0.1 kg s−1) to the model with these two sensors. The vibrations caused by the displacement of the conveyor during the transport of the billets, together with the vibrations resulting from the harvester displacement, may interfere with the strain gauge signals. On the other hand, this can explain the low performance (compared to the other models tested in this study) of the NARX and MLR in predicting the mass flow using only the four points instrumented with a strain gauge. MLR with the input variables RCSG, LCSG, RBSG, and LBSG has the worst performance of the models tested in this study. The lower deformation sensitivity of the base material related to the sugarcane mass in the elevator is a factor that may be related to low performance. That is, the mass contained in the elevator may be insufficient for the sensor-capable deformation at the base of the elevator.
Using only the variables EPS and CPS obtained excellent results using the NARX, with RMSE values of 0.4 kg s−1. RMSE equal to the previous model without the variables RBSG and LBSG, but with a higher MAPE value (1.6%). This shows that the power required in the chopper hydraulic system [46] and the elevator conveyor are directly correlated with the sugarcane mass flow. A previous study [47] has shown that the feeding rate is one significant variable affecting chopping quality. In this way, the increase in the chopping quality can be with the automation of chopper roller speed utilizing the accurate mass flow prediction by the NARX. The mass flow can also be used to increase the power demand efficiency during sugarcane chopping. The chopper system is the one that demands the highest potency in the sugarcane harvester [48], and the automation of the chopper system using mass flow predicted can significantly increase the harvester efficiency.
MLR decreases the estimated mass flow variability in the sugarcane harvester (Figure 5). With MLR, the error was higher in the extreme values. It decreased close to the mean value, increasing the linear trend, with results similar to the use of engine parameters as data input of the MLR on sugarcane mass prediction [14]. The MLR overestimated the lower mass flow observed (errors closed to 19.2 kg s−1) and underestimated the high mass flow observed with errors of 20.4 kg s−1. The smoothing of mass flow data leads to loss of detail of the variability of essential information within the context of PA, especially for decision-making and management of variability of sugarcane fields. Knowing the sugarcane yield at small spatial distances is fundamental to the site-specific investigations in sugarcane rows [45]. Erroneous mass flow values can influence decision-making during the automation of the harvester’s control processes, reducing machine efficiency.
NARX has lower dispersion error than MLR (Figure 5). The nonlinear model has a lower error dispersion than the MLR because of its ability to explain more response variability [32]. Moreover, the nonlinear method performs a better job at re-calibration, as NARX is better at prediction in the short term. Its descending gradient learning tends to become more effective due to its integrated memory that provides a smaller path for propagating the information gradient when the network is opened, instead of back-propagating the error signal [49].
Using the validation data it was possible to spatially analyze the performance of the MLR and NARX models (Figure 6). Using MLR and NARX with all sensors (Fusion 1, Table 2), which had the best result in the previous tests (Table 3), it was possible to obtain prediction errors under 2%. It was evident that the MLR underestimated the yield within the field, with the predicted error upper of 1%, while the data predicted by NARX were similar to the observed data. However, it is probably less accurate in short-distance predictions, because there has been smoothing in the short-distance. Nevertheless, the model was able to capture larger-scale patterns of spatial variability, which is suitable for AP applications. The spatial variability found in the yield map can be used to identify patterns with high yield potential or patterns with low yield potential where yields are unlikely to increase.
From this study, a harvester instrumented with multi-sensors to predict mass flow with the NARX does not require different calibrations in the field. However, a study with multiple harvesters within the same field is necessary. Multi-harvesters with different calibration settings are typical in agricultural fields; these cause discontinuity of the data values within the field, and require post-processing of the data to remove this calibration difference [50,51]. Overall, the approaches in this study help support different applications in PA. The data sensors and the processing needed to generate the mass flow data can be easily embedded and implemented in a sugarcane harvester to develop its automation in the adjustments over the operation time, facilitate adoption, and promote site-specific management of sugarcane fields.

4. Conclusions

The fusion of sensor signals improves the accuracy of mass flow prediction. The nonlinear autoregressive networks with exogenous input (NARX) may be a viable alternative to conventional regression models for predicting mass flow from this study. It generates accurate mass flow data without the need to post-process the data. These data provide accurate information essential for the automation of the control processes in the harvester. The sugarcane mass flow approach with multiple sensors is a potential approach to replace conventional yield monitors. The system generates accurate data with high sample density within sugarcane rows. With a GNSS, it will be possible to spatialize the data within the field, generating spatialized yield information, which is the most valuable information for the localized management of sugarcane fields.

Author Contributions

Conceptualization, J.d.J.A.d.L. and J.P.M.; methodology, J.d.J.A.d.L. and J.P.M.; software, J.d.J.A.d.L.; validation, J.d.J.A.d.L. and L.F.M. formal analysis J.d.J.A.d.L.; writing—original draft preparation, J.d.J.A.d.L., L.F.M. and J.P.M.; writing—review and editing, J.d.J.A.d.L., L.F.M. and J.P.M.; funding acquisition, J.d.J.A.d.L. and J.P.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by AGCO Corporation (Canoas, RS, Brazil).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the São Martinho group (Pradópolis, SP, Brazil) for allowed to conduct this study in one of its mills.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ramos, C.R.G.; Pereira Lanças, K.; Albuquerque de Lyra, G.; Millani, T.M. Qualidade da colheita mecanizada de cana-de-açúcar em função da velocidade de deslocamento e rotação do motor da colhedora. Rev. Energ. Agric. 2014, 29, 87–94. [Google Scholar] [CrossRef] [Green Version]
  2. Ma, S.; Karkee, M.; Scharf, P.A.; Zhang, Q. Sugarcane harvester technology: A critical overview. Appl. Eng. Agric. 2014, 30, 727–739. [Google Scholar] [CrossRef]
  3. Xavier, W.D.; Silva, D.C.; da Costa, R.B.; Ribeiro, D.O.; Sousa, V.S.; João, J.V. Losses in the mechanized harvesting of sugarcane as of speed function of two harvester models in tropical savanna environment. Aust. J. Crop Sci. 2020, 14, 675–682. [Google Scholar] [CrossRef]
  4. da Silva, M.J.; de Lucas, L.; Correa, M.H.F.; de Souza, C.H.W. Quality Indexes and Performance in Mechanized Harvesting of Sugarcane at a Burnt Cane and Green Cane. Sugar Tech 2021. [Google Scholar] [CrossRef]
  5. Sanches, G.M.; Graziano Magalhães, P.S.; Junqueira Franco, H.C. Site-specific assessment of spatial and temporal variability of sugarcane yield related to soil attributes. Geoderma 2019, 334, 90–98. [Google Scholar] [CrossRef]
  6. Canata, T.F.; Chan, M.; Wei, F.; Maldaner, L.F.; Molin, J.P. Sugarcane Yield Mapping Using High-Resolution Imagery Data and Machine Learning Technique. Remote Sens. 2021, 13, 232. [Google Scholar] [CrossRef]
  7. Cox, G.; Cox, D.R.V.; Zillman, S.R.; Pax, R.A.; Bakker, D.M.; Harris, H.D. Mass Flow Rate Sensor for Sugar Cane Harvester. U.S. Patent No. 6,508,049, 21 January 2003. [Google Scholar]
  8. Cox, G.J. A Yield Mapping System for Sugar Cane Chopper Harvesters. Ph.D. Thesis, Uiversity of Southern Queensland, Toowoomba, Austrlia, 2002. [Google Scholar]
  9. Magalhães, P.S.G.; Cerri, D.G.P. Yield Monitoring of Sugar Cane. Biosyst. Eng. 2007, 96, 1–6. [Google Scholar] [CrossRef]
  10. Price, R.R.; Johnson, R.M.; Viator, R.P.; Larsen, J.; Peters, A. Fiber Optic Yield Monitor for a Sugarcane Harvester. Trans. ASABE 2011, 54, 31–39. [Google Scholar] [CrossRef]
  11. Quaderer, J.G.; Cash, M.F. Sugarcane Yield Mapping. U.S. Patent n. 8,955,402, 17 February 2015. [Google Scholar]
  12. Darr, M.J.; Corbett, D.J.; Herman, H.; Vallespi-Gonzalez, C.; Dugas, B.E.; Badino, H. Yield Measurement and Base Cutter Height Control Systems for a Harvester. U.S. Patent n. 10,371,561, 6 August 2019. [Google Scholar]
  13. Momin, M.A.; Grift, T.E.; Valente, D.S.; Hansen, A.C. Sugarcane yield mapping based on vehicle tracking. Precis. Agric. 2019, 20, 896–910. [Google Scholar] [CrossRef]
  14. Maldaner, L.F.; Corrêdo, L.d.P.; Canata, T.F.; Molin, J.P. Predicting the sugarcane yield in real-time by harvester engine parameters and machine learning approaches. Comput. Electron. Agric. 2021, 181, 105945. [Google Scholar] [CrossRef]
  15. Abbas, I.; Liu, J.; Faheem, M.; Noor, R.S.; Shaikh, S.A.; Solangi, K.A.; Raza, S.M. Different sensor based intelligent spraying systems in Agriculture. Sens. Actuators A Phys. 2020, 316, 112265. [Google Scholar] [CrossRef]
  16. Emmi, L.; Le Flécher, E.; Cadenat, V.; Devy, M. A hybrid representation of the environment to improve autonomous navigation of mobile robots in agriculture. Precis. Agric. 2021. [Google Scholar] [CrossRef]
  17. Kanagasingham, S.; Ekpanyapong, M.; Chaihan, R. Integrating machine vision-based row guidance with GPS and compass-based routing to achieve autonomous navigation for a rice field weeding robot. Precis. Agric. 2020, 21, 831–855. [Google Scholar] [CrossRef]
  18. Milella, A.; Reina, G.; Nielsen, M. A multi-sensor robotic platform for ground mapping and estimation beyond the visible spectrum. Precis. Agric. 2019, 20, 423–444. [Google Scholar] [CrossRef]
  19. Jang, J.S.R.; Sun, C.T. Neuro-Fuzzy Modeling and Control. Proc. IEEE 1995, 83, 378–406. [Google Scholar] [CrossRef]
  20. Thai, C.N.; Shewfelt, R.L. Modeling sensory color quality of tomato and peach. Neural networks and statistical regression. Trans. Am. Soc. Agric. Eng. 1991, 34, 950–955. [Google Scholar] [CrossRef]
  21. Pham, H.T.; Tran, V.T.; Yang, B.S. A hybrid of nonlinear autoregressive model with exogenous input and autoregressive moving average model for long-term machine state forecasting. Expert Syst. Appl. 2010, 37, 3310–3317. [Google Scholar] [CrossRef] [Green Version]
  22. Runge, J.; Zmeureanu, R. Forecasting energy use in buildings using artificial neural networks: A review. Energies 2019, 12, 3254. [Google Scholar] [CrossRef] [Green Version]
  23. Li, X.; Chen, W.; Chan, C.; Li, B.; Song, X. Multi-sensor fusion methodology for enhanced land vehicle positioning. Inf. Fusion 2019, 46, 51–62. [Google Scholar] [CrossRef]
  24. Aslinezhad, M.; Malekijavan, A.; Abbasi, P. ANN-assisted robust GPS/INS information fusion to bridge GPS outage. Eurasip J. Wirel. Commun. Netw. 2020, 2020, 1–18. [Google Scholar] [CrossRef]
  25. Andrade, A.; Lopes, K.; Lima, B.; Maitelli, A. Development of a methodology using artificial neural network in the detection and diagnosis of faults for pneumatic control valves. Sensors (Switzerland) 2021, 21, 853. [Google Scholar] [CrossRef]
  26. Raptodimos, Y.; Lazakis, I. Application of NARX neural network for predicting marine engine performance parameters. Ships Offshore Struct. 2020, 15, 443–452. [Google Scholar] [CrossRef]
  27. Ali, W.; Khan, W.U.; Raja, M.A.Z.; He, Y.; Li, Y. Design of Nonlinear Autoregressive Exogenous Model Based Intelligence Computing for Efficient State Estimation of Underwater Passive Target. Entropy 2021, 23, 550. [Google Scholar] [CrossRef]
  28. Ang, M.R.C.O.; Gonzalez, R.M.; Castro, P.P.M. Multiple data fusion for rainfall estimation using a NARX-based recurrent neural network-the development of the REIINN model. In IOP Conference Series: Earth and Environmental Science; Institute of Physics Publishing: Beijing, China, 2014; Volume 17, p. 012019. [Google Scholar]
  29. Le, V.M.; Pham, B.T.; Le, T.T.; Ly, H.B.; Le, L.M. Daily Rainfall Prediction Using Nonlinear Autoregressive Neural Network. In Lecture Notes in Networks and Systems; Springer: Singapore, 2020; Volume 106, pp. 213–221. [Google Scholar] [CrossRef]
  30. Corrêdo, L.d.P.; Canata, T.F.; Maldaner, L.F.; de Lima, J.d.J.A.; Molin, J.P. Sugarcane Harvester for In-field Data Collection: State of the Art, Its Applicability and Future Perspectives. Sugar Tech 2020, 23, 1–14. [Google Scholar] [CrossRef]
  31. Barcelo-Ordinas, J.M.; Doudou, M.; Garcia-Vidal, J.; Badache, N. Self-calibration methods for uncontrolled environments in sensor networks: A reference survey. Ad Hoc Netw. 2019, 88, 142–159. [Google Scholar] [CrossRef] [Green Version]
  32. Ferrer-Cid, P.; Barcelo-Ordinas, J.M.; Garcia-Vidal, J.; Ripoll, A.; Viana, M. A Comparative Study of Calibration Methods for Low-Cost Ozone Sensors in IoT Platforms. IEEE Internet Things J. 2019, 6, 9563–9571. [Google Scholar] [CrossRef] [Green Version]
  33. Chen, S.; Billings, S.A. Neural networks for nonlinear dynamic system modelling and identification. Int. J. Control 1992, 56, 319–346. [Google Scholar] [CrossRef]
  34. Di Piazza, A.; Di Piazza, M.C.; Vitale, G. Solar and wind forecasting by NARX neural networks. Renew. Energy Environ. Sustain. 2016, 1, 39. [Google Scholar] [CrossRef] [Green Version]
  35. Xiu, Y.; Zhang, W. Multivariate Chaotic Time Series Prediction Based on NARX Neural Networks. In Advances in Engineering Research; Atlantis Press: Paris, France, 2017; pp. 164–167. [Google Scholar] [CrossRef] [Green Version]
  36. Jung, D.H.; Kim, H.S.; Jhin, C.; Kim, H.J.; Park, S.H. Time-serial analysis of deep neural network models for prediction of climatic conditions inside a greenhouse. Comput. Electron. Agric. 2020, 173, 105402. [Google Scholar] [CrossRef]
  37. Wendte, K.W.; Skotnikov, A.; Thomas, K.K. Sugar Cane Yield Monitor. U.S. Patent n. 6,272,819, 14 August 2001. [Google Scholar]
  38. Hernandez, B.; Fernandez, F.; Ponce, E.; Esquivel, M.; Rodriguez, J.; Quintana, L.; Interian, S.; Alvarez, R. Application of precision farming technologies for sugarcane crop. In Proceedings of the 5th European Conference on Precision Agriculture; Heemskerk, C., Engval, M., Eds.; Wageningen Academic Publishers: Wageningen, The Netherlands, 2005; pp. 9–12. [Google Scholar]
  39. Fernandez, F.; Hernandez, B.; Esquivel, M.; Marrrereo, S.; Ponce, E.; Quintana, L. Yield mapping sugarcane in Australia. In Proceedings of the 6th European Conference on Precision Agriculture, Skiathos, Greece, 7 July 2007; pp. 3–6. [Google Scholar]
  40. Tukey, J.W. Exploratory Data Analysis; Addison-Wesley Publishing Company: London, UK, 1977. [Google Scholar]
  41. Zounemat-Kermani, M.; Stephan, D.; Hinkelmann, R. Multivariate NARX neural network in prediction gaseous emissions within the influent chamber of wastewater treatment plants. Atmos. Pollut. Res. 2019, 10, 1812–1822. [Google Scholar] [CrossRef]
  42. Mustapa, R.F.; Dahlan, N.Y.; Yassin, A.I.M.; Nordin, A.H.M. Quantification of energy savings from an awareness program using NARX-ANN in an educational building. Energy Build. 2020, 215, 109899. [Google Scholar] [CrossRef]
  43. Zhang, F.; Murphy, M.D.; Shalloo, L.; Ruelle, E.; Upton, J. An automatic model configuration and optimization system for milk production forecasting. Comput. Electron. Agric. 2016, 128, 100–111. [Google Scholar] [CrossRef]
  44. Murphy, M.D.; O’Mahony, M.J.; Shalloo, L.; French, P.; Upton, J. Comparison of modelling techniques for milk-production forecasting. J. Dairy Sci. 2014, 97, 3352–3363. [Google Scholar] [CrossRef] [Green Version]
  45. Maldaner, L.F.; Molin, J.P. Data processing within rows for sugarcane yield mapping. Sci. Agrícola 2020, 77, e20180391. [Google Scholar] [CrossRef]
  46. Xie, L.; Wang, J.; Cheng, S.; Zeng, B.; Yang, Z. Performance Evaluation of a Chopper System for Sugarcane Harvester. Sugar Tech 2019, 21, 825–837. [Google Scholar] [CrossRef]
  47. Kent, G.A. Issues Associated with Using Trash as a Cogeneration Fuel. Sugar Tech 2014, 16, 227–234. [Google Scholar] [CrossRef] [Green Version]
  48. Alencastre-Miranda, M.; Davidson, J.R.; Johnson, R.M.; Waguespack, H.; Krebs, H.I. Robotics for Sugarcane Cultivation: Analysis of Billet Quality using Computer Vision. IEEE Robot. Autom. Lett. 2018, 3, 3828–3835. [Google Scholar] [CrossRef]
  49. Siegelmann, H.T. Stochastic analog networks and computational complexity. J. Complex. 1999, 15, 451–475. [Google Scholar] [CrossRef] [Green Version]
  50. Sams, B.; Litchfield, C.; Sanchez, L.; Dokoozlian, N. Two methods for processing yield maps from multiple sensors in large vineyards in California. Adv. Anim. Biosci. 2017, 8, 530–533. [Google Scholar] [CrossRef]
  51. Leroux, C.; Jones, H.; Pichon, L.; Taylor, J.; Tisseyre, B. Automatic harmonization of heterogeneous agronomic and environmental spatial data. Precis. Agric. 2019, 20, 1211–1230. [Google Scholar] [CrossRef]
Figure 1. Flowchart of mass flow prediction.
Figure 1. Flowchart of mass flow prediction.
Sensors 21 04530 g001
Figure 2. NARX neural network structure. d is order delay. W is the network input vector and the feedback vector weight matrix. b is bias. f1 is the neural network hidden layer activation function. f2 is the neural network output layer activation function [35].
Figure 2. NARX neural network structure. d is order delay. W is the network input vector and the feedback vector weight matrix. b is bias. f1 is the neural network hidden layer activation function. f2 is the neural network output layer activation function [35].
Sensors 21 04530 g002
Figure 3. Sensor locations in the harvester (a). Potentiometric displacement sensor in the roll feeder number 1 (b) and roll feed number 2 (c). The pressure sensor in the hydraulic motor of the chopper (d). Strain gauges in the elevator base (e) and the hydraulic cylinder of the elevator (f). The pressure sensor in the hydraulic motor of the elevator (g).
Figure 3. Sensor locations in the harvester (a). Potentiometric displacement sensor in the roll feeder number 1 (b) and roll feed number 2 (c). The pressure sensor in the hydraulic motor of the chopper (d). Strain gauges in the elevator base (e) and the hydraulic cylinder of the elevator (f). The pressure sensor in the hydraulic motor of the elevator (g).
Sensors 21 04530 g003
Figure 4. Boxplot of the mass flow (kg s−1) predicted by the models. The same letters above the box indicate no significant difference between the mean mass flow predicted value of the models (Tukey test; p < 0.05). MLR—multiple linear regression (blue box). NARX—nonlinear autoregressive networks with exogenous input (red box). The description of the different fusion models is in Table 2.
Figure 4. Boxplot of the mass flow (kg s−1) predicted by the models. The same letters above the box indicate no significant difference between the mean mass flow predicted value of the models (Tukey test; p < 0.05). MLR—multiple linear regression (blue box). NARX—nonlinear autoregressive networks with exogenous input (red box). The description of the different fusion models is in Table 2.
Sensors 21 04530 g004
Figure 5. Scatter plot of the Multiple Linear Regression (MLR) and NARX Neural Network (ANN) models prediction error.
Figure 5. Scatter plot of the Multiple Linear Regression (MLR) and NARX Neural Network (ANN) models prediction error.
Sensors 21 04530 g005
Figure 6. Yield map of the observed data predicted by multiple linear regression (MLR) and nonlinear autoregressive networks with exogenous input (NARX). Below the yield map contains the prediction error (%), spatialized.
Figure 6. Yield map of the observed data predicted by multiple linear regression (MLR) and nonlinear autoregressive networks with exogenous input (NARX). Below the yield map contains the prediction error (%), spatialized.
Sensors 21 04530 g006
Table 1. Amount of data used for training, testing, and validation of the models in this study.
Table 1. Amount of data used for training, testing, and validation of the models in this study.
Row NumberTrain and TestValidation
138969
235863
330153
430955
535162
633158
734461
833259
934360
Table 2. Combinations between the different sensors tested in this study.
Table 2. Combinations between the different sensors tested in this study.
Models Input VariablesSensor
Fusion 1 = {CPS, EPS, FRO1, FRO2, RCSG, LCSG, RBSG, LBSG}PS, SGS, DPS
Fusion 2 = {CPS, EPS, FRO1, FRO2, RCSG, LCSG}PS, SGS, DPS
Fusion 3 = {EPS, RCSG, LCSG}PS, SGS
Fusion 4 = {CPS, FRO1, FRO2}PS, DPS
Fusion 5 = {FRO1, FRO2}DPS
Fusion 6 = {CPS, EPS}PS
Fusion 7 = {RCSG, LCSG, RBSG, LBSG}SGS
Fusion 8 = {EPS}PS
Fusion 9 = {CPS}PS
PS: Pressure sensor. SGS: Strain gauge sensor. PDS: Potentiometric displacement sensor. LBSG: SGS in the left base. RBSG: SGS in the right base. LCSG: SGS in the right hydraulic cylinder. RCSG: SGS in the left hydraulic cylinder. FRO1: PDS in the feeder roll #1. FRO2: PDS in the feeder roll #2. CPS: PS in the chopper hydraulic motor. EPS: PS in the conveyor hydraulic motor.
Table 3. Mass flow prediction assessment metrics of the models tested in this study.
Table 3. Mass flow prediction assessment metrics of the models tested in this study.
Models Input VariablesCorrelationRMSE (kg s−1)MAPE (%)
MLRNARXMLRNARXMLRNARX
CPS, EPS, FRO1, FRO2, RCSG, LCSG, RBSG, LBSG0.630.991.60.34.90.7
CPS, EPS, FRO1, FRO2, RCSG, LCSG0.620.981.60.44.90.8
EPS, RCSG, LCSG0.560.961.70.56.32.4
CPS, FRO1, FRO20.500.971.80.55.41.3
FRO1, FRO20.320.961.90.57.31.0
CPS, EPS0.610.971.60.45.11.6
RCSG, LCSG, RBSG, LBSG0.170.932.00.78.81.2
EPS0.490.951.80.65.71.2
CPS0.560.941.70.66.52.5
LBSG: strain gauge in the left base. RBSG: strain gauge in the right base. LCSG: strain gauge in the right hydraulic cylinder. RCSG: strain gauge in the left hydraulic cylinder. FRO1: potentiometric sensor in the feeder roll #1. FRO2: potentiometric sensor in the feeder rolls #2. CPS: pressure sensor in the chopper hydraulic motor. EPS: pressure sensor in the conveyor hydraulic motor.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lima, J.d.J.A.d.; Maldaner, L.F.; Molin, J.P. Sensor Fusion with NARX Neural Network to Predict the Mass Flow in a Sugarcane Harvester. Sensors 2021, 21, 4530. https://doi.org/10.3390/s21134530

AMA Style

Lima JdJAd, Maldaner LF, Molin JP. Sensor Fusion with NARX Neural Network to Predict the Mass Flow in a Sugarcane Harvester. Sensors. 2021; 21(13):4530. https://doi.org/10.3390/s21134530

Chicago/Turabian Style

Lima, Jeovano de Jesus Alves de, Leonardo Felipe Maldaner, and José Paulo Molin. 2021. "Sensor Fusion with NARX Neural Network to Predict the Mass Flow in a Sugarcane Harvester" Sensors 21, no. 13: 4530. https://doi.org/10.3390/s21134530

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop