Early-Stage Sensor Data Fusion Pipeline Exploration Framework for Agriculture and Animal Welfare

Abstract

Internet-of-Things (IoT) approaches are continually introducing new sensors into the fields of agriculture and animal welfare. The application of multi-sensor data fusion to these domains remains a complex and open-ended challenge that defies straightforward optimization, often requiring iterative testing and refinement. To respond to this need, we have created a new open-source framework as well as a corresponding Python tool which we call the “Data Fusion Explorer (DFE)”. We demonstrated and evaluated the effectiveness of our proposed framework using four early-stage datasets from diverse disciplines, including animal/environmental tracking, agrarian monitoring, and food quality assessment. This included data across multiple common formats including single, array, and image data, as well as classification or regression and temporal or spatial distributions. We compared various pipeline schemes, such as low-level against mid-level fusion, or the placement of dimensional reduction. Based on their space and time complexities, we then highlighted how these pipelines may be used for different purposes depending on the given problem. As an example, we observed that early feature extraction reduced time and space complexity in agrarian data. Additionally, independent component analysis outperformed principal component analysis slightly in a sweet potato imaging dataset. Lastly, we benchmarked the DFE tool with respect to the Vanilla Python3 packages using our four datasets’ pipelines and observed a significant reduction, usually more than 50%, in coding requirements for users in almost every dataset, suggesting the usefulness of this package for interdisciplinary researchers in the field.

1. Introduction

This section provides an overview of the need for data fusion in precision agriculture and animal welfare research fields, categorizes common techniques and formats used in data fusion, and summarizes the four datasets to which we have access for proof-of-concept demonstrations.

1.1. The Need for Data Fusion in Precision Agriculture and Animal Welfare

The past few decades have seen a remarkable achievement in the development of new sensors and improvement of their operational performance (e.g., sensitivity, specificity, power consumption) combined with novel network technologies enabling the Internet-of-Things (IoT) to become ubiquitous in every aspect of our lives. The information age has brought forth data centers, cloud computing, smart technologies, and distributed personal smart devices. As much as 95% of industry is investing in cloud solutions and embedded sensors in some way [1] and the market value of IoT systems in the healthcare, automotive, and consumer electronics industries has nearly doubled from 2014 to 2020 to more than USD 1 trillion in each [2]. Precision agriculture and smart farming have attracted increased attention more recently due to growing awareness about the challenges of feeding the growing global population.

Improvements in the accuracy, cost, and resolution of sensor systems and devices have also improved our understanding of the physical world in these domains. However, multiple sensors measuring various physical signals in an environment pose a significant challenge in interpreting these data streams due to the complexity and volume of data representing a diverse set of phenomena. There is a growing need for data fusion and fission tools to handle such large volumes of data acquired by multiple sensors simultaneously. Sensors work because physical phenomena corresponding to events move through a space and interact with the sensor. The splitting of information from the event to various observations in a one-to-many relationship defines data fission. Various sensors allow scientists to record these pieces of information from a various points of view [3]. The methods used to fuse these data back together are problem-specific. In the last few decades, many successful attempts have been made to expand these techniques to precision agriculture domains.

1.2. The Data Flow in Precision Agriculture and Animal Welfare

The Dasarathy model groups data fusion techniques that can be used in precision agriculture and animal welfare applications by level of abstraction [3]. Information is evaluated by whether it is data (low level), features (mid level), or decisions (high level). Unfortunately, there is no perfect technique that works for all problems. Even more advanced data fusion techniques perform worse in certain scenarios [4]. Instead, design iteration based on trial-and-error testing is necessary. Idiosyncrasies also exist with each data fusion problem, such as pre-processing needs, certain formatting corrections, etc., which in turn require unique solutions. Fortunately, there are common procedures that can be grouped. Data typically starts in one of three formats:

• Singlets: information is low-dimensional, such as a temperature from a thermistor, or physical activity level from an inertial measurement unit.
• Arrays: data is collected over a range. Examples include spectral data, soil moisture across a field, etc.
• Images: pixel-based images, most commonly from camera-based sensors for computer vision-based analysis, such as of diseased crops and fruits.

Array-style data often requires dimensional reduction due to large redundancies in arrays. Image-style data also requires image processing before proper feature selection can take place. Temporal and spatial alignment is required early on to ensure proper comparison of various sensors (Figure 1). This is typically followed by feature extraction to reduce the size of the data while retaining the main information. Decision making, usually performed with a classifier or predefined ruleset, must be carried out before fusion from different sources can then take place. All of these different data formats emerging in the precision agriculture and animal welfare domains set the framework upon which a data fusion strategy can be built.

1.3. The Focus and Contribution of This Paper

In addition to presenting our proposed data fusion pipeline methodology to the scientific community, here we demonstrate the feasibility and effectiveness of the framework in completing custom data fusion objectives with four datasets as application examples. Two of these datasets are related to crop health and the other two to animal monitoring. We also share unplanned sub-objectives that organically arose during the data pipeline development. These were easy to implement with our corresponding toolkit despite not being pre-planned, indicating a type of emergent behavior within this module. This highlights the potential of the tool, as its modular setup allows for easy customization. The objectives we were able to complete include the following:

• Determined when to perform a data fusion procedure in the data pipeline;
• Compared low-level and mid-level fusion approaches for a dataset; and
• Prototyped different pipeline events to determine an optimal ordering.

Overall, the pipeline methodology and the corresponding tool allow researchers to obtain a broad and high-level perspective on the structure of their data fusion processes from raw data to final inference without being burdened by the intricacies of data formatting or best coding practices.

2. Review of the State of the Practice

Data fusion methods have been extensively applied across various industries, especially with the rapid growth of IoT and sensor technologies. In the food and beverage industry, data fusion has increased classification reliability and foodstuff specifications [4]. In the sports and biomedical fields, where multiple features are often required to make accurate detections, fusing data from various sensors enabled more complex and robust analyses [5]. In precision farming, data fusion has led to reduced network traffic and the development of innovative methodologies such as virtual fencing [6]. Similarly, in smart homes, data fusion has shown potential in energy management, uncertainty reduction, and the handling of unstructured data [7].

The fundamental concept behind multi-sensor data fusion is to integrate data in a way that produces synergistically improved outcomes, such as results that are actionable, explainable, and personalized [7]. Other advantages of data fusion include compensating for inaccurate data, lowering energy consumption in wireless networks, enhancing reliability, reducing noise, increasing robustness to individual node failures, and improving precision [8]. Together, these benefits contribute to higher quality of service and more efficient power usage. Furthermore, such networks offer greater flexibility and can support localized decision making within smaller network subsets [3].

These are all broad, industry-wide applications of sensor data fusion. However, since the intended application area for our data fusion framework is agriculture and animal monitoring, we investigated common research topics in these fields that specifically use data fusion. We would like to emphasize that providing a comprehensive and systematic review is beyond the scope of this paper. Instead, we provide a compilation of exciting examples to offer a snapshot of the state-of-the-practice in sensor fusion to study animals and plants. Subsequently, we review the current potential data fusion Python software libraries and explain how our proposed software is unique.

2.1. Crop and Agriculture-Related Sensor Data Fusion

In agricultural applications focusing on crops, previous efforts on data fusion can be broadly categorized in three main categories: plant health ([9,10,11,12]), food source quality ([4,10,13,14,15,16]), and soil profiling ([17,18,19,20]). Many of these applications address key agricultural aspects. As the IoT continues to evolve, crops can now be evaluated at multiple levels: the high (field- or crop-) level, the medium (organ-) level, and the low (cellular-) level [21]. This section concludes with an overview of the constraints and opportunities for sensor fusion in farm environments.

Healthy plants are disease- and defect-free and well-hydrated. Data fusion techniques have been applied to these indicators of plant health. For example, apples were classified based on spoilage, ripeness, and flavor using electronic noses [10]. Plant hydration levels have been assessed using bioimpedance spectroscopy, where surface electrodes placed on Arabidopsis sprouts were used to track the impedance over time, enabling precision hydration assessments [12]. Various mosaic viruses and fungal infections in cucumbers were monitored over several weeks using a combination of temperature sensors, thermography, and fluorometry with data fused using stepwise discriminant analysis [9]. In some cases, genetic diseases can be monitored using data fusion with unmanned aerial vehicles (UAVs) or drones [11]. These researchers fused large-scale traditional, multispectral and thermal images using extreme learning machines to predict chlorophyll a and b levels.

Ensuring food quality is obviously critical for public health. Electronic noses, near-infrared spectroscopy (NIRS), and machine vision have been fused with decision trees to detect unripe apples [16]. Interactance spectroscopy has been used to identify internal defects in sweet potatoes [15]. Banned dyes, such as Sudan III and IV, have been successfully detected in culinary spices using a combination of nuclear magnetic resonance and ultraviolet–visible spectroscopy, followed by data fusion using concatenation and fuzzy logic. This achieved over 97% accuracy [13]. Sensor fusion also contributed to other culinary applications; for example, electronic noses and tongues have been combined using a fuzzy neural network to identify wine types [14]. A comprehensive review of other fusion methods applied to vegetables, fruits/juices, cheeses, and honey is provided in [4].

Data fusion techniques have also been applied to soil profiling. In one study, pH, organic carbon, nitrogen and phosphorus levels were combined with measurements such as soil texture, trunk diameter, and carbon dioxide efflux to study the relationship between soil quality and poplar tree growth [18]. To address soil heterogeneity in precision agriculture, gamma-ray spectrometry, visible and near-infrared spectrometry, and a galvanic soil sensor were used to determine soil electrical conductivity and radioisotope content [19]. The Land Surface Data Toolkit, a multi-model simulation tool [17], integrates data using artificial neural networks to support agricultural and water management practices.

Lastly, a broader scope of agricultural applications of sensor fusion is discussed in [6], covering aspects such as location and user awareness, network traffic reduction, and implementation of virtual fences. One common trade-off in these applications involves the exchange of processing power with high-level fusion schemes for reduced network traffic, a concept known as edge mining. Common hindrances to such agricultural applications include managing data distribution across large farming areas and dealing with poor communication environments. Sensors must also be robust to withstand harsh environmental conditions found in large agricultural settings, including significant temperature and humidity fluctuations, as well as contamination from dirt and other waste. Despite these challenges, there are opportunities that need to be considered. Farm environments, combined with edge computing, can reduce reliance on cloud infrastructure and improve data security. In addition, small-scale energy harvesting technologies, such as solar and wind energy, can significantly support these distributed systems.

2.2. Animal-Monitoring-Related Sensor Data Fusion

In animal-related applications, sensor data fusion has been primarily applied to animal classification ([22,23,24,25]) and gait analysis ([5,26,27,28,29]). When extended to humans, data fusion has also supported fall detection in the elderly ([28,30,31]), assisted living and smart home applications [32], and biometric health monitoring, such as in patient care, chronic disease management and stress detection ([5,28,33,34]). Since our platform is designed to support animal-focused fusion applications, we concentrate here on the most relevant domains, including animal classification, gait analysis, and biometric health monitoring.

Animal classification is widely used in wildlife animal tracking to understand migration patterns, species endangerment, and biodiversity. Data fusion methods vary depending on species and sensor types, which may range from satellite images to local acoustic sensors. For example, zebra and nyala were classified using a residual neural network-based computer vision system, achieving a top-1 accuracy of 72% for the zebras but only 4.7% for the nyalas. The poor latter performance was due to effective camouflage [25]. In marine environments, plankton were classified using two video cameras and eight acoustic receivers, with confidence-weighted fusion reducing classification error by over 50% [24]. Population estimates of 35 animal species have also been achieved using combined data from visual transects, dung transects, and camera traps [23]. More recently, satellite geomagnetic imaging has been proposed to complement traditional tracking techniques for studying migratory behavior [22].

Gait classification typically employs wearable sensors. In one study, high-dimensional raw sensor data was used to classify dog behaviors such as standing, walking, running, sitting, and resting, with quaternion-based models achieving 93% precision [26]. Other research has modeled gait at the physiological and biomechanical levels. For example, fuzzy logic and multiple surface electromyography (EMG) sensors were used to estimate skeletal muscle forces [29]. Additional studies combined inertial measurement units (IMUs) with EMG sensors to calculate energy expenditure and muscle activation [27]. Sensor fusion techniques have also been applied using pressure sensors in footwear and sports equipment, including basketballs, helmets, and hockey sticks, to analyze motion and biomechanics [5].

Although primarily applied to humans, many health biometrics fusion techniques are adaptable to animal monitoring. A study combined step count, calorie intake, and sleep quality data to predict heart disease achieving 98% accuracy by focusing on the most predictive features [33]. Stress detection has also been explored using various biosignals, including electrocardiography (ECG), EMG, electrodermal activity (EDA), and electroencephalography (EEG). Interestingly, ECG alone produced the most significant results [34]. Additionally, researchers have found that data fusion can help reduce false positive rates in medical testing [5]. A comprehensive overview of wearable health monitoring systems is available in [28].

2.3. Open-Source Tools

Considered as the first open-source platform for the development of biomarkers, the Digital Biomarker Discovery Pipeline (DBDP) was designed to work with the developing field of mobile health (mHealth) to assist in the monitoring of chronic diseases [35]. As an open-source tool, its aim is to help facilitate collaboration between various medical communities and to strongly encourage contributions from other like-minded researchers in the community. Upon release, the DBDP provided three example modules—“Resting Heart Rate”, “Sleep and Circadian Rhythms”, and “Continuous Glucose Monitoring”. In 2024, the publicly available DBDP included common modules like “Preprocessing” and “Exploratory Data Analysis”, as well as specific task-oriented modules, such as “Glucose Variability”, “Heart Rate Variability”, etc. (https://github.com/DigitalBiomarkerDiscoveryPipeline/DBDP, accessed on 17 April 2025).

A very popular Python package for machine learning applications, Scikit-learn, was originally designed as a high-level language for medium-scale machine learning problems [36]. Its aim is to bring machine learning algorithms to non-software specialists, especially in other scientific fields such as biology. The creators focused on making Scikit-learn workable with other common packages such as NumPy, scipy, and matplotlib. Despite the focus on creating a high-level language with good computing efficiency, this tool was planned to scale to larger datasets in future research [36].

These state-of-the-practice packages and platforms can be compared to our tool in the following. Similarly, all of our earlier work and our tool utilize open-source code for widespread adoption using public GitHub repositories. These use common platforms such as NumPy, pandas, matplotlib, and Scikit-learn. These tools are designed as end-to-end platforms with expectations that future modules will be able to be implemented easily. The main difference is that the DBDP code bases were explicitly designed to provide access to the medical community for mHealth applications. Scikit-learn makes machine learning accessible to the scientific community mainly related to biology and physics. Neither of these has a heavy emphasis on data fusion.

While similar in modularity and intention, our data fusion tool is uniquely designed to bring data fusion exploration distinctly to early-scale testing in animal and plant analytics. Its focus area is early-stage pipeline prototyping within the veterinary and agricultural application areas, and the codebase is more generalized for end-to-end pipeline development for researchers in these fields.

3. Methods

The overall schema for our data fusion framework was designed to align with the five fundamental elements of a typical data fusion process (Figure 1). In summary:

• Data Alignment assists with temporal and spatial alignment between different sources;
• Feature Extraction performs temporal feature extraction from a time series and includes feature selection strategies;
• Dimensional Reduction reduces the set of features of a dataset;
• Data Fusion incorporates data from multiple various sources to arrive at a decision;
• Decision Making sets the decision space for the model using classification and regression models.

As a theoretical example, we have a network of motion sensors on a single animal harness and want to know how much movement takes place during demonstrated behaviors: running, walking, and eating. Figure 2 shows an overview of the following process. We could see how data alignment would be necessary to account for sensors on different body parts; these would also have to be temporally aligned to ensure the proper timing of each behavior. We then perform feature extraction techniques, such as finding the average impulse of movements or integrating motion into temporal windows. These metrics tell us what we want to know. If the dataset is too large, or highly correlated, as can be expected of sensors on the same animal, we may then reduce the dimension size with principal component analysis (PCA). We fuse the data into one large dataset, using raw data or the features extracted before. Lastly, a classifier would be trained to distinguish running from walking and vice versa.

3.1. Towards an Open Source Tool for Precision Agriculture and Animal Welfare Researchers

In this paper, we also present the “Data Fusion Explorer (DFE),” a software toolbox for precision agriculture and animal welfare researchers to perform early-stage discovery without requiring a deep level of expertise on multi-sensor data fusion (Figure 1). This is an open-source Python software to assist users in rapidly prototyping fusion schemes to gain valuable insights in even earlier stages of their research. We refer to each sequence, from raw data through processing to regression or classification outcomes, as an individual data pipeline. Due to the open-ended nature of data fusion and the multitude of possible sequence rearrangements, we find iterative prototyping to be more beneficial than intricate model design. This work uniquely presents a generalized data fusion platform designed for rapid experimental prototyping and enabled by a distinct design framework derived from common applications in the animal monitoring and growing precision agriculture spaces. The study presented here also includes the applicability and translation of this framework as a new open-source Python package.

The DFE allows users to work at the pipeline level in a plug-and-play manner. The purpose of this effort is to facilitate early exploration and development of a data pipeline, to organize the most common tasks within a global framework, and to be functionally useful for different data formats in the field of agricultural and animal monitoring. Following the development and operations (DevOps) strategy, we released a reasonably useful product before formal completion. This also includes updated documentation and 102 unit test cases that have been automated with PyTest using GitHub Actions [37] as well as multiple example cases that act as informal integration tests.

As indicated, the DFE was designed with data problems in agricultural and animal monitoring spaces in mind. These application areas span fields where data fusion is common, such as soil measurements [19], the food industry [4], determination of food quality, and animal welfare and monitoring. Datasets were also selected to vary in formatting between singlet, array-style, and image-style data.

Coded in Python3, using open-source modules like scipy, numpy, pandas, sklearn, etc. and accessible using the pip installer, the DFE can realize these corresponding data fusion pipelines. Users create a DFE_object and all further calls are made through this object. An associated reference guide that describes all available methods as well as their inputs and outputs is included in the GitHub repository. Modifications to the software undergo automated checks in accordance with continuous integration and deployment (CI/CD) to ensure continued compatibility with past versions.

3.2. Dataset Descriptions

We had access to four diverse early-stage datasets (

Table 1. Description of the datasets used for evaluating the data fusion pipeline framework.

Dataset	Dog Collar	Moth Detector	Plant Impedance	Sweet Potato
Research	Animal	Environmental	Agrarian	Food Quality
Area	Monitoring	Monitoring	Monitoring	Monitoring
Dataset Type	Singlet	Singlet	Array	Image
Class/Reg	Classification	Regression	Regression	Classification
Temporal/Spatial	Temporal	Spatial	Temporal	Spatial
Sensors	Envir., Audio,	Envir.,	Impedance	Camera
	Inertial	Moth Trap	Spectrum

) that we used to demonstrate the usefulness of the presented data fusion framework for a wide variety of agricultural and animal applications. It should be noted that the analysis of these databases is beyond the scope of this paper and not the intention of the research presented here. Rather, the three common types of data formatting (singlets, arrays, and images) contained in these databases are used as a medium to compare and benchmark various features of pipelining, modularization, and rapid prototyping. The array-style data is separated from singlet-style data because array data requires a dimensional reduction step before further processing. This is to prevent over-analyzing highly redundant data.

3.3. Dog Collar Dataset: Singlets to Classification

The Dog Collar Dataset was acquired from a sensor system attached to the collars of guide dog puppies being trained for one of the largest guide dog organizations in the United States: Guiding Eyes for the Blind [38]. The collar sensor collects data related to the local environment (i.e., ambient temperature, relative humidity, barometric pressure, ambient light level, ambient acoustic noise level) and dog behavior (i.e., inertial measurement units including accelerometers) simultaneously (Table 1). This database is used to better understand and predict the performance of guide dog training. During guide dog puppy evaluation and testing, the dogs performed tasks in response to several dozen activity states.

For this dataset, we used our data fusion framework to classify the activity state of the dog. We compared three pipelines spanning low-level (data to decision, see P1) and mid-level (feature to decision) fusion. We also compared two mid-level pipelines to observe the effect of fusion followed by classification as separate steps, such as principal component analysis (PCA)—Random Forest (P2) versus fusion and classification performed in the same step, in linear discriminant analysis (LDA) (P3). The different pipelines are shown in Figure 3.

3.4. Moth Detector Dataset: Singlets to Regression

The second dataset was related to a sensing platform developed to count corn earworm moths emerging in agriculture fields [39]. It is a moth trap with an infrared-based counter implemented at the trap entrance. The system also measured local environmental data, including ambient temperature, relative humidity, barometric pressure, wind, rain, and ambient light levels (Table 1). These sensors were installed in farmland regions around North Carolina for several months.

The objective of using the dataset in this paper was to explore the relation between moth emergence density and local environmental conditions. Using the presented data fusion framework, we compared three different pipeline schemas varying by window scheme (see Figure 4).

3.5. Plant Impedance Dataset: Arrays to Regression

The third dataset was a set of sensors placed on the leaves of maize plants to measure the impedance of the leaf in parallel with the environmental conditions in an effort to optimize irrigation and fertilizer applications. This plant leaf impedance dataset contained months of impedance spectroscopy data with the sensor system producing a spectrum every ten minutes [12] (Table 1). Previous work with this dataset used a MATLAB-based (R2023b, MathWorks Inc., Natick, MA, USA) script to fit a double-shell equivalent circuit to the spectrum and monitored the corresponding parameters over days of collected data. The objective here was to extract drought-related model parameters from features of the impedance spectrums. Three pipelines were performed with this dataset, varying according to the order dimension reduction, feature extraction, and linear regression (see Figure 5). The two upper paths (P1 and P2) change the order of dimension reduction and feature extraction, while the bottom path (P3) merges dimension reduction and regression using the least absolute shrinkage and selection operator (LASSO).

3.6. Sweet Potato Dataset: Images to Classification

The final dataset consisted of image data captured in a sweet potato processing facility aimed at automating the high-throughput quality assessment of food resources [40]. The completed image processing steps and other details can be found in [40]. Starting with the features presented in this study, the aim was to classify each potato quality as either “U.S. No. 1” or “Cull” quality (Table 1). We achieved classification with two dimension reduction techniques followed by classification (Figure 6). The hypothesis was that independent component analysis (ICA) would perform better with a Naïve Bayes classifier, as this classifier assumes independence in the data, which aligns with ICA’s objective of maximizing independence.

3.7. Metrics of Comparison

For pipeline comparisons, we report different metrics based on the task type:

For classification tasks, we report the following metrics (all averaged over classes unless otherwise stated):

• Precision measures the portion of correctly classified instances among all instances classified as a given class.
• Recall measures the portion of actual instances of a class that were correctly identified.
• F1-Score is the harmonic mean of precision and recall and a common metric for overall classification performance.
• Accuracy is the overall proportion of correctly predicted instances across all classes.

For regression tasks, we report the following:

• The Pearson Correlation Coefficient (R) measures the linear correlation between predicted values and ground truth.
• Root Mean Square Error (RMSE) measures the average squared differences between predicted and actual values.
• Relative Root Mean Square Error (RelRMSE) is similar to RMSE, but normalized by the magnitude of predictions.
• Mean Absolute Error (MAE) computes the average absolute difference between predictions and actual values, treating all errors equally.
• Relative Absolute Error (RAE) is a normalized version of MAE, relative to the errors in a naive model or baseline.

We also report space and time complexity metrics for each pipeline to enable performance comparisons. Furthermore, to assess the usefulness of the corresponding DFE, we measured the number of lines of code saved when using the five DFE modules versus implementing the same functionality without them.

4. Results

We calculated metrics to determine the best fusion pipeline for each dataset. With each model, we divided the data into a 80% training set and a 20% testing set. For classification models, standard metrics were precision, recall, F1-score, and accuracy between estimated categories and true categories. For regressions, we calculated the correlation between the estimates and true values; this would ideally form a line of slope 1. We also included standard metrics such as RMSE, MSE, and relative MSE.

The pipeline comparison methods were primarily relative, as the objective was to optimize the performance of a data fusion pipeline. The true values used for the evaluation were derived from the test scores, although these values were inherently subject to standard measurement errors. To illustrate specific examples, the performance of the Dog Collar dataset was impacted by poor sensor fitting and attachment, with ergonomic challenges exacerbated by the presence of fur [41]. In the Moth dataset [39], significant errors remained due to both missed detections and false positives, the latter often caused by moth fluttering behavior as they entered the trap. The weather sensor also contributed to random noise and spatial interpolation of field data introduced an additional modeling error. The Plant Impedance dataset [12] was highly sensitive to plant hydration levels and local weather conditions. Electrode orientation could subtly affect measurements. Additionally, the plant’s immune response may have increased impedance by covering or attacking the electrodes. Lastly, the Sweet Potato dataset relied heavily on image data, making it susceptible to image quality issues [40]. Additionally, potato orientation and accurately determining the main axes posed further challenges during image analysis.

As is common in animal welfare and agriculture studies, our aim was to understand the trade-off between model fit and computational complexity. We monitored time and space complexity using the $time$ and $sys$ Python packages, respectively, and present these metrics as well.

Lastly, we evaluated the usefulness of the DFE as a tool by its simplified capability for the user. We achieved this by estimating the number of lines of code required to complete the pipeline. We compared the lines of code required with and without the DFE tool. Lines used exceeding those required using the DFE module were considered superfluous.

4.1. Dog Collar Dataset: Singlets to Classification

The Dog Collar dataset had three pipelines, one that performed low-level fusion (P1), another that performed mid-level fusion using dimension reduction followed by classification (P2), and a third that performed dimension reduction and classification simultaneously (P3) (Figure 3). A particularity of this dataset was the pre-alignment. With the many different modalities of different sampling rates, we interpolated the datapoints to a common time axis, and then concatenated each sensor as a separate feature. This was a classification model, and the model fit scores for each pipeline are presented in Figure 7. We see that P3 performs overwhelmingly better than P1 and P2 in all metrics, suggesting that LDA was the best choice for this classification dataset.

The computational complexity of time and space is shown in Figure 8a. Here, we see that all pipelines perform roughly the same in both time and space complexity. With its additional significantly higher accuracy (Figure 7), there was no reason not to use P3.

Lastly, the level of code simplification provided by the DFE is shown in Figure 9. All pipelines had a drastic reduction in the number of code lines required by about three times. The simpler design of P1 required about 40% of the amount of code as the other pipelines to begin with, but it was still dramatically improved with the use of DFE.

4.2. Moth Detector Dataset: Singlets to Regression

The Moth Detector dataset had three pipelines, all of which varied in window scheme for temporal feature extraction (Figure 4). The window variation (5 min or 20 min or 1 h) was used to test the temporal resolution of the data; if the resolution was poor, then longer window sizes should not have affected the model. All windowing had a 50% overlap, but P1 used a short window of 5 min, P2 used a medium window of 20 min, and P3 a long window at 1 h. This was a regression model, and the model fit scores for each pipeline are presented in Figure 10. We see that all three pipelines performed approximately equally in both R and RAE. R values close to 1 indicated a strong correlation between the predictions and the test values. However, the RMSE and MAE were not informative in this case as they yielded zero for all pipelines. The relative metric RAE revealed overall similarity between the prediction plots, while RelRMSE showed that the P3 pipeline had a higher relative error compared to P1 and P2. This suggests that P3 may have included several outliers that contributed to the increased RelRMSE—an effect that was less apparent in the RAE results. Overall, this suggests that the window length was largely irrelevant, at least up to 1 h. This dataset was also unique for its use of Kernel Density Estimation (KDE) before temporal alignment.

The computational time and space complexity are shown in Figure 8b. Here, we see that pipeline P1 used about twice as much time and space as the other pipelines and that pipeline P2 used more time than P3. This is likely because the short windowing scheme in P1 generated the greatest number of features, followed by the windowing scheme in P2, then P3. P3 used slightly less space as expected. However, space usage was similar between P2 and P3. We hypothesize that this is due to a similar backend array initialization process that sets a standard minimal storage size. Since the accuracy of P3 is on par with P1, there is no reason not to use P3.

Lastly, the level of code simplification provided by the DFE is shown in Figure 9. All pipelines have about a 5-fold reduction in lines of code compared to Vanilla Python. All three pipelines used the same amount of code because the window design was a simple change in a single for-loop parameter.

4.3. Plant Impedance Dataset: Arrays to Regression

The Plant Impedance dataset had three pipelines (P1, P2, and P3), varying the order of dimensional reduction, feature extraction, and linear regression (Figure 5). This was a regression model, and the model fit scores for each pipeline are presented in Figure 10. We saw that R values were close to 1 across all pipelines, indicating a strong correlation between the predictions and test values. The RMSE and MAE identified pipeline P2 as having the highest error, followed by P3, with P1 performing the best. In contrast, the relative metrics (RelRMSE and RAE) were not effective in this case, as they yielded zero for most pipelines. Overall, these results suggest that P1 achieved the best fit, indicating that reducing dimensionality prior to feature extraction was an effective approach for this dataset. Computational complexity (Figure 8c) shows many differences between these pipelines. All three performed within subseconds, but P1 was the fastest. This is likely because it began with dimensional reduction, lowering the number of later calculations. However, P1 also used about twice the space requirements of the other two pipelines. The level of code simplification is shown in Figure 9. All three pipelines had similar numbers of lines of code, and all three were reduced three-fold in comparison to Vanilla Python.

4.4. Sweet Potato Dataset: Images to Classification

The Sweet Potato dataset had two pipelines (P1, P2), varying the dimension reduction process (Figure 6). This was a classification model, and the model fit scores for each pipeline are presented in Figure 7. We see that the two perform equally well, suggesting that these differences were irrelevant for fit purposes. Computational complexity analysis (Figure 8d) showed that time usage was almost identical but that pipeline P2 used lower space requirements. The level of code simplification is shown in Figure 9. Both pipelines had similar user input requirements and produced a reduction of about 50%. The modest improvement was primarily due to the brevity of the script, as the initial dataset had already undergone extensive preprocessing. Temporal and spatial alignment were therefore not required.

5. Discussion

In the Dog Collar dataset, the three pipelines demonstrated significantly better results with mid-level LDA-based fusion compared to the other two schemes. The time and space requirements showed similar results within all pipelines tested, so the LDA-based fusion pipeline was overall better at extracting the information from the data. The DFE tool was able to quickly identify this.

The Moth Detector dataset evaluated the effects of temporal feature windowing on moth density prediction using environmental data. We tested window lengths of 5 min, 20 min, and 1 h, with environmental data collected approximately once per minute. We concluded that the longest window length (1 h) was the best pipeline due to its lowest time complexity. However, the 20-min window also performed similarly, so either setup is a reasonable choice. Additionally, this information could help adjust the environmental sampling rate to align more closely with the theoretical Nyquist sampling rate for weather data, potentially reducing upstream data-space requirements.

With the Plant Impedance dataset, we investigated the effects of the order of dimensional reduction, feature extraction, and classification. Starting with dimensional reduction (P1) performed best. However, pipeline P1 incurred approximately twice the space costs of the other pipelines, which is an important consideration for certain applications. For example, for a space-sensitive IoT device, pipeline P3 would be more preferable.

In the Sweet Potato dataset, we compared the performance of ICA (pipeline P1) or PCA (pipeline P2) with a Naïve Bayes classifier. We hypothesized that ICA would yield better performance, as it produces statistically independent components, an assumption that aligns well with the core assumption of the Naïve Bayes model. In fact, the ICA dimensionality reduction pipeline outperformed PCA, achieving 69% accuracy, which was 6% higher than that of PCA. To assess whether this performance difference was statistically significant, we repeated each pipeline 30 times and applied a one-tailed paired t-test to the resulting accuracy scores. The analysis confirmed that ICA significantly outperformed PCA, with a p-value of 0.041. We attribute this improvement to the alignment between ICA’s independence objective and the Naïve Bayes assumption of feature independence. This suggests that pipeline P1 was likely a better engineered solution. Additionally, the Sweet Potato dataset contained high levels of redundancy, with many measurements collected along a central axis. It was plausible that PCA, focusing on variance rather than independence, discarded some seemingly redundant but informative components that ICA was able to retain.

In almost every dataset, a reduction of at least a 60% in code length was observed, with reductions of 70–80% being common. This indicated that the tool was a valuable module for setting up and evaluating data pipeline flows. This open-source Python package is expected to simplify the required coding skills for users, particularly researchers working on early-stage studies on animal and plant monitoring. Excluding the need for custom dataset formatting, the average number of lines of code for a pipeline using the DFE was 15, compared to 52 lines when the tool was not used.

These four scenarios were intentionally diverse, representing a broad range of tasks typically encountered by beginner modelers in the fields of agriculture or animal welfare. Our data fusion framework successfully accommodated all four use cases, offering multiple pipeline options for each. This demonstrated both the versatility and practical utility of the proposed pipeline methodology. In its current form, the DFE serves as a a minimum viable product. It provides a solid foundation for future development as we continue to identify and integrate new, application-driven features. It is a fact that these four examples may not be comprehensively covering the full spectrum of data fusion scenarios likely to emerge in the future, and that generalizability limitations may exist. Nonetheless, our framework’s flexible and extensible architecture is well positioned for future enhancements and adaptations as new requirements potentially arise.

Specifically for our DFE tool, we included a standard user guide for maintenance and quick start purposes. While the function names are designed to be intuitive, they do make assumptions and may require further study by researchers to ensure appropriate use depending on the specific application.

Designed for early-stage pipeline development, the DFE was not intended for large-scale projects, nor has it been evaluated on such scales. However, there are several ways to scale up. For example, when datasets involve multiple subjects or arrays, parallelization can be easily implemented. In a cluster, each subject could be assigned to its own node, where calculations and dimensionality reduction can be performed separately and then aggregated for larger datasets. Performing varying levels of reduction a priori can make the size of the aggregated dataset more manageable. Additionally, as demonstrated in the four test cases, users can identify pipelines that may not scale well in terms of time or space and explore alternative methods for these cases. Users can also conduct tests to determine whether scaling up is advisable. By combining data at lower levels, applying data reduction techniques, and then aggregating, users can confirm whether such groupings yield similar results. Alternatively, intermediate data can be extracted or processed manually for further scaling, while pipeline design may still provide valuable insights.

6. Future Work and Conclusions

The development of user-friendly multisensor data fusion tools is a pressing need and an open challenge within the precision agriculture field, particularly when it involves plant and animal data. Due to the complexity of physical, environmental, and biological phenomena, data fusion approaches often benefit from iterative design processes. Since multisensor data fusion pipelines involve numerous operations and are highly versatile, modular frameworks that enable rapid prototyping of different pipeline configurations are essential. We proposed such a framework along with an open-source tool called the Data Fusion Explorer (DFE). This is the first generalized data fusion platform with a unique design framework specifically tailored for animal monitoring and agricultural applications. In this article, we introduce this framework to the research community engaged in early-stage animal and plant monitoring research. We also demonstrated the applicability of this framework to four early-stage datasets, which span various disciplines, data types (singlet, array, image), objectives (classification or regression), and distributions (spatial or temporal). For each dataset, we derived comparison metrics for the customized pipeline, comparing these with traditional regression and categorical metrics. Using the DFE, we observed a significant reduction (averaging over 50% less written lines) in coding requirements for users across almost all datasets, highlighting the tool’s potential to simplify workflow for common users.

The data fusion framework is a work in progress, and we anticipate that future updates will introduce new functionalities as emerging needs are identified. The framework was evaluated using custom-designed pipelines applied to the four datasets available to the authors. As the framework is adopted more broadly, we anticipate that additional, commonly encountered use cases will emerge, informed by the evolving needs of the community. While future versions of the DFE tool may include new modules, the core framework is expected to remain consistent. Looking ahead, GitHub engagement metrics, such as stars and pull requests, will eventually provide insight into community interest and serve as proxies for the framework’s usefulness. However, since this is a newly released package, such metrics are not yet available.

Planned enhancements for the DFE tool include built-in space and time-complexity analysis using decorator functions and new feature extraction methods. Examples include Fourier and Wavelet transforms, as well as user-defined feature functions. Once an optimal pipeline is identified, parallelization strategies may be implemented to enable large-scale deployments and improve efficiency. We recognize the potential to further abstract the DFE’s pipelines into a unified pipeline object that accepts processing functions along with their parameters. For example, the short window Moth pipeline (P1) could be expressed as: P1 = pipeline(moth_dataset, [{temporal_features, 5, “min”}, {PCA, 20}, {linear_regression}]). Such an abstraction would significantly streamline the user experience and reduce code overhead, making the tool more intuitive and scalable for diverse applications. The DFE has also been designed with dependencies on well-established packages like NumPy, pandas, and Scikit-learn. Future enhancements could facilitate easier integration with these libraries along with easy exporting to common file formats. Lastly, we may expand support to include data input formats like streaming datasets or time series-specific data.

A particularly interesting future application of the framework lies in its potential use with Rashomon sets [42]. These are collections of models that achieve near-equivalent prediction accuracy. When multiple distinct models yield similar results, the existence of a large Rashomon set becomes plausible [42]. In such cases, the likelihood of identifying a simpler and more interpretable model increase, aiding in model transparency and usability. The DFE framework facilitates rapid prototyping across diverse learning strategies, enabling the users the identify simpler models when performance is comparable. In such cases, the simpler model is preferred, or users may further explore even more minimal pipelines.