Synergizing Crop Growth Models and Digital Phenotyping: The Design of a Cost-Effective Internet of Things-Based Sensing Network ^†

Abstract

Plant-soil sensing devices coupled with Artificial Intelligence autonomously collect and process in situ plant phenotypic data. A challenge of this approach is the limited incorporation of phenotype data into decision support systems designed to harness agricultural practices and forecast plant behavior within the intricate context of genotype, environment, and management interactions (G × E × M). To enhance the role of digital phenotyping in supporting Precision Agriculture, this paper proposes a sensing network based on the Internet of Things. The developed system comprises three modules: data collection, communication, and a cloud server. Several processes co-occur in the server, namely data visualization to confirm the correct sensors and data stream functioning. In addition, a crop growth model (CGM) runs on the server, which is powered by the collected data. The simulations generated by the model will support agricultural decisions, obtaining, in advance, insights about plant behavior considering several G × E × M scenarios. To assess the performance of the proposed network to provide reliable data to the model, a greenhouse was equipped with several sensors that collect plant, environment, and soil data (e.g., leaf numbers, air temperature, soil moisture). The proposed network can provide real-time causal support for advanced agricultural practices, evolving from a data-driven approach to an integrative framework where context (G × E × M) drives decision making.

1. Introduction

Precision Agriculture (PA) based on the continuous monitoring of plant growth is of paramount importance. It involves taking into consideration the profound impact that environmental conditions and agricultural management practices can exert on the performance of a specific genotype (G × E × M). This understanding forms the foundation for crafting robust decision support systems (DSSs) aimed at optimizing input applications and bolstering crop yields, profitability, and the environment [1]. Digital phenotyping (DP) is a cutting-edge application that combines advanced sensing devices (e.g., RGB/hyperspectral cameras) and data analysis techniques (e.g., Artificial Intelligence (AI)) to diagnose plant phenotypic traits (i.e., observable plant traits resulting from the performance of a genotype in a specific environment), namely morphological [2], physiological [3], and phenological [4] traits related to growth, health, and development [5]. Most of the literature describes high-throughput phenotyping facilities that analyze model plants in expansive laboratory conditions (e.g., [6]), while low-cost field applications are limited [7]. Nevertheless, DP data can be analyzed to identify trends and relations between phenotypes and G × E × M conditions, enabling more knowledgeable agronomic decisions.

Autonomous sensing systems such as robots and drones represent a great advancement in the realm of data collection for field phenotyping, offering remarkable improvements in terms of speed, repeatability, and accuracy [8]. However, beyond the technical challenges like localization and path planning, there exist critical constraints related to data management and analysis. Given the diverse array of phenotypic data sources and the complexity of the spatiotemporal scales involved, it becomes imperative to develop robust data management techniques that not only preserve data relevance but also facilitate easy access and analysis [6].

Therefore, the establishment of resilient sensing networks is paramount to comprehensively characterize prevailing environmental conditions and seamlessly link them to the collected phenotypic data. In this context, it is essential to accompany phenotypic data with metadata, thereby promoting their reuse and ensuring interoperability in contexts distinct from their original acquisition [9,10].

Regarding data analysis, although DP uses advanced AI techniques that establish genotype–phenotype relationships within G × E × M interactions [11,12], it has constraints depicting the dynamics of these relationships. Some progress has been made in combining DP and process-based models, optimizing data analysis through multi-scale frameworks. Process-based models (a group of crop growth models (CGMs)) simulate plant growth and predict crop yield through differential equations that consider the mechanistic understanding of how a plant grows [13]. In this way, fundamental processes and their interactions over time are represented (e.g., nutrient cycling, water fluxes). Thus, it is possible to assess the crop’s behavior in future climate and management scenarios, improving decision making [14,15].

A process-based model can extract relevant traits using knowledge in advance, simplifying the actual analysis systems (AI-based) [16,17,18]. Furthermore, DP can be integrated into a process-based model to estimate unknown parameters, replacing its subroutines and describing complex processes (e.g., nitrogen dynamics [19]).

Yet, few studies present joint approaches, barely integrating phenotype data in advanced DSSs [10]. To overcome this shortcoming, we propose a sensing network based on the Internet of Things (IoT). The network comprises three modules: data collection, communication, and data management/analysis. The aim is to test the feasibility of cost-effective sensors to collect high-throughput phenotypic and environmental data, establish methods that guarantee data relevance and interoperability, and integrate data into a CGM. Thus, a continuous swap of data will be created between the physical entities and the simulated ones. This digital twin [20] approach can provide real-time, spatiotemporal causal support for advanced PA practices, evolving from a data-driven approach to an integrative framework, where G × E × M conditions are the driver of advanced decision making.

2. Methods

Figure 1 describes the overall architecture of the proposed sensing network.

To allow the network to be versatile, given the diversity of data sources, it is proposed that a microprocessor be used to ensure uniform data transfer, regardless of the sensor’s intrinsic communication protocol. In order to ensure robust spatiotemporal communication that can be transferred to an agricultural environment, the connection between sensors and the microcontroller and from this to the microprocessor must be physical (e.g., USB). The role of the microprocessor is to ensure the transfer of data to the server. In this case, the transfer must be wireless (e.g., Wi-Fi).

On the server, the information is routed to its proper destination via the communication broker. This is connected to the visual interface, allowing data visualization in real time. It is also connected to the programming interface, which allows the conditional execution of scripts; this results in actions such as sending data to the database or activating the CGM.

The programming interface must ensure that the data received are matched by the relevant metadata. It must also deploy the appropriate processing operations. In this case, numerical data can be distinguished from non-numerical data. While the former can be sent directly to the intended destination, the latter must be processed in order to extract information from the raw data. For example, to extract phenotypic traits from images, classic techniques (e.g., color thresholding) or more complex ones (e.g., Deep Learning models) must be applied.

3. Results and Discussion

To test the proposed network a sensing network was installed in a greenhouse at INESC TEC headquarters in Porto, Portugal. Figure 2 depicts the installation.

Stationary sensors were in charge of collecting environmental parameters (e.g., air temperature), phenotypic traits (e.g., actual evapotranspiration), and soil parameters (e.g., moisture). The choice of devices was based on cost-effective commercial solutions compatible with the remaining network’s components. Furthermore, some devices were developed from scratch, namely a weighing lysimeter (Figure 3).

All the sensors share a common feature: they are connected to custom hardware based on the RP2040 microcontroller, which allows the signals to be processed from the sensors’ intrinsic protocol to the CAN protocol. This protocol was chosen because it applies differential communication, which minimizes noise in the signal and allows for a longer range between connections, which is a must in agricultural environments. The sensors’ microcontrollers, “slaves”, are connected to another microcontroller, the “master”. This, in turn, is connected via USB to a Raspberry Pi Zero W, which sends data requests to the “master” microcontroller that distributes them to the respective “slaves”. The Raspberry is also connected to a camera (Raspberry Pi Camera) for imaging operations. The data received by the Raspberry are sent to the server via Wi-Fi, according to the MQTT (Message Queuing Telemetry Tracking) publish–subscribe protocol.

The greenhouse was also equipped with robotics-assisted sensors. PixelCropRobot, a mobile cartesian robot designed for phenotyping operations [21,22], was implemented for autonomous phenotypic data collection. In addition to 2D RGB imaging operations, the robot is equipped with a custom multispectral sensor and a LiDAR that allows the measurement of leaf pigments—related to the physiological response to abiotic stresses—and the canopy characterization, respectively. The robot is equipped with a Raspberry Pi 4 and, as mentioned above, the data are sent to the server via Wi-Fi, according to the MQTT protocol.

This means that in both cases, the Raspberry Pi acts as a client and sends the messages to the MQTT broker, which filters the messages by topic and distributes them to the corresponding subscribers, which are defined in the scripts of the programming interface or in the functions of the visual interface. By default, all the data received by the broker are subscribed to a Python script that combines the relevant metadata, according to the metadata guidelines of the DEMETER-AIM ontology, and then forwards them to the database.

The visual interface was developed using Node-RED (Figure 4). To ease real-time data visualization (e.g., air temperature), some functions of the visual interface act as subscribers, directly receiving the corresponding messages from the broker. Furthermore, through this interface, it is possible to retrieve historical data (stored in the database) and trigger the CGM.

The dynamic process-based model STICS (Simulateur mulTIdiscplinaire pour les Cultures Standard) [23] was the chosen CGM. STICS is a daily time-step model with input variables relating to soil, climate, and the cropping system. The model simulates the growth of a defined genotype for which a physical medium and a crop management schedule are defined. This model presents some features that fit with the sensing network designed, namely its generality, robustness, and modularity, enabling its application to a wide range of crops, climate conditions (even several ones), and the design of new modules or functions, complementing the model.

To ensure that the proposed network provides reliable data to run STICS, continuous data collection was monitored during a lettuce growing season (42 days), according to the frequencies shown in

Table 1. Characterization of the data collected by the sensing network during the lettuce growing season.

Sensor	Quantity (n)	Daily Requests (n)	Average Size
Stationary
RPi Camera	1	24	10 MB
AS7341	2	24	400 B
HTU21D	2	24	170 B
SEN0159	1	24	120 B
Lysimeter	12	24	160 B
SEN0308	12	24	170 B
PixelCropRobot
RPi Camera	1	5	10 MB
Multispectral sensor	1	5	400 B
LiDAR	1	5	370 B

.

Given the daily time-step of STICS, it is likely that the dataflow shown in Table 1 is enough to run the simulations. However, losses were detected during data transfer to the server. These did not exceed 5% and were mainly due to interruptions in the Wi-Fi connection. Although these are significant losses, since the aim is to keep the model online continuously, they can be easily addressed. For example, one can reinforce the Wi-Fi connection or create a local database that stores the data in the event of Wi-Fi interruptions. In line with Droutsas et al. [24], who proposed the integration of machine learning models into a process-based model, the described network aims to enhance actual data analysis systems and reduce modeling fine-tuning processes. Although further tests are needed, the proposed sensing network has the potential to overcome the phenotyping pitfalls identified by Saint-Cast et al. [10], namely the lack of common semantics and thorough data exchange platforms.

4. Conclusions

This article presents an IoT-based sensing network for digital phenotyping. Associated with this network, a DSS was developed, based on a CGM with the purpose of optimizing agricultural practices. However, further testing is needed to validate the network when fully working under real field conditions. In the future, we intend to enhance the capabilities of this approach. The model simulations will support decision rules, processed by an actuator that will carry out a specific operation. Thus, a continuous swap of data will be created between the physical entities and the simulated ones. This digital twin approach will provide real-time, spatiotemporal causal support for advanced Precision Agriculture practices, evolving from a data-driven approach to an integrative framework, where G × E × M conditions are the driver of advanced decision making.