Computer Vision-Based Monitoring and Data Integration in a Multi-Trophic Controlled-Environment Agriculture Demonstrator

Abstract

Controlled-environment agriculture (CEA) and circular production systems require coordinated monitoring of biological and physicochemical processes across trophic levels. This project report presents the implementation of a multi-trophic controlled-environment agriculture demonstrator that integrates computer-vision-based monitoring with established sensor infrastructure for aquaculture, poultry, plants, microalgae, duckweed, and insect modules. Stereo imaging and RGB-D systems are deployed for non-invasive quantification of fish biomass and plant growth, while continuous water-quality and environmental measurements (e.g., pH, dissolved oxygen, nitrate, ammonium, temperature, CO₂) provide complementary process data. These data streams are synchronized within a shared database architecture to enable cross-module evaluation of nutrient dynamics, growth progression, and operational stability under real facility conditions. The implemented framework demonstrates how computer vision can extend conventional sensor-based monitoring by directly capturing biological performance indicators across aquatic, terrestrial, and microbial domains. While advanced predictive modeling and full digital twin simulation remain future development steps, the realized data-integration architecture establishes a structural foundation for the systematic evaluation of circular indoor food-production systems. The demonstrator illustrates how multimodal monitoring can support nutrient recirculation, transparency of biological variability, and data-driven assessment within controlled multi-trophic environments.

1. Introduction

In the face of climate change, water scarcity, and rising global food demand, agricultural systems must shift toward resource-efficient closed-loop processes [1]. Controlled environment agriculture (CEA) in urban settings offers a particularly promising pathway, enabling minimal resource losses and stable yields under carefully regulated conditions [2,3,4]. CEA offers resilient food production but remains limited by high energy use and costs, requiring transdisciplinary innovation that integrates life-cycle analysis, digitalization, flexible energy management, and engineered biological systems to enhance sustainability [5,6,7]. Key to sustainable systems are the recycling of nutrients, the recovery of water, and the stabilization of microbial processes that drive decomposition, nutrient mobilization, and resilience [8].

The idea of coupling animal and plant production has a long tradition, ranging from manure fertilization to integrated crop–livestock systems. Today, novel approaches such as aquaponics combine aquaculture and hydroponics to recycle fish excrement into nutrients for plants, while plants purify the water for fish [9,10]. Beyond plant–fish coupling, additional trophic interfaces enable the recovery of gaseous and dissolved emissions. Microalgae and other photosynthetic organisms can capture CO₂ and NH₃ from animal exhaust and convert them into valuable biomass [8]. In previous work, a photobioreactor was physically connected to a poultry housing unit to cultivate Arthrospira platensis (Spirulina) using exhaust air as a substrate, effectively purifying the air while producing high-yield algal biomass [11]. Similarly, duckweeds (family Lemnaceae) represent highly efficient nutrient sinks in aquatic environments. Among the fastest-growing angiosperms, they absorb nitrogen, phosphorus, and trace elements from residual process water, thereby improving water quality while generating protein-rich biomass suitable for feed or as a substrate for insect cultivation [12]. Insects such as black soldier fly larvae (BSFL) add a further trophic level by efficiently upcycling organic residues into protein-rich feed ingredients [13]. Together, these interconnected biological modules form the basis for multi-trophic circular production systems in which solid, liquid, and gaseous emissions are systematically valorised across trophic levels.

These elements converge in the vision of a highly integrated closed-loop CEA, schematically illustrated in Figure 1. Solid, liquid, and gaseous emissions from livestock are upgraded by microalgae, duckweed, and insects into high-value biomass that can be used as feed for aquaculture and terrestrial livestock. Aquaculture produces both food and residual products, which serve as fertilizer for vegetable farming under aquaponic principles. In turn, vegetables provide food for humans and residuals for animal feed. Powered by renewable energy, the recirculation system enables optimal reuse of nutrients and, above all, water, while minimizing environmentally harmful emissions.

Realizing such a complex system requires more than biological integration: it depends on advanced process control and comprehensive analytical technologies. Here, computer vision emerges as a key enabler. By providing non-invasive, real-time monitoring of plant and animal growth and health, computer vision supports machine-assisted control of closed-loop agriculture. Integrated with automation and digital manufacturing, these methods can stabilize community dynamics, anticipate perturbations, and increase the resilience of circular food production.

In this project report, we document the design, implementation, and integration of computer-vision-based monitoring systems within a multi-trophic circular indoor agriculture demonstrator. Building on the concept illustrated in Figure 1, we demonstrate the technical feasibility of coupling multi-trophic biomass production with advanced monitoring technologies under operational conditions. The presented system architecture illustrates how digital monitoring can be integrated into controlled-environment agriculture to support resource-efficient and resilient production processes.

The scope of this contribution is explicitly limited to controlled-environment agricultural systems, in which biological processes are operated within technologically regulated infrastructures. The framework presented here is not intended to replicate open-field agriculture but to enhance resilience, circularity, and resource efficiency in indoor or protected cultivation systems. Such controlled environments enable systematic integration of sensing, automation, and digital twin methodologies while decoupling production from external climatic variability.

The work presented in this manuscript is based on a demonstrator-scale circular indoor agriculture facility established to explore integrated closed-loop production concepts under operational conditions. Specific research components, particularly the integration of multiple trophic levels and the development of data-driven monitoring and fusion methodologies, were investigated within publicly funded research projects, such as UrbanAqua and Amigem [14,15]. These projects focus on implementing multi-trophic biomass production, combined with advanced digital monitoring and data integration strategies, in a demonstrator-scale facility.

The demonstrator facility is being developed as an iterative and staged innovation infrastructure, in which biological modules and digital monitoring components are progressively integrated, evaluated, and refined under operational conditions. Accordingly, this project report documents both the implemented system components realized to date and the architectural framework guiding their integration and further expansion.

Despite rapid advances in individual monitoring technologies, integrated frameworks that systematically couple computer vision, multimodal sensing, and digital twin methodologies across multiple biological modules within a unified circular production system remain underexplored. The central objective of the present project is, therefore, to implement and demonstrate such an integrated digital architecture, addressing the following goals:

(i)

deployment of multimodal monitoring across aquaculture, poultry, plant production, algae, and insect modules;

(ii)

establishment of a unified data infrastructure enabling hierarchical data fusion and digital twin integration;

(iii)

exploration of system-level interoperability, scalability pathways, and modular retrofit strategies within controlled-environment settings.

2. General Aspects of Computer Vision

Human vision is an adaptive, context-driven system that has evolved to interpret complex and dynamic environments. It continuously combines sensory input from the retina with higher-level processes such as memory, expectation, and context [16,17]. This interaction between bottom-up perception and top-down cognition enables humans to recognize objects even under challenging conditions, such as poor illumination, occlusion, or distortion. Human vision integrates multiple cues—color, motion, depth, and texture—to extract meaning from scenes and infer relationships between objects. It is inherently robust, flexible, and guided by attention and feedback loops that link perception to action [18].

Computer vision (CV), in contrast, is a data-driven technological counterpart designed to replicate selected aspects of human perception. CV systems acquire optical information using digital sensors and process these data using mathematical models and machine-learning algorithms, particularly convolutional neural networks and transformer architectures [19]. While such models can perform extremely well in tasks like object detection, classification, or segmentation, they still lack the general understanding and interpretive flexibility characteristic of biological vision [20]. Figure 2 illustrates this contrast: while human perception relies on contextual reasoning and prior experience, computer vision systems convert raw sensor data into statistical predictions, typically without awareness of broader environmental meaning.

Human vision is also highly context-sensitive. Scene interpretation influences object recognition, and vice versa [21]. Depth perception in humans results from integrating binocular disparity with monocular cues such as motion parallax, occlusion, and texture gradients [18]. Most CV systems, however, process each image independently, without linking it to semantic context or prior knowledge. This limitation can reduce generalization and reliability when environmental conditions differ from those seen during training [20]. Figure 3 summarizes common strategies for visual interpretation in computer vision. (a) Area-based methods, such as semantic segmentation, classify every pixel in an image to create spatial maps of relevant regions—for instance, distinguishing foliage from background. (b) Object-based methods detect individual items using bounding boxes or keypoints, allowing both object identification and structural localization. These complementary techniques underpin most agricultural applications of computer vision, from growth monitoring to disease detection in CEA settings.

Another central challenge lies in depth estimation and three-dimensional reconstruction. Humans perceive depth by integrating geometric and contextual cues, whereas CV systems typically rely on geometric triangulation. Figure 4 shows the principle of stereo vision: two cameras capture an object from slightly different viewpoints, and the disparity between corresponding pixels allows the estimation of its three-dimensional position. While this approach yields accurate geometric reconstructions, it lacks the contextual understanding that humans apply to infer depth under uncertain or incomplete visual conditions [17]. Recent deep-learning models, such as RAFT-Stereo and monocular depth estimation networks, have advanced geometric reasoning, yet complex or dynamic scenes remain difficult to interpret reliably.

To narrow the gap between human and artificial perception, current research increasingly integrates mechanisms inspired by cognitive processes. Recurrent and attention-based architectures emulate selective focus and iterative refinement during visual processing [22]. Multi-task learning allows simultaneous object detection, segmentation, and classification, resembling the integrative character of human perception [19]. Reinforcement learning introduces elements of active exploration, similar to human eye movement and attentional strategies [23]. Moreover, self-supervised and few-shot learning approaches help reduce dependence on large annotated datasets, improving adaptability to novel environments.

These developments move computer vision toward greater robustness and contextual awareness. Within sustainable food production systems, such capabilities enable non-invasive, real-time monitoring of biological processes and support adaptive control strategies in controlled agricultural environments. By enhancing perception, decision-making, and automation, computer vision forms a cornerstone of intelligent, resource-efficient circular agriculture.

Building on the closed-loop framework introduced in Figure 1, the following sections examine the implementation of computer vision across the individual trophic modules of the system. The focus lies on application-specific monitoring and control strategies within aquaculture, livestock production, plant cultivation, and biological upconversion.

We therefore present representative computer vision approaches for each major trophic level, beginning with aquaculture, followed by poultry production and vertical plant farming, and concluding with microalgae, duckweed, and insect bioconversion as key nutrient-recovery modules.

3. Computer Vision for Indoor Aquaculture

Aquaculture plays a central role in the closed-loop system shown in Figure 1, linking nutrient recovery, feed production, and protein generation. Monitoring fish health, growth, and behavior is essential for production efficiency, animal welfare, and environmental stability. Conventional monitoring relies on manual sampling, which may induce stress and limit temporal resolution. Recent advances in computer vision (CV) provide non-invasive alternatives for real-time monitoring in aquatic systems [24].

Deep-learning-based CV methods enable automated detection, tracking, and phenotypic analysis of fish under challenging conditions such as high stocking densities and dynamic lighting [25]. Core functional domains include detection, counting, behavior analysis, and health assessment [26]. Vision-based feeding analysis supports quantitative evaluation of feeding responses and feed management [27]. Three-dimensional (3D) stereo vision is particularly relevant for biomass estimation. By combining geometric reconstruction with learning-based segmentation, surface reconstruction and weight estimation can be achieved [28,29]. These methodological advances inform the implemented monitoring pipeline in the demonstrator.

Within our closed-loop demonstrator facility, CV is implemented in an integrated recirculating aquaculture module. The system combines stereo imaging, depth sensing, and water-quality analytics to support continuous biomass estimation and growth assessment under operational conditions. Commercial aquaculture environments remain challenging due to turbidity, occlusion, and illumination variability [24,25], and standardization across systems is limited [26]. In the present facility, CV-derived biomass metrics are synchronized with water-quality parameters within a shared data infrastructure to support operational monitoring.

Water-quality monitoring within the recirculating aquaculture system includes continuous measurement of pH, dissolved oxygen, and electrical conductivity (EC) at 1-min intervals using integrated in-line sensors. Additional chemical parameters, including nitrate (NO₃⁻), ammonium (NH₄⁺), chlorine, and alkalinity, are assessed using colorimetric and photometric test systems (e.g., Hanna Instruments) at a sampling frequency of once per day. This interval reflects the comparatively slow response dynamics of the RAS configuration. Water exchange rates and feeding rates are documented and evaluated in relation to image-based growth curves derived from stereo vision analysis. Nutrient concentrations are recorded in parallel with stereo-derived biomass metrics to document growth dynamics under operational conditions. Automation of sampling procedures and the integration of ion-selective electrodes for extended temporal resolution are currently under development.

Within this setup, fish populations are continuously observed to estimate biomass and track growth under operational conditions. Figure 5 illustrates the implemented workflow for live fish weight estimation, from underwater image acquisition to length and weight computation.

Figure 5. System pipeline for live fish weight estimation in aquaculture. RGB images and depth maps are captured using a stereo camera. Synchronized streams are transmitted to an edge device for preprocessing and an inference server for main processing. A custom keypoint detection model identifies anatomical landmarks, which are combined with depth data for 3D length and weight estimation. Detailed inference and aggregation stages are illustrated in Figure 6 and Figure 7.

Figure 5. System pipeline for live fish weight estimation in aquaculture. RGB images and depth maps are captured using a stereo camera. Synchronized streams are transmitted to an edge device for preprocessing and an inference server for main processing. A custom keypoint detection model identifies anatomical landmarks, which are combined with depth data for 3D length and weight estimation. Detailed inference and aggregation stages are illustrated in Figure 6 and Figure 7.

Images are acquired using a consumer-grade stereo camera in a waterproof housing, with intrinsic camera parameters calibrated in submerged conditions using a chessboard target to correct for underwater refraction effects. Synchronized RGB and depth streams are transmitted via USB to a local edge device for preprocessing and subsequently via Ethernet to an inference server, where lengths and weights are estimated in real time, aggregated, and transferred to a cloud database.

Figure 6 provides a detailed view of the inference sequence used to generate validated length and weight estimates for every detection. Applying a two-stage inference pipeline, an object detection model first locates all visible fish using bounding boxes, followed by a keypoint detection model trained on a dataset spanning multiple growth stages that identifies anatomical landmarks, such as the nose, tail base, tail tip, and eye of each individual. Combining two-dimensional keypoints with corresponding depth data and camera parameters enables three-dimensional reconstruction and length estimation, and results are validated for detection confidence and geometric plausibility.

Detected individuals are assigned unique identifiers and tracked across frames, allowing the system to reconstruct multiple views of the same fish over time. Length estimates are averaged across these multiple views to mitigate minor keypoint localization errors, depth noise, and body curvature effects during swimming. Statistical outliers are automatically detected and removed before averaging, further improving robustness. Individual weight is then derived from length using an allometric growth formula. The validated measurements are subsequently aggregated to produce weight-distribution histograms that form the basis of the growth curves presented in Figure 8.

Figure 6. Inference pipeline for estimating fish weight distribution. Synchronized RGB and depth streams are processed through an object detection model, producing bounding boxes, followed by a keypoint detection model identifying anatomical landmarks for 3D reconstruction. Fish are tracked across frames, and validated length and weight estimates are aggregated into weight distributions.

Figure 6. Inference pipeline for estimating fish weight distribution. Synchronized RGB and depth streams are processed through an object detection model, producing bounding boxes, followed by a keypoint detection model identifying anatomical landmarks for 3D reconstruction. Fish are tracked across frames, and validated length and weight estimates are aggregated into weight distributions.

A representative example for validating the length-estimation pipeline on a single isolated fish is shown in Figure 7, demonstrating correct 3D reconstruction under idealized visibility conditions before applying the method at the population scale. The figure highlights the sequential reconstruction steps from RGB and depth acquisition to three-dimensional geometry extraction. The application of this approach to entire fish populations is illustrated in Figure 8, showing growth curves for three cultivation systems over a 20-week monitoring period.

Figure 7. Verification of length estimation for a single, isolated fish. The figure visualizes the processing pipeline used for length estimation. (a) Original RGB image with annotated key points and bounding box. (b) Corresponding depth map. (c) Reconstructed 3D coordinates of the fish. (d) Denoised depth values along the fish body in both 3D and 2D space.

Figure 7. Verification of length estimation for a single, isolated fish. The figure visualizes the processing pipeline used for length estimation. (a) Original RGB image with annotated key points and bounding box. (b) Corresponding depth map. (c) Reconstructed 3D coordinates of the fish. (d) Denoised depth values along the fish body in both 3D and 2D space.

Across all populations, standard deviations and residual errors increased with fish size and system density. This trend reflects the cumulative effects of higher feeding rates, reduced filtration efficiency, and overlapping fish, which together degrade image quality and depth accuracy. A mild systematic negative bias in length estimation is present throughout all cultivation stages. This bias originates partly from residual camera-calibration inaccuracies and partly from the geometric properties of keypoint-based length extraction: fish may appear curved during swimming, yielding shorter reconstructed lengths, whereas overestimation is geometrically impossible. Furthermore, keypoints are filtered based on depth consistency, preventing detections outside the body contour and thereby reinforcing this negative bias. Environmental influences such as biofilm accumulation on the camera lens and variable illumination conditions further affect length estimation accuracy. In later cultivation stages, these effects are compounded by increased turbidity and frequent occlusions, which introduce an additional sampling bias toward smaller and more visible individuals. As illustrated in Figure 8, from week 16 onward, not enough valid automatic measurements could be obtained for the RASF1 population because of excessive turbidity and occlusions. Because weight is derived via a non-linear allometric growth equation, small deviations in length propagate disproportionately into the final weight estimates, contributing to the divergence between automatic and manual measurements visible in the later cultivation stages of Figure 8. Despite these limitations, the combined use of visual and reference measurements provided a robust basis for modeling population-level growth dynamics and validating the performance of the inference system under real farming conditions.

Figure 8. Growth curves of three fish populations cultivated in individual recirculating aquaculture systems (RAS1–3) over a 20-week monitoring period. Automated computer-vision-based measurements (solid lines) are compared with periodic manual reference measurements (symbols).

Figure 8. Growth curves of three fish populations cultivated in individual recirculating aquaculture systems (RAS1–3) over a 20-week monitoring period. Automated computer-vision-based measurements (solid lines) are compared with periodic manual reference measurements (symbols).

The integration of computer vision into the aquaculture module demonstrates the feasibility of real-time monitoring within the circular demonstrator system. In the implemented setup, visual biomass metrics are synchronized with water-chemistry measurements and other analytical parameters to assess growth dynamics and system stability under operational conditions. Combined image-based and sensor-derived data are analyzed to identify deviations in feeding patterns, growth progression, and water-quality indicators, supporting data-informed management decisions within the facility.

At the same time, the findings highlight current limitations of optical monitoring. Increasing biomass density and turbidity, as observed in the RASF1 population, can compromise visibility and model accuracy. These challenges emphasize the importance of integrating complementary data sources such as dissolved oxygen, pH, redox potential, and nutrient concentrations. Together with computer vision, these measurements provide the basis for developing digital twins that continuously simulate and optimize system performance.

The experience gained from the aquaculture module illustrates how multimodal sensing, computer vision, and machine learning can jointly improve circularity and resilience in food production. Building on this foundation, the following section focuses on terrestrial livestock, using poultry as an example to demonstrate how vision-based monitoring supports welfare management, emission reduction, and nutrient recycling in a controlled environment. Although implemented in a recirculating aquaculture research environment, the described monitoring pipeline can be adapted to commercial RAS facilities by adjusting camera positioning and recalibrating species-specific growth models.

4. Computer Vision for Livestock Production (Poultry)

Computer vision (CV) has become an established component of precision livestock farming, enabling non-invasive monitoring in controlled production environments. In cattle, swine, and sheep systems, CV is widely applied for behavior recognition, lameness detection, and weight estimation under comparatively stable conditions [30,31]. Poultry farming, however, presents substantially greater technical challenges. Chickens are smaller, move more erratically, and are housed in large groups with high stocking densities and frequent occlusions, complicating reliable detection and tracking [32]. Variable lighting, airborne dust, and reflective surfaces further reduce image quality under commercial conditions.

Within our circular indoor agriculture demonstrator, we implemented a computer-vision-based poultry monitoring system for continuous behavioral observation under controlled housing conditions. The system integrates RGB image acquisition, object detection, multi-frame tracking, and behavioral feature extraction, as summarized in Figure 9.

Deep-learning-based detection models, including YOLO architectures, are commonly used for individual bird identification and activity analysis in dense flocks [33,34,35]. In the implemented system, a YOLO-based model detects individual birds in consecutive frames, enabling trajectory reconstruction. From these trajectories, intra-object metrics such as movement speed and total distance traveled are computed, as well as inter-object metrics including clustering behavior and nearest-neighbor distances.

A dedicated feeding-detection module identifies feeding events based on spatial proximity to defined feeding zones. Environmental parameters within the housing unit, including temperature, CO₂, and ammonia concentrations [11], are recorded and evaluated alongside behavioral metrics. Figure 9 illustrates the implemented processing pipeline and representative RGB images recorded under operational farming conditions.

The YOLO-based detection model was trained and evaluated on manually annotated image datasets collected within the demonstrator facility. Individual birds were labeled using bounding boxes under varying lighting conditions and stocking densities to ensure robustness across operational scenarios. Data augmentation techniques, including horizontal flipping, brightness variation, and scaling, were applied to improve generalization performance. Object detection combined with multi-frame tracking enables reconstruction of movement trajectories over defined observation intervals. From these trajectories, baseline activity distributions are established under stable housing conditions. Deviations from these baseline patterns, such as sustained increases or decreases in locomotor activity or altered feeding frequency, are flagged for further inspection. In practice, identified deviations are evaluated in relation to recorded environmental parameters (temperature, CO₂, ammonia) and management variables to support operational welfare assessment within the facility.

To interpret behavioral metrics within a welfare-oriented perspective, established assessment frameworks provide important reference points. The Animal Welfare Indicators (AWIN) framework is internationally recognized for structured evaluation of behavioral and environmental welfare criteria. AWIN assessments are typically performed manually through visual inspection. Several behavioral parameters extracted in the present monitoring system (e.g., activity levels, clustering behavior, feeding frequency) correspond to observable indicators defined within such welfare frameworks. Physiological indicators, including hormone-based stress markers, are rarely measured in commercial practice due to their invasive nature. Emerging research has investigated epigenetic biomarkers, such as DNA methylation signatures, as potential complementary indicators of long-term stress responses [36]. Although molecular indicators are not part of the current demonstrator implementation, they illustrate future pathways for linking automated behavioral monitoring with broader welfare analytics.

Within the framework of circular agriculture, poultry monitoring is also relevant for evaluating feed strategies and nutrient-cycle integration. Automated observation of feeding behavior supports assessment of alternative protein sources, such as insect- or algae-based feeds, by quantifying acceptance rates and behavioral responses. The combined analysis of behavioral and environmental parameters contributes to operational welfare monitoring within the integrated multi-trophic system.

5. Computer Vision for Vertical Plant Farming

CV is widely applied in vertical and greenhouse farming for non-invasive growth monitoring under controlled environmental conditions [37,38]. Image-based phenotyping supports quantification of plant morphology and development, enabling data-driven cultivation strategies. Recent advances include RGB-D-based biomass estimation and machine-learning-assisted segmentation approaches for growth analysis [39,40,41].

In our demonstrator system, CV is integrated into aquaponic plant production units as part of the multi-trophic circular architecture. Growth metrics derived from image data are evaluated alongside environmental and nutrient measurements to document cultivation performance under operational conditions. Advanced imaging modalities such as hyperspectral systems can provide detailed physiological information in specialized research environments [42]. However, the present demonstrator focuses on RGB-D-based morphological monitoring as a cost-efficient and operationally robust solution.

Figure 10 illustrates the implemented monitoring setup for leafy-green production across three aquaponic units. A mobile RGB-D camera mounted on a belt-driven linear rail traverses multiple cultivation beds and captures synchronized color and depth maps of the plant canopy. Machine-learning-based segmentation distinguishes plant regions from background and extracts quantitative parameters such as visible plant area and canopy height profiles.

Segmentation is performed using a lightweight statistical model based on a support vector machine (SVM). The model is trained using manually selected representative pixels from plant and background regions and can be adapted within minutes to new crop types or lighting conditions. Unlike deep-learning approaches, this method requires only limited training data and is suited to research environments with frequently changing cultivation parameters.

The derived growth metrics are integrated into the facility’s data infrastructure and evaluated in relation to light intensity, temperature, CO₂ concentration, and nutrient composition. Concepts of data integration and digital-twin-based system modeling described in the recent literature [43,44] inform the architectural design of the monitoring framework, although full predictive simulation is not the focus of the current implementation. The temporal development of visible leaf area and average canopy height over a 16-day monitoring period is exemplified in Figure 10c. These measurements support comparative assessment of cultivation units and documentation of nutrient-management strategies within the aquaponic system.

The lightweight statistical segmentation approach demonstrates that effective growth monitoring can be achieved without computationally intensive deep-learning architectures or large annotated datasets. Fast-trainable models based on a limited number of manually selected examples provide a practical solution for research environments with frequently changing crop varieties and cultivation parameters. The modular imaging setup allows adjustment of camera positioning and rapid retraining of the segmentation model when cultivation conditions are modified within the facility.

6. Integration of Data Streams and System Architecture in the Aquaponic Modules

In aquaponic systems, plant growth, fish performance, and water chemistry are intrinsically coupled through shared nutrient and energy flows. Continuous monitoring of water-quality parameters such as pH, dissolved oxygen, ammonia, nitrate, temperature, and turbidity is an established practice in recirculating aquaculture systems and represents the current industry standard for operational control. Similarly, environmental parameters in plant production modules, including light intensity, CO₂ concentration, humidity, and temperature, are routinely recorded in controlled-environment agriculture.

Within the demonstrator facility, these established sensor streams are integrated with image-derived metrics from computer-vision-based fish and plant monitoring. Stereo-derived fish biomass estimates and RGB-D-based plant growth metrics are time-synchronized with water-quality, nutrient, feeding, and environmental measurements in a shared database infrastructure. This unified data architecture enables cross-module evaluation of nutrient dynamics, growth progression, and system stability under operational conditions.

The structural design of this data framework is informed by digital-twin concepts described in the recent literature [45,46,47,48,49,50,51], in which physical production systems are mirrored by synchronized digital representations. In the present demonstrator, however, the implementation focuses on systematic data acquisition, harmonization, and comparative analysis rather than predictive simulation or automated control.

By incorporating computer vision into an already sensor-rich aquaponic infrastructure, the system extends conventional parameter monitoring to include direct biological performance indicators. This integration allows documented linkage between nutrient availability, environmental conditions, fish growth, and plant development across trophic levels. While full digital-twin simulation and model-based optimization remain future development steps, the realized data architecture establishes the structural foundation for such approaches.

7. Upconversion Modules for Nutrient and Biomass Recovery

Closed-loop agriculture (Figure 1) relies on biological upconversion processes that transform solid, liquid, and gaseous by-products into new sources of biomass and nutrients. These processes tighten nitrogen and phosphorus cycles, enhance carbon fixation, and minimize waste within the overall food-production system. In our framework, three complementary modules fulfill this role: microalgae that assimilate inorganic nutrients and capture carbon dioxide, duckweed that rapidly converts dissolved nitrogen into protein-rich biomass, and insects that upcycle organic residues into valuable feed components. Together, these organisms act as biological recyclers, converting residual material into new chemical energy stored in organic matter. This energy is retained in the form of proteins, lipids, and carbohydrates that re-enter the production cycle, rather than being released as electricity or heat.

In this context, the term energy recovery refers not to thermochemical energy generation, such as biogas or combustion, but to the retention of chemical energy bound in biomass for reuse within the trophic network. The focus is therefore on nutrient and biomass recovery: converting waste streams into biologically useful and energetically valuable material. To achieve this efficiently, each subsystem must be tightly monitored and controlled, linking visual, chemical, and physical data within a shared analytical and control framework.

As outlined in Chapter 6, recent work on data-driven aquaponics and controlled-environment systems has demonstrated that time-series modeling and IoT-based sensing can predict key water-quality variables, including dissolved oxygen, pH, ammonia, nitrate, temperature, and turbidity. These predictive tools enable proactive management actions such as adjusting aeration, recirculation, or nutrient dosing before imbalances propagate through the coupled fish–plant–microbe ecosystem [41,49,50]. Nutrient management models that integrate feature selection and machine learning, including XGBoost and random forest, identify the most influential predictors and automate supplementation, while plant spectral analytics directly quantify nitrogen, phosphorus, and potassium uptake [45,46]. Climate-adaptive simulations further improve stability and energy efficiency under variable environmental conditions [51].

Building on these foundations, the upconversion modules in our circular research facility combine computer vision with analytical sensing to manage biomass production in real time. Vision systems provide non-invasive assessments of growth, morphology, and system health, while digital-twin models fuse these observations with environmental and chemical data to regulate conversion rates, residence times, and harvest schedules. Following the established distinction between monitoring twins, which synchronize digital and physical systems for situational awareness, and predictive twins, which simulate future states and guide proactive decisions [47,48], each module contributes to a harmonized feedback architecture for sustainable, data-driven nutrient and biomass recovery.

The following sections present representative implementations for microalgae, duckweed, and insect bioconversion, highlighting their roles as interconnected trophic levels within the larger closed-loop food system.

7.1. Microalgae Cultivation

Microalgae serve as efficient converters of gaseous and dissolved emissions into high-value biomass. In controlled circular agriculture, they assimilate carbon dioxide and ammonia, thereby linking emission reduction with resource recovery. Their biomass retains chemical energy in the form of proteins, lipids, and carbohydrates that can be reintroduced into the production cycle as feed or fertilizer. Recent work highlights that integrating microalgae into recirculating aquaculture systems can further enhance nutrient recovery, oxygenation, and effluent valorization, providing environmentally and economically viable pathways for upgrading all major waste streams [52]. In our research facility, a cone-shaped helical photobioreactor was developed for the cultivation of Arthrospira sp. (Spirulina) directly coupled to the exhaust air of a poultry house [11]. The reactor design allows continuous gas exchange between the livestock unit and the algal culture, converting CO₂ and NH₃ from the exhaust into biomass while purifying the air.

Computer vision supports the process by providing non-invasive, real-time information on culture morphology and growth dynamics (Figure 11). Microscopic imaging is used to distinguish helical and straight Arthrospira filaments and to identify co-occurring microorganisms such as ciliates or diatoms. Beyond their implications for harvesting, the relative abundance of the two Arthrospira phenotypes can serve as an indicator of physiological status, as a predominance of straight filaments has been associated with environmental stress and altered metabolic activity [53].

To illustrate this relationship, Figure 11 presents an example workflow in which microscopic images are segmented into “spiral” and “straight” Arthrospira classes. The relative abundances of these morphological types are tracked over time to identify shifts in filament structure that may indicate physiological stress or altered metabolic activity. Automated classification thereby supports continuous process monitoring within the integrated production system.

Automated image analysis quantifies filament morphology, aggregation patterns, and the presence of co-occurring microorganisms, complementing routine measurements such as optical density and pH. Variations in the relative proportions of straight and helical filaments are documented over time and evaluated in relation to operational parameters of the photobioreactor, including gas supply, illumination, and mixing conditions. These measurements are stored within the facility’s data infrastructure, where visual and chemical data streams are synchronized for comparative analysis.

Within this integrated monitoring framework, the microalgae module functions as a carbon sink and nutrient recycler within the circular system. The produced Arthrospira biomass captures chemical energy from exhaust gases and provides a protein-rich feed resource, contributing to nutrient recirculation within the demonstrator facility.

7.2. Duckweed Systems

Duckweeds (family Lemnaceae) are among the fastest-growing angiosperms and play an important role in nutrient recovery from water streams [12]. Due to their rapid clonal propagation and high protein content, they represent an efficient alternative biomass source for feed and fertilizer applications. When integrated into circular agricultural systems, duckweeds absorb nitrogen, phosphorus, and trace elements from residual process water, thereby improving water quality while generating protein-rich biomass suitable for feed or as substrate for insect cultivation [12].

Automated monitoring of duckweed growth is essential for stable operation and continuous process control. Manual sampling is labor-intensive and unsuitable for high-frequency management. Recent studies have demonstrated the feasibility of image-based growth quantification and automated phenotyping using microscopy-based imaging systems, deep-learning segmentation (e.g., StarDist), and modular laboratory automation platforms [54,55,56]. Modular vertical farming concepts with automated nutrient dosing and recirculation further highlight the scalability of duckweed biomass production [12].

Building on these developments, our facility integrates duckweed cultivation into the closed-loop production system as a nutrient sink downstream of aquaculture and livestock modules. An automated monitoring setup combines computer vision with controlled environmental parameters to stabilize biomass output and support continuous water-quality management.

The monitoring workflow is shown in Figure 12, which illustrates a lightweight CV pipeline designed for robust daily operation. The system uses an RGB camera mounted on a vertical, spindle-driven axis to capture angled images of the cultivation trays, with the capture angle chosen to maximize the visible water surface while minimizing the vertical spacing between shelves. Captured images are orthorectified to ensure consistent spatial scaling before further analysis. Instead of pixel-accurate deep-learning segmentation, the system employs a lightweight statistical regression model trained on a limited set of biomass-derived surface-coverage reference values. Because the model operates on global color and texture descriptors rather than pixel-level annotations, it requires minimal manual labeling and can be retrained rapidly to accommodate different Lemna species or variable illumination conditions.

Surface-coverage data are continuously analyzed, triggering automated overflow harvesting once a defined threshold is exceeded. By linking visual monitoring, water analytics, and automated control, the duckweed module functions as an active interface between aquatic and terrestrial production lines. It converts dissolved nutrients into plant biomass, stabilizes water quality for recirculation, and provides a renewable protein resource within the integrated circular system.

7.3. Insect Bioconversion

Insect bioconversion represents a crucial trophic component within circular agricultural systems, transforming organic side streams into protein-rich biomass suitable for animal feed or aquaculture [57]. Among the species used for this purpose, the black soldier fly (Hermetia illucens, BSF) has gained particular attention for its efficiency in converting diverse waste substrates into valuable nutrients and for its ability to close nutrient loops between livestock, aquaculture, and plant production.

Despite this potential, large-scale insect farming still faces challenges in process control, labor efficiency, and product standardization. Manual monitoring of larval density, developmental stages, or substrate degradation is time-consuming and error-prone. Computer vision and deep learning offer non-invasive tools for automation and continuous monitoring, enabling the precise control of key production parameters [57].

Recent advances demonstrate the applicability of deep learning to insect production monitoring. Convolutional neural networks combined with optical flow analysis enable reliable classification of BSF developmental stages under practical conditions [58]. Detection and regression networks such as YOLOv8 and ResNet have shown strong agreement between estimated and measured larval weights, supporting automated trait-based management [59]. Experiments with macronutrient-balanced artificial substrates revealed the importance of hydration and protein content for efficient bioconversion [60]. Machine-learning models such as XGBoost have further improved weight-gain predictions and enabled the optimization of mixed organic waste diets, significantly enhancing feed conversion efficiency and biomass yield [61].

In our integrated research and development facility, computer vision forms the basis of a non-invasive monitoring strategy for BSF bioconversion. RGB image sequences are analyzed to extract larval density, size distribution, color variation, and motion activity as proxies for developmental stage and substrate degradation. These visual features are fused with environmental measurements, such as temperature, humidity, and CO₂ concentration, to generate a multimodal data stream for predictive modeling and process control. Figure 13 illustrates this workflow, showing how BSFL are visually monitored in the context of their use as a feed supplement within the circular production system. The derived population indicators are integrated with environmental sensor data to enable adaptive decisions regarding substrate dosing, moisture management, and harvest timing within the insect bioconversion module.

By combining vision-based phenotyping with environmental sensing and machine learning, the insect module functions not only as a nutrient recycler but also as a data-rich regulatory component within the overall circular system. Larval growth patterns and activity profiles inform dynamic substrate allocation, moisture control strategies, and optimized harvest scheduling, while simultaneously contributing to the balancing of nutrient flows between fish, poultry, and plant modules. While the implementation is still evolving, these methods highlight how insect bioconversion can become an analytically transparent and optimizable element of future closed-loop food production systems [57,60].

8. System Integration and Data Fusion

The realization of a closed-loop agricultural system requires not only the coupling of biological modules but also the seamless integration of data flows across all components. Each subsystem (fish, poultry, plants, algae, and insects) produces heterogeneous data, including RGB images and depth maps, spectral signatures, water chemistry, temperature, humidity, and gas concentrations. To transform these multimodal data streams into actionable information, they must be captured, synchronized, and analyzed within a unified digital infrastructure [47,48].

In our facility, sensor and imaging data from all modules are collected via local edge devices and transmitted to a central inference server for preprocessing, fusion, and analysis. The architecture follows a modular design: each biological module contributes structured and unstructured data to a common database, enabling joint interpretation of biological, chemical, and physical variables. This integration forms the backbone for a digital representation of the entire system, or digital twin, which connects real-time measurements with simulation and prediction models [44].

Machine-learning methods play a central role in this architecture. Time-series models such as recurrent neural networks (RNNs), long short-term memory (LSTM), and gated recurrent unit (GRU) networks are particularly well suited for modeling complex temporal dependencies, for example, in water-quality dynamics or nutrient cycling [49,50]. By combining vision-based observations with sensor-based measurements, predictive models can identify anomalies, anticipate shifts in system equilibrium, and support adaptive control decisions. Similar data-driven optimization approaches have already demonstrated strong potential in aquaponics, where nutrient supply and water quality are jointly optimized through continuous learning loops [45].

A key objective of the ongoing implementation is to enable data fusion across multiple layers of information. High-level features extracted from computer vision models, such as biomass growth rates or color indices, are merged with low-level environmental variables, such as pH, ammonia, and dissolved oxygen. This combined dataset allows the inference engine to estimate latent variables that are otherwise inaccessible to direct measurement, such as metabolic activity or system-level nutrient balance. The integration also supports higher-level decision-making processes, including the coordination of feeding, lighting, and harvesting cycles across modules [43].

In the present architecture, multimodal data fusion follows a hierarchical and robustness-oriented design. Heterogeneous datasets, such as image-derived features, spectral information, and physicochemical sensor measurements, are first processed using modality-specific preprocessing pipelines and then transformed into structured feature representations. These features are temporally synchronized via a unified timestamp framework and normalized to comparable scales.

Fusion is performed through feature-level and representation-level integration within the central inference server. In neural multimodal architectures, heterogeneous datasets can be fused by transforming modality-specific inputs into shared latent representations through specialized subnetworks. These representations can subsequently be integrated at the feature, representation, or decision level, depending on the desired balance between robustness and interpretability. This multimodal learning strategy enables flexible late-stage fusion while preserving the interpretability of individual modalities.

To enhance robustness against sensor disturbances and incomplete data, the framework incorporates principles of disturbance-aware training and multisensor redundancy. Building on prior work in robust multisensor system design [62] and adaptive training strategies for disturbance-resilient object detection [63], the approach integrates both functional sensor data and simulated perturbations during model development. Such disturbance-aware training improves resilience against noise, occlusion, turbidity, illumination changes, and partial sensor degradation. These robustness-oriented fusion strategies are essential for reliable operation in biologically dynamic agricultural environments.

Although the complete integration of all modules is still under development, the conceptual framework and technical implementation are largely established. The current focus is on ensuring interoperability between data sources, standardizing interfaces, and refining automated feedback loops for process control. Once fully operational, this infrastructure is intended to serve as a scalable reference model for digitalized circular agriculture, demonstrating how multimodal data analytics and digital twins can translate biological complexity into measurable, controllable processes.

While certain hardware configurations described here (e.g., specific camera placements or tank geometries) reflect the infrastructure of our research facility, the underlying data architecture, inference pipelines, and digital twin framework are modular by design. The separation of edge acquisition, centralized inference, and database integration allows adaptation to different farm sizes, climatic conditions, and production focuses. Core components such as multimodal data fusion, probabilistic modeling, and cross-module feedback control can be implemented independently of the specific biological modules described here.

9. Discussion and Outlook

The transition toward fully integrated, circular agricultural systems relies not only on technological innovation but also on our ability to manage the biological and operational uncertainties that arise from complex, interlinked trophic networks. While data-driven control and digital twins provide unprecedented insight and automation potential, they must be reconciled with the intrinsic variability of living systems and the need for adaptive configuration across production modules.

9.1. Biological Variability and Uncertainty Management

Growth dynamics in biological systems exhibit substantial variability, both within and across trophic levels. Factors such as microbial composition, genetic diversity, temperature fluctuations, and stochastic feeding behavior can result in large deviations from predicted growth trajectories. Classical experimental strategies based on replication and controlled variance reduction are difficult to apply in closed-loop systems, where long generation times and interdependencies between modules limit the feasibility of replicated trials.

To address these challenges, data science must explicitly model uncertainty rather than treat it as experimental noise. Probabilistic and Bayesian approaches can estimate confidence intervals for key parameters such as growth rates, biomass conversion efficiency, or nutrient uptake. Ensemble methods and Gaussian process regression can propagate these uncertainties through coupled models, allowing the system to operate with quantified rather than assumed stability. In practice, such methods enable robust control strategies that tolerate deviations while maintaining the balance between energy input, nutrient recycling, and biomass output. Over time, incremental learning and continuous data acquisition are expected to narrow uncertainty margins, improving predictive accuracy even in the absence of extensive replication.

9.2. Adaptive Configuration of Circular Modules

The modular structure of the circular system allows flexible coupling between trophic levels, for example, by operating microalgae and duckweed as a hybrid aquaponic system or by directing insect biomass as feed for fish or poultry. Each configuration alters nutrient flow, microbial composition, and overall system efficiency. Identifying the optimal setup under changing environmental and production conditions, therefore, represents a multidimensional optimization problem.

Data-driven optimization frameworks can help explore this design space. Reinforcement learning, multi-objective optimization, and digital twin simulations can test hypothetical configurations virtually before implementation. By simulating feedback loops between modules, such as nutrient recycling rates, water quality, or energy demand, these systems can identify operating points that maximize overall resource efficiency while maintaining ecological balance. Coupled with real-time data fusion from all modules, adaptive configuration becomes a continuous process rather than a fixed design choice, allowing the system to evolve dynamically in response to external perturbations or production goals.

9.3. Future Perspectives

The integration of computer vision, sensor networks, and machine learning across multiple trophic levels marks a paradigm shift toward autonomous and self-regulating agricultural systems. As these technologies mature, their success will increasingly depend on interoperability standards, transparent data governance, and accessible system architectures that allow scaling from laboratory setups to industrial applications.

Beyond technical feasibility, broader questions emerge concerning sustainability assessment, ethical considerations in automation, and the socio-economic integration of such systems into local food networks. Quantifying the true ecological benefits of circular agriculture will require harmonized life-cycle assessments that integrate environmental, economic, and social indicators. In parallel, participatory design approaches can ensure that automation serves both environmental and human well-being, aligning technological progress with public trust and policy frameworks.

From a system-design and commercial perspective, the transferability of the presented framework depends on modular implementation rather than full-scale replication of the entire facility. The pilot installation described here serves as a demonstrator for the gradual retrofitting of existing agricultural operations. Individual modules, such as computer-vision-supported aquaculture monitoring, poultry behavior analytics, or plant-growth quantification, can be integrated into existing infrastructure without requiring a complete system replacement. This modular retrofit approach lowers entry barriers for small- and medium-scale farms and enables incremental digitalization aligned with available resources and investment capacities.

While the fully integrated demonstrator represents an advanced configuration, its architecture is designed to allow staged deployment. Farms may adopt selected monitoring and data-fusion components first and progressively expand toward more comprehensive circular integration as economic viability and operational expertise increase. Such scalable implementation pathways support practical commercialization and reduce financial risk, thereby strengthening the real-world applicability of digitally assisted circular agriculture.

From an environmental and systemic sustainability perspective, the framework enables resource-efficient food production through integrated nutrient recycling, water reuse, and data-driven process optimization. Its modular structure facilitates implementation in urban agriculture initiatives, research infrastructures, and regions facing climatic constraints where spatial efficiency and emission reduction are critical. Controlled-environment approaches are inherently designed to decouple agricultural production from external climatic variability, making them particularly suitable for regions exposed to heat stress, water scarcity, or unstable seasonal patterns.

At the same time, broader deployment requires careful consideration of practical constraints, including initial hardware investment costs, the need for technical expertise in data integration, and the challenge of harmonizing heterogeneous data streams across modules. Addressing these factors will be essential for translating experimental demonstrators into widely adopted circular production systems.

From an economic and societal perspective, the presented framework provides a pathway for the structural transformation of existing agricultural infrastructure toward digitally supported circular production. By enabling the retrofitting of conventional farms rather than requiring complete system replacement, the approach supports long-term modernization while preserving existing assets. Process transparency, automated monitoring, and predictive analytics contribute to yield stabilization, reduce feed and nutrient losses, and lower operational risks associated with biological variability. In aquaculture modules, recirculating systems significantly reduce water consumption compared to flow-through systems, while controlled indoor environments limit evaporation losses and improve overall water-use efficiency.

The increasing availability of renewable energy sources further strengthens the economic viability of controlled-environment systems, particularly in regions with high solar irradiation, where photovoltaic integration can offset operational energy demand. Beyond direct economic effects, the framework contributes to societal resilience by supporting regional food production, reducing dependency on long supply chains, and enabling climate-adaptive agriculture in water-scarce or temperature-sensitive regions. In this way, digitalized circular agriculture can combine environmental responsibility with economic competitiveness and social stability.

In conclusion, this project demonstrates the implementation of an integrated multi-trophic circular indoor agriculture facility in which computer-vision-based monitoring and multimodal sensing are systematically deployed across aquaculture, poultry, plant, and microalgae modules. The realized data infrastructure enables synchronized acquisition of biological, chemical, and environmental parameters under operational conditions, providing a documented basis for analyzing nutrient dynamics, growth performance, and system stability.

The demonstrator shows how established water and environmental sensor systems can be complemented by computer vision to directly quantify biological performance across trophic levels. While full predictive simulation and automated optimization remain future development steps, the implemented monitoring architecture establishes a scalable framework for data-driven evaluation of circular production systems under controlled-environment conditions.