Agentic AI-Driven Cultivation Advisory and Symptom-Level Diagnostic Support in a Controlled Indoor Farming System

Abstract

Small-scale and urban indoor farms typically rely on manual observation, which delays stress detection and yields inconsistent crop quality. While large language models (LLMs) and retrieval-augmented generation (RAG) have been explored for agricultural advisory systems, their integration into a single cloud-free indoor-farming platform that couples multimodal symptom interpretation with autonomous environmental control remains largely unexamined. This study presents an integrated platform built around an agentic AI advisory pipeline that runs entirely on-device on commodity hardware. The pipeline couples a RAG-grounded Mistral 7B language model with a LLaVA 7B vision-language model through condition-based routing, intent classification, multi-step reasoning, and an LLM validation gate, delivering context-aware text and image-based symptom-level guidance from a conversational interface. The advisory layer operates alongside vision-based plant monitoring and a deliberately isolated threshold-based control layer, in which an ESP32 microcontroller autonomously actuates irrigation and lighting against predefined thresholds while a Raspberry Pi 5 performs continuous plant detection and browning monitoring. On Cos lettuce, the advisory pipeline achieved 82.00% weighted accuracy across 50 queries spanning health, symptom, watering, pest, root-health, and growth-stage categories, scored against established plant pathology and postharvest references, with no incorrect responses recorded. The study contributes the design of an agentic advisory pipeline and its integration into a working, cloud-free indoor-farming platform, providing an on-device foundation for intelligent small-scale farming.

1. Introduction

The global demand for food continues to rise alongside population growth and increasing pressure on arable land, making food security one of the most pressing challenges of the twenty-first century [1]. Indoor farming has emerged as a promising alternative to conventional open-field cultivation, enabling year-round crop production in controlled-environment enclosures that are decoupled from seasonal variability, extreme weather events, and geographic constraints [2,3]. By precisely regulating light, irrigation, humidity, and temperature, indoor systems can achieve substantially higher yield consistency compared with traditional agriculture [4,5]. Despite these advantages, the majority of small-scale and urban indoor farming operations continue to depend on manual observation for plant monitoring and cultivation decision-making, resulting in delayed detection of stress conditions, inconsistent crop outcomes, and inefficient use of water and energy resources [6,7].

The integration of Internet of Things (IoT) technologies into agricultural systems has enabled continuous, automated monitoring of key environmental parameters, including soil moisture, temperature, humidity, and photosynthetically active light intensity [8,9]. IoT-based platforms can trigger automated actuation such as irrigation pulses and grow-light scheduling, substantially reducing the burden of manual intervention and supporting resource-efficient cultivation [10,11]. Recent deployments in smart greenhouse environments have demonstrated stable closed-loop control of temperature, humidity, and fertigation with measurable improvements in crop yield and resource use [12,13]. However, translating raw sensor streams into actionable cultivation guidance remains a significant challenge, particularly for non-expert users who lack the agronomic knowledge to interpret environmental readings or identify early signs of plant disease [14].

Recent advances in large language models (LLMs) and retrieval-augmented generation (RAG) have opened new avenues for intelligent advisory systems across a broad range of domains, including agriculture [15,16]. RAG-based agricultural chatbots ground LLM responses in domain-specific knowledge bases such as cultivation guides and pest management manuals, substantially reducing hallucination and improving factual reliability compared with vanilla LLMs [17,18]. Systems such as CottonBot [19] have demonstrated that RAG-augmented LLMs integrated with IoT soil-moisture sensors and weather APIs can deliver expert-level irrigation recommendations responsive to real-time field conditions. Similarly, PEZEGO [20] illustrated how LLM reasoning coupled with IoT telemetry can support pest management decisions in precision agriculture. However, most existing systems rely either on static knowledge bases without real-time sensor integration, or on single-task advisory pipelines without multimodal disease diagnosis capabilities. The concept of agentic AI, where an LLM autonomously routes inputs through condition-based pipelines and invokes external knowledge retrieval as part of its reasoning chain [21], offers a qualitative advance over standard RAG by enabling context-aware multi-step responses, yet end-to-end agentic deployments within unified indoor farming platforms remain largely unexplored.

Accurate and timely plant disease diagnosis is equally critical to cultivation advisory systems as undetected pathogens and nutrient deficiencies can severely reduce crop quality and yield [22]. Traditional diagnosis depends on expert visual inspection, which is both labor-intensive and inaccessible for non-specialist growers. Deep learning-based approaches have demonstrated strong performance in plant disease classification and segmentation tasks [23,24], and transformer-based models combining in situ sensor data with plant images have further improved diagnostic accuracy under realistic field conditions [25]. More recently, LLM-based knowledge graph construction for plant diseases [26] and multimodal vision-language models have shown the potential to support conversational, image-grounded diagnosis accessible to end users. Nevertheless, integrating such multimodal diagnosis capability within a real-time controlled indoor farming system, where automated closed-loop control and symptom-level diagnosis are delivered through a single natural-language interface, has not been demonstrated in the prior literature.

To address these gaps, this paper presents a system for agentic AI-driven cultivation advisory and symptom-level diagnostic support deployed in a controlled indoor farming system. The proposed system, built around Cos lettuce cultivation as the validation crop, integrates an ESP32-based sensing and control layer for real-time monitoring of soil moisture and grow light status together with automated closed-loop control of irrigation and LED grow lighting. An agentic LLM pipeline, grounded through an RAG pipeline constructed from domain-specific cultivation knowledge, employs condition-based routing and multi-step reasoning to deliver context-aware cultivation recommendations. A multimodal vision-language model provides symptom-level diagnostic support from user-submitted image and text inputs, enabling non-expert users to receive symptom interpretation and management guidance through a web-based conversational interface.

While the individual building blocks employed in this work are established open-source techniques and are not claimed as contributions in themselves, the contribution lies in the architecture that governs how inputs are routed between the text and vision paths, how responses are grounded in retrieved documentation and validated before delivery, and how the advisory layer is isolated from the control layer. Table 1 illustrates that existing systems address these capabilities in isolation, and the present work targets the identified gap.

Table 1. Design scope of representative IoT-, LLM-, and vision-based agricultural systems in relation to the proposed platform.

System	Closed-Loop Control	RAG Advisory	Agentic Routing	Multimodal Diagnosis	On-Device Cloud-Free
IoT smart greenhouse [4,12]	✓	×	×	×	∘
AgroLLM [22]	×	✓	×	×	×
RAG advisory [17]	×	✓	×	×	×
CottonBot [19]	✓	✓	✓	×	×
PEZEGO [20]	∘	✓	∘	×	×
Pardo-Pina et al. [21]	✓	✓	✓	×	×
Transformer diagnosis [25]	×	×	×	✓	∘
pdKGraph [26]	×	∘	×	∘	×
This work	✓	✓	✓	✓	✓

✓: within reported scope; ∘: partially addressed; ×: outside the system’s reported scope. Bold indicates the system proposed in this work.

Two points of scope should be stated at the outset. First, this article reports a systems-engineering contribution. The research question is whether a platform integrating threshold-based actuation, on-device vision monitoring, and an agentic LLM advisory pipeline can be constructed and operated stably on commodity hardware without cloud dependency. Whether such a system improves agronomic outcomes relative to conventional management is a separate question, requiring a replicated and randomized cultivation trial with crop-performance measurement, and is not addressed here. Second, the term diagnosis is used throughout in the sense of symptom-level diagnostic support: the identification of visible symptoms, the enumeration of plausible differential causes, and the provision of corresponding management guidance. It does not denote pathogen confirmation, which requires microbiological or molecular verification and lies outside the scope of this study.

The main contributions of this work are as follows:

• Agentic advisory pipeline. An LLM pipeline combining condition-based routing, intent classification, RAG-grounded multi-step reasoning, and an LLM validation gate. It is specified as an explicit dual-path dispatch algorithm rather than a monolithic prompt.
• Multimodal diagnostic support. A LLaVA 7B module that interprets plant symptoms from images and text. It identifies visible symptoms, lists plausible causes, and returns management guidance for users without agronomic training.
• Browning detection. A vision routine pairing YOLO-based plant localization with color-ratio analysis to flag abnormal leaf coloration. Each alert logs the incident image, growth stage, and sensor state for follow-up.
• Isolated control layer. A threshold-driven architecture for irrigation and lighting actuation, separated from the advisory layer so that no language model output can reach an actuator (Section 3.1.6). Continuous unattended operation was demonstrated across a Cos lettuce deployment period.

The remainder of this paper is organized as follows. Section 2 reviews related work and positions the proposed system against the existing literature. Section 3 presents the materials and methods, covering the system architecture and design (Section 3.1) and the materials and experimental methods (Section 3.2). Section 4 presents the results. Section 5 discusses their implications, limitations, and directions for future research. Section 6 concludes the paper.

2. Related Work

This section reviews prior work across three themes directly relevant to the proposed system, closed-loop control in indoor farming, LLM- and agentic-AI-based advisories, and vision-based plant monitoring and diagnosis, and then synthesizes the recurring limitations that motivate the present integration.

2.1. IoT-Enabled Monitoring and Closed-Loop Control

Controlled-environment agriculture has been advanced substantially by IoT instrumentation that continuously senses soil moisture, temperature, humidity, and light and drives automated actuation [8,9,27]. Smart indoor and greenhouse platforms have demonstrated stable closed-loop regulation of temperature, humidity, and fertigation with measurable gains in yield and resource efficiency [4,10,12,28], and AI-assisted extensions have added fault detection [13], nutrient optimization [5], and low-cost sensing on commodity hardware [29,30]. These systems establish that reliable autonomous control is achievable on affordable hardware. While their primary focus is on sensing and actuation, the translation of raw telemetry into natural-language cultivation guidance remains an open challenge, particularly for non-expert operators [6,14]. The proposed system builds on this closed-loop foundation and adds a conversational advisory layer to address this gap.

2.2. LLM- and RAG-Based Agricultural Advisory

Large language models grounded through retrieval-augmented generation (RAG) have emerged as a means of delivering domain-specific advice while mitigating hallucination [15,16]. RAG-based advisory frameworks anchor model responses in curated agronomic corpora, improving factual reliability over vanilla LLMs [17,18,22]. Beyond static retrieval, a smaller body of work applies agentic designs, in which the LLM autonomously routes inputs, selects tools, and reasons over multiple steps. CottonBot [19] couples an RAG-augmented LLM with agentic tool invocation and IoT soil-moisture and weather sources to produce irrigation recommendations for outdoor cotton, and PEZEGO [20] pairs LLM reasoning with IoT telemetry for pest management. At a larger scale, Pardo-Pina et al. [21] propose a multi-agent service architecture for smart greenhouses with RAG-grounded LLM agents performing telemetry analytics and decision support. These systems demonstrate the value of integrating RAG and agentic reasoning for agricultural advisory systems. However, none incorporates image-grounded disease diagnosis alongside text-based advisories within a single pipeline. The present work extends this direction by combining dual-modality routing (text and image), intent classification, and output validation within a single pipeline, and by adding multimodal disease diagnosis as a complementary capability alongside cultivation advisory systems, deployed entirely on-device without external API dependency.

2.3. Vision-Based Plant Monitoring and Disease Diagnosis

Deep learning has become the standard for plant monitoring and disease recognition. Convolutional detectors and segmentation models achieve strong performance in disease classification [23,24,31], single-stage YOLO detectors have been applied to plant and crop localization across growth stages [32,33], and transformer-based models that fuse in situ sensor data with plant images further improve diagnostic accuracy under field conditions [25]; broader reviews document the rapid uptake of deep learning for plant-stress imaging [34]. More recent work explores LLM-driven knowledge representation for plant disease, such as knowledge-graph construction from the disease literature [26]. These approaches are primarily designed for recognition tasks and operate independently of cultivation control systems. In the proposed system, YOLO11 detection serves a dual role, localizing the plant for growth-stage classification and for region-constrained browning analysis, while a vision-language model makes diagnosis accessible through the same conversational interface that delivers cultivation advice.

2.4. Synthesis and Research Gap

Table 1 summarizes the preceding review along five capability dimensions that a non-expert-accessible indoor farming platform would ideally satisfy: autonomous closed-loop control, RAG-grounded advisory, agentic routing, multimodal image diagnosis, and cloud-free on-device operation. Existing systems each address a subset. Control-oriented IoT platforms provide autonomous actuation but no advisory intelligence; RAG and agentic advisory systems provide language-based guidance but omit multimodal diagnosis and depend on cloud infrastructure; and vision-based diagnosis models provide image understanding in isolation from control and advisory delivery. No prior system spans all five dimensions within a single deployment. The present work targets exactly this gap, integrating threshold-based control, agentic RAG-grounded advisories, and multimodal symptom interpretation on commodity hardware without cloud dependency, which is the integration this paper claims as its contribution.

A recurring constraint in this literature is that detection models deployed on low-power IoT nodes are typically restricted to compact architectures such as Tiny-YOLO in order to remain within the compute budget of the edge device, which limits achievable detection accuracy. The present work addresses this constraint through the placement of the inference workload rather than through further model compression: the detection model executes locally on a Raspberry Pi 5 acting as a supervisory node above the ESP32 sensing layer, at a five-second inference interval, so that a mid-capacity detector can be used without cloud offloading while the microcontroller retains responsibility only for deterministic threshold actuation.

3. Materials and Methods

3.1. System Architecture and Design

3.1.1. Overall System Architecture

The proposed system integrates three distinct architectural tiers into a unified indoor farming platform: (i) a physical sensing and control layer responsible for environmental sensing and automated actuator control, (ii) a backend AI intelligence layer running on the Raspberry Pi 5 comprising a FastAPI server, a YOLO11m plant-monitoring module, and a structured local data logging system, and (iii) a frontend layer hosted on a laptop comprising the Flowise-based agentic chatbot pipeline, Ollama LLM inference engine, and a React 19.2.0/Vite 7.2.5 web application accessible to the end user via a local network browser. Figure 1 illustrates the data-flow relationships among these three layers.

Figure 1. Three-layer system architecture of the proposed controlled indoor farming platform, comprising the physical sensing and control layer (ESP32, sensors, actuators), Raspberry Pi 5 backend AI intelligence layer, and frontend layer (Flowise agentic chatbot, Ollama LLM inference, and web application). Dashed borders group components within each layer; bidirectional arrows indicate two-way data/command exchange, both between layers (via UART serial and REST API/Wi-Fi, as labeled) and among components within the same layer.

Data flow within the system follows a bidirectional closed-loop pattern. The ESP32 microcontroller periodically samples environmental sensors and transmits structured readings to the Raspberry Pi 5 via USB serial (UART) communication. The Raspberry Pi evaluates incoming data against predefined thresholds and issues actuation commands back to the ESP32 through the same serial channel, closing the control loop autonomously. Concurrently, the Raspberry Pi backend exposes RESTful API endpoints over the local WiFi network that the laptop-hosted frontend queries for dashboard updates, enabling real-time monitoring and manual control. The agentic chatbot pipeline runs as a separate service on the laptop, operating independently from the sensor data pipeline and communicating with the frontend over the same local network to provide cultivation advisory and symptom-level diagnostic support through a conversational interface embedded within the web application.

3.1.2. Physical Sensing and Control Layer

The physical layer comprises the indoor farming enclosure together with all embedded sensing, actuation, and computation hardware. The enclosure houses Cos lettuce plants in a soil-based growing medium and provides a controlled environment through overhead LED grow lighting and a submerged drip-irrigation pump. All electrical components are housed within enclosed waterproofed casings to protect against moisture and dust inherent to the indoor cultivation environment.

Two microcontrollers implement a hierarchical control structure. The ESP32 operates as a dedicated low-level controller, executing sensor data acquisition, relay switching, and a hardware watchdog routine. If serial communication with the Raspberry Pi 5 is interrupted for longer than a defined timeout, the ESP32 automatically deactivates all relay outputs, preventing unintended continuous pump or light operation. The Raspberry Pi 5 serves as the supervisory processing unit, executing the backend server, AI inference pipeline, and data logging service. Communication between the two microcontrollers is conducted over USB serial (UART) at fixed intervals, with structured data packets carrying sensor readings and acknowledgment flags in both directions.

The sensing suite monitors two key cultivation variables that drive automated control. Soil moisture is sampled by a resistive dual-electrode probe inserted at root depth and read through the ESP32 analog input on pin D34. The 12-bit ADC reading (0–4095) is mapped linearly to a 0–100% scale for display and for threshold logic, giving a relative moisture index specific to this probe, substrate, and placement. The irrigation setpoint on this index was established in consultation with the growers at the collaborating farm, who assessed substrate condition directly during commissioning and identified the point at which irrigation should be triggered for this crop and growing medium. The Raspberry Pi compares each reading against this threshold and triggers an irrigation pulse when the index falls below it, then monitors recovery until an upper threshold is reached before deactivating the pump. Ambient illumination is monitored by a BH1750 digital light sensor read over I²C on remapped pins (SDA on D18, SCL on D22, as D21 is reserved for relay control). Actuation is driven by two digital outputs: D19 switches relay 1 for the water pump and D21 switches relay 2 for the LED grow light. Table 2 summarizes the key hardware components together with their specifications, stated accuracies, and interface assignments, and Figure 2 presents the corresponding wiring schematic.

Table 2. System components, specifications, and interface assignments.

Component	Model/Type and Range	Manufacturer (Country)	Accuracy	Interface (ESP32 Pin)
Light sensor	BH1750FVI digital ambient light sensor; 1–65,535 lx	ROHM Semiconductor (Kyoto, Japan)	±20% of reading, 1 lx resolution	I2C (D18 SDA, D22 SCL)
Soil moisture sensor	Resistive dual-electrode probe with comparator module, 3.3–12 V DC, <20 mA; uncalibrated analog output mapped linearly from the 12-bit ADC range to a 0–100% relative index	Generic module	Not specified (uncalibrated resistive probe; used as a relative threshold index)	Analog input (D34)
Relay 1	Optocoupler-isolated, normally open	Generic	—	Digital output (D19), switches water pump
Relay 2	Optocoupler-isolated, normally open	Generic	—	Digital output (D21), switches grow light
Microcontroller	ESP32 dual-core MCU, Wi-Fi/BT	Espressif Systems (Shanghai, China)	—	USB serial (UART) to Raspberry Pi 5
Edge computer	Raspberry Pi 5, quad-core, 4 GB RAM	Raspberry Pi Ltd. (Cambridge, UK)	—	Hosts FastAPI backend, YOLO11m inference, data logging
Camera	HD 1080p USB webcam, 30 fps	Generic	—	USB-A to Raspberry Pi 5
Water pump	12 V DC submersible	Generic	—	Relay 1 controlled
LED grow light	Full-spectrum, mains powered	Generic	—	Relay 2 controlled
Laptop	Asus TUF Gaming A15 FA507UV	ASUSTeK Computer Inc. (Taipei City, Taiwan)	—	Hosts Flowise, Ollama (Mistral 7B, LLaVA 7B), React/Vite web application

Figure 2. Wiring schematic of the control subsystem. Mains supply feeds 12 V and 5 V DC converters powering the relay modules and the Raspberry Pi 5. The ESP32 reads the BH1750 light sensor over I²C (SDA D18, SCL D22) and the soil moisture probe on analog input D34, and switches the water pump and grow light through optocoupler-isolated relays on D19 and D21. Both the ESP32 and the USB camera connect to the Raspberry Pi 5 over USB.

3.1.3. Backend AI Intelligence Layer

The backend AI intelligence layer runs on the Raspberry Pi 5 and is composed of three integrated components: a FastAPI server, a local data logging module, and a YOLO11m plant-monitoring module. This on-device architecture leverages edge computing to eliminate cloud dependency, preserving low-latency control response and data privacy.

FastAPI Server and Data Logging Module

The backend server is implemented in Python using the FastAPI framework, which exposes RESTful API endpoints following the client-server architectural pattern. Defined endpoints handle sensor data ingestion from the ESP32, device control command dispatch, and AI processing requests. All incoming sensor readings are validated using Pydantic data models before being logged locally on the Raspberry Pi 5. Telemetry fields recorded at each polling cycle include: timestamp, soil moisture percentage, grow light state, plant detection stage, and plant health alarm status. This data store serves as a historical log for trend visualization on the frontend dashboard and supports retrospective assessment of cultivation performance.

Plant Monitoring Module (YOLO11m and Computer Vision)

Plant health monitoring is performed by a YOLO11m object detection model running locally on the Raspberry Pi 5. The 1080p webcam connected via USB captures frames at defined intervals, which are passed to the YOLO11m inference pipeline to detect plant regions and classify growth-stage categories (small, medium, and large). A supplementary computer vision post-processing step computes a browning ratio from the detected plant region by analyzing the proportion of discolored pixels relative to the total detected leaf area. When the browning ratio exceeds a configurable threshold, the system generates a health alert that is logged locally and surfaced to the user on the monitoring dashboard. The model was trained on a custom dataset of 301 annotated Cos lettuce images collected across multiple growth stages. After an image-level split, spatial and lighting augmentation was applied to the training partition only, yielding 964 images (training 772, validation 145, test 47); the validation and test partitions received auto-orientation only.

3.1.4. Frontend Layer

The frontend layer is hosted on a laptop connected to the same local WiFi network as the Raspberry Pi 5. This separation of the AI inference workload from the Raspberry Pi preserves backend responsiveness for real-time sensor processing and control, while enabling the computationally intensive LLM inference to run on more capable consumer hardware. The frontend layer comprises two tightly integrated components: the agentic chatbot pipeline and the web application.

Agentic Chatbot Pipeline

The advisory module is designed as an agentic pipeline rather than a single-shot RAG query. Following the characterization of agentic AI as systems that autonomously route inputs through condition-based pipelines, classify intent, invoke external tools, and reason over multiple steps as part of a single response chain [19,21], the pipeline exhibits four agentic properties: (i) condition-based routing that dispatches inputs to distinct processing paths by modality; (ii) intent classification that conditions subsequent tool selection; (iii) tool invocation that chooses between vector retrieval and external HTTP data sources at run time; and (iv) an LLM-based validation step that gates the generated response before delivery. These properties distinguish the pipeline from a static retrieve-then-generate design and provide the multi-step, context-aware behavior that the evaluation in Section 4 assesses.

The chatbot module is implemented using Flowise as a workflow orchestration framework in combination with Ollama for local LLM inference, enabling the entire pipeline to operate without external API subscriptions. The pipeline adopts a dual-path agentic architecture governed by a Condition Agent (Mistral 7B) that autonomously routes incoming user inputs based on input type: text-only queries are directed to a text advisory path, while queries containing image attachments are directed to a vision-based symptom-interpretation path.

For the text advisory path, an Intent Router (LLM) first classifies the user query before passing it to a secondary Condition Agent that determines whether to invoke an RAG retrieval node or an HTTP-based external data node. The RAG knowledge base is constructed from domain-specific cultivation documents including Cos lettuce growing guidelines, pest identification references, and nutrient deficiency manuals, chunked into fixed-length segments with overlapping windows, embedded using a local embedding model, and indexed in a PostgreSQL vector database via the pgvector extension for efficient semantic similarity search. At inference time, the top-k semantically nearest document chunks are retrieved and injected into a structured prompt template before being passed to Mistral 7B for response generation. An LLM Checker node then validates each generated response against predefined quality conditions before final delivery, reducing the risk of non-compliant or unreliable recommendations.

For the image-based symptom-interpretation path, the user uploads a plant photograph through the web interface alongside an optional text description. The input is first processed by an LLM Vision node (LLaVA 7B) before passing through its own Intent Router and Condition Agent, following the same RAG or HTTP branching logic. The LLaVA 7B model identifies visible symptoms such as leaf discoloration, necrotic spots, wilting, or pest damage, and generates a natural-language diagnostic report including observed symptoms, possible causes, recommended actions, and environmental factors to monitor. This multimodal capability enables non-expert users to receive actionable disease guidance without requiring specialist agronomic knowledge.

The dual-path routing, tool selection, and validation logic described above are formalized in Algorithm 1, and the corresponding data flow is illustrated in Figure 3. Given a user message, the pipeline first branches on the presence of an image: image inputs are passed to the LLaVA 7B vision node for symptom extraction and composed with any accompanying text, while text-only inputs proceed directly. The resulting query is then classified by the Intent Router, a Condition Agent selects between RAG retrieval and an HTTP data tool, the selected context is injected into a structured prompt for Mistral 7B, and the LLM Checker validates the generated response before delivery.

Algorithm 1 AgenticDispatch—dual-path routing and validated response generation
Require: User message $m=(t,\mathit{img})$ with text t and optional image $\mathit{img}$; RAG store $\mathcal{K}$; HTTP tool set $\mathcal{H}$ Ensure: Structured advisory response r
1: if $\mathit{img}\ne ⌀$ then   2:     s ← LLaVA_Vision(img, t)   3:     q ← Compose(t, s)   4: else   5:     $q\leftarrow t$   6: end if   7: intent ← IntentRouter(q)   8: route ← ConditionAgent(intent, q)   9: if route = Rag  then 10:     $\mathcal{C}\leftarrow$ TopK(Retrieve)$(q,\mathcal{K})$ 11:     ctx ← PromptTemplate $(q,\mathcal{C})$ 12: else 13:     ctx ← PromptTemplate(q, HttpFetch)$(q,\mathcal{H})$) 14: end if 15: r ← Generate(Mistral7B, ctx) 16: if ¬ LLMChecker(r, q) then 17:     r ← Revise(r, q) 18: end if 19: return r	▹ image present: vision path ▹ visual symptom extraction ▹ text-only path ▹ query intent classification ▹ select tool ▹ pgvector semantic search ▹ structured generation ▹ validation gate

Figure 3. Agentic chatbot dual-path pipeline. A Condition Agent routes each input by modality; image inputs pass through the LLaVA 7B vision node before rejoining the shared advisory path of intent classification, tool selection (RAG or HTTP), Mistral 7B generation, and LLM Checker validation.

• Design rationale. Three design choices in this pipeline warrant justification. First, a dual-path architecture governed by an explicit Condition Agent is preferred over a single unified prompt because text and image queries impose different processing requirements: routing by modality invokes the comparatively expensive vision stage only when an image is present, conserving compute on the consumer-grade host, and prevents text queries from being forced through an image-conditioned path. Second, agentic routing with intent classification and run-time tool selection, rather than a fixed retrieve-then-generate chain, allows the system to ground different query types appropriately, retrieving from the domain corpus when internal knowledge suffices and invoking external HTTP data when it does not, which improves the relevance of the injected context. Third, the LLM Checker validation step is included because the system advises non-expert users on crop decisions, where a confidently incorrect recommendation is more harmful than a non-answer; gating responses on validation directly targets this risk and is reflected in the zero incorrect-response rate reported in Section 4. Finally, on-device inference with 7B-parameter models (Mistral 7B and LLaVA 7B) served through Ollama eliminates per-query API cost and cloud dependency and keeps cultivation data local, matching the connectivity- and budget-constrained conditions of small-scale and urban deployment while remaining within the memory budget of the laptop host.

• Comparison of candidate configurations. Four pipeline configurations were considered during design, and each fell short of at least one requirement of the final architecture. A base language model without retrieval has the lowest latency and complexity, but cannot be grounded in the cultivar-specific documentation on which the advisory function depends, and its outputs cannot be traced to a source. A standard retrieve-then-generate configuration supplies that grounding, but applies one fixed retrieval path to every query regardless of intent, and provides no pathway by which an image can enter the context. An agentic configuration without the validation gate adds condition-based routing and intent-dependent tool selection, addressing both limitations, but delivers whatever the generator produces. That is the risk this application cannot carry: the system advises non-expert users on crop decisions, where a confidently incorrect recommendation acted upon costs more than a response that declines to commit. The full pipeline adopted here retains that routing and grounding and adds the validation stage, and the absence of incorrect responses across the 50-case evaluation in Section 4 is consistent with that stage performing its intended function.

Two components cannot be removed without the system failing its stated requirements, and are therefore not refinements whose contribution ablation could quantify. Without the dual-path router, image-bearing queries would not be answered less accurately but would not be admitted at all, leaving the measurement undefined rather than degraded. Without retrieval grounding, traceability to source documents is not reduced but eliminated. An earlier text-only configuration was developed and was functional for text queries; the move to the present dual-path architecture followed the requirement to accept image input, not any deficiency in text handling. A restricted ablation on the text-only subset would not be representative either, since only 11 of the 50 queries are text-only and these fall disproportionately in the watering diagnosis, pest-detection, and knowledge categories, while health assessment and disorder interpretation are almost entirely image-bearing.

The validation stage is the component whose marginal contribution could meaningfully be isolated, since removing it leaves the system functionally complete across both modalities. A controlled comparison under identical query sets and fixed decoding parameters was not performed here and is identified as a priority in Section 4.

Web Application

The user-facing web application is developed using Vite and React, hosted on the laptop and accessible via a local network browser. The application communicates with the Raspberry Pi 5 FastAPI backend exclusively through RESTful API requests over the local WiFi network. The interface is designed to be accessible to both technical and non-technical users, with a simplified layout that clearly separates monitoring from control functions.

Upon first access, users proceed through an onboarding flow that introduces the system capabilities before reaching the login and account registration screen. Following authentication, users are directed to the main dashboard, which presents four principal functional panels: (i) a real-time monitoring panel displaying current soil moisture and grow light status; (ii) a device control panel providing toggle switches for manual override of the irrigation pump and grow light, including configurable pump run durations and a grow light scheduling interface; (iii) a plant analysis panel displaying the most recent YOLO11 detection result, browning risk, and AI confidence score; and (iv) a chatbot interaction panel through which users submit text queries and image uploads to receive cultivation advisory and symptom-level diagnostic responses. The Flowise chatbot is integrated directly into the web application via an embedded HTML interface, enabling direct access to the agentic pipeline without navigating to a separate service. Figure 4 shows the resulting interface, with the plant health monitoring panel, the actuator control and scheduling dashboard, and the embedded conversational advisory panel.

Figure 4. Web application interface displaying the plant health monitoring panel with YOLO11m confidence score, automated water and light control dashboard with scheduling, and embedded agentic chatbot providing multimodal root health diagnosis from a user-submitted image.

3.1.5. Fail-Safe and Safety Design

The system incorporates multiple layers of fail-safe logic to ensure continuous safe operation across both hardware and software levels. At the hardware level, the ESP32 implements a watchdog timer that automatically reboots the microcontroller if no heartbeat signal is detected within a five-second interval, preventing system lockup during software failure. All relay outputs are set to the off state by default at startup and revert to off when no valid command is received from the Raspberry Pi, preventing actuators from remaining permanently energized during a supervisory failure or communication loss. Optocoupler relays provide complete electrical isolation between the 3.3 V microcontroller logic and the higher-voltage actuator circuits, protecting both the sensing electronics and the end user.

At the software level, the Raspberry Pi backend incorporates a soft restart mechanism that refreshes the system every 24 h cycle to prevent memory accumulation during continuous operation. Runtime cycle counts and system heartbeat are continuously tracked; if a cycle anomaly is detected, a restart command is issued to the ESP32 to reinitialize the communication channel. An auto-shutdown function prevents overwatering by disabling the irrigation pump when no actuation command is received within a defined timeout interval.

The system additionally implements critical error detection logic that monitors for abnormal operating conditions including plant health deterioration and repeated unsuccessful actuation cycles. To prevent erroneous notifications, a false positive safeguard confirms the presence of an error condition across multiple consecutive monitoring frames before triggering an alert, ensuring that transient detection failures do not generate unnecessary user notifications.

3.1.6. Separation of the Control and Advisory Layers

The control and advisory layers are deliberately isolated, and this separation is a design decision rather than an incidental property of the implementation. Irrigation and lighting are actuated exclusively by the ESP32 against fixed thresholds evaluated on the current sensor reading. The agentic LLM pipeline receives no live telemetry, adjusts no control parameters, and issues no actuation commands. It operates instead on stored telemetry and captured images, allowing the user to query recorded conditions and observed symptoms and to receive guidance, while leaving execution of any recommended action to the operator.

The rationale is bounded failure behavior. A language model produces stochastic output, and an actuation path depending on such output admits a failure mode the safety mechanisms described above cannot bound: the watchdog timer, default-off relay state, and actuation timeout all detect the absence or interruption of a valid command, but none can detect a syntactically valid command that is agronomically wrong. Confining actuation to deterministic threshold logic keeps every command reaching a relay traceable to a sensor reading and a configured setpoint. Figure 2 shows this directly: the actuation path runs from the ESP32 digital outputs through the relay modules to the pump and grow light, with no control path from the Raspberry Pi 5, which hosts the advisory pipeline, to any actuator. The system is accordingly described throughout as threshold-based automated control operating alongside an independent agentic AI advisory layer.

3.2. Materials and Experimental Methods

3.2.1. Cultivation Environment and Plant Material

All experiments were conducted within a purpose-built indoor farming system constructed from a plastwood outer frame measuring 65 cm (width) × 40 cm (depth) × 75 cm (total height), divided into two compartments: an upper plant chamber (65 cm × 40 cm × 60 cm) and a lower electronics drawer (65 cm × 40 cm × 15 cm). Transparent acrylic panels formed the viewing sides of the plant chamber, enabling visual monitoring without opening the system. All electrical components were secured within waterproofed enclosures to protect against moisture and dust from the cultivation environment. Cos lettuce (Lactuca sativa var. longifolia) was selected as the study crop on the basis of its short growth cycle, high sensitivity to environmental variation, and relevance to urban indoor farming. Seedlings sourced from a commercial nursery were transplanted into the system on the first day of the experiment and cultivated under the system’s automated control throughout the cultivation period. The lighting schedule was configured through the web application interface to provide 12 h of illumination per day followed by a 12 h dark period, with the grow light operating from 06:00 to 18:00 daily throughout the cultivation period.

A three-layer soil medium was formulated to ensure adequate moisture retention while maintaining sufficient drainage to prevent root rot under the intermittent drip-irrigation regime. The bottom layer consisted of lightweight expanded clay aggregate (LECA), selected for its high drainage capacity to prevent waterlogging in the enclosed system. The middle layer comprised coconut husk, serving as a buffer between the topsoil above and the drainage aggregate below. ’The top layer combined topsoil with coco coir (coconut fiber) as the primary rooting medium, with the coco coir component improving drainage and aeration within the root zone.

3.2.2. Automated Sensing and Control Protocol

Environmental conditions inside the enclosure were monitored and controlled automatically throughout all experimental periods. The ESP32 microcontroller polled all sensors at five-second intervals and transmitted structured data packets to the Raspberry Pi 5 over the USB serial (UART) interface. On receipt, the Raspberry Pi evaluated each reading against predefined control thresholds and issued actuation commands back to the ESP32 as required.

Irrigation was managed by a threshold-based hysteresis controller. When the soil moisture percentage fell below the lower irrigation threshold, the Raspberry Pi commanded the ESP32 to activate the 12 V submersible water pump via the optocoupler relay; the pump remained active for approximately seven seconds or until soil moisture recovered to the upper threshold, at which point the relay was deactivated. This hysteresis approach prevented rapid pump cycling and allowed soil moisture to stabilize between irrigation events. The LED grow light was activated and deactivated according to a user-configured photoperiod schedule implemented in the Raspberry Pi control loop, with start and end times adjustable through the web application interface. For this experiment, the schedule was set to 06:00–18:00, providing a 12 h on/12 h off photoperiod. The BH1750 lux sensor provided continuous confirmation that the light was functioning within its expected operating range. All sensor readings were timestamped and logged locally at each five-second polling cycle, including soil moisture percentage, light state, plant stage classification detected by YOLO11m, and alarm status. Table 3 summarizes the key operational timing parameters, determined empirically to balance system responsiveness against processing load on the Raspberry Pi 5.

Table 3. Operational timing parameters.

Operation	Interval/Setting
Operation cycle duration	24 h
Plant detection (YOLO11m) inference interval	Every 5 s
Sensor data logging interval	Every 5 s
Webcam image storage interval	Every 15 min
Plant anomaly alert delay	3 min after detection event
Irrigation cooldown period	10 min minimum (moisture-dependent)
Automatic pump shutoff timeout	7 s
Automatic grow-light shutoff timeout	1 h

3.2.3. YOLO11m Plant Monitoring Model

Dataset Collection and Annotation

A custom image dataset was collected during a field visit to Hydro Green Garden Farm, a commercial hydroponic greenhouse, to capture real-world variation in Cos lettuce appearance across growth stages, lighting conditions, and viewing angles, separately from the deployment trial. The photographed subjects were commercial growing beds in which many plants appear within each frame, and every plant visible in a frame was annotated individually, so annotated plant instances substantially exceed image counts in every partition.

A total of 301 images were collected and manually annotated with bounding boxes across three growth-stage classes. Because each frame contains multiple plants, individual annotation yielded 2222 plant instances in total (Small: 1392; Medium: 714; Large: 116), an average of more than seven annotated plants per image. Class assignment followed morphological criteria applied consistently during annotation: Small denotes the seedling stage, with cotyledons and first true leaves present and no head formation; Medium denotes mid-growth, with an expanded rosette and no compact head; and Large denotes the near-harvest stage, with a compact elongated head consistent with the romaine maturity indices of Cantwell and Suslow [35]. Because growth is continuous rather than discrete, the boundary between adjacent classes is a matter of annotator judgment, which is reflected in the class-level results in Section 4.

Assembly and annotation were performed on the Roboflow platform, which also applied auto-orientation to normalize image rotation. Images were assigned to the training, validation, and test partitions at the image level and spatial and lighting augmentation was applied to the training partition only, yielding 964 images; the validation and test partitions received auto-orientation only and were not augmented, and no augmented variant of a held-out image appears in training. Image-level partitioning is the convention adopted in detection benchmarks in agricultural computer vision, where the unit of prediction is a bounding box rather than a plant. Partition sizes are given in Table 4.

Table 4. YOLO11m dataset split and preprocessing configuration.

Split	Images	Proportion (%)	Preprocessing
Training	772	80.1	Auto-orientation; spatial augmentation (flip, crop, rotate); lighting augmentation (brightness, exposure)
Validation	145	15	Auto-orientation only
Test	47	4.9	Auto-orientation only
Total	964	100	—

Spatial augmentation comprised horizontal flipping, random cropping (up to 20% of image area), and rotation in the range of $-15°$ to $+15°$. Lighting augmentation included random brightness adjustments ($-15\%$ to $+15\%$) and exposure variation ($-10\%$ to $+10\%$), applied exclusively to the training split to improve model tolerance to illumination variation within the enclosure.

Model Training

The YOLO11m variant of the YOLO11 family was used, comprising 126 layers, 20,032,345 parameters, and 67.7 GFLOPs at the training input resolution. The original architecture was used without structural modification; training was confined to transfer learning from the pretrained COCO weights onto the custom Cos lettuce dataset. The medium variant was selected because inference is executed on the Raspberry Pi 5 supervisory node rather than on the microcontroller, so its additional capacity is affordable within the latency requirement of the monitoring loop.

Training was performed using the Ultralytics framework on Google Colab with an NVIDIA Tesla T4 GPU. The hyperparameters are given in Table 5. Training converged at epoch 115, at which point the best weights were saved. Convergence was monitored via box loss, classification loss, and distribution focal loss for both the training and validation partitions across epochs. The trained weights were transferred to the Raspberry Pi 5 for local real-time inference. During system operation the model ran on the live 1080p webcam feed at five-second intervals for continuous detection, while images were stored to disk at the separate fifteen-minute interval used for browning analysis (Table 3).

Table 5. YOLO11m training hyperparameters.

Parameter	Value	Description
Epochs	180	Total training iterations
Image size	$640\times 640$ px	Input image resolution
Batch size	16	Images per training step
Optimizer	AdamW	Adaptive optimizer with decoupled weight decay
Learning rate	0.001	Initial learning rate
Weight decay	0.0005	L2 regularization to prevent overfitting
Cosine LR	True	Cosine annealing for learning rate decay
Patience	60	Early stopping threshold

The software environment comprised Python 3.12.13, Ultralytics 8.4.51, and PyTorch 2.10.0 with CUDA 12.8. Dataset preparation, annotation, and augmentation were performed on the Roboflow platform. TensorFlow and Keras were not used in this work.

Detection Performance Metrics

Detection performance is reported using the standard object-detection metrics. A predicted box is counted as a true positive (TP) when its intersection over union with a ground-truth box of the same class meets the stated threshold; a predicted box with no matching ground truth is a false positive (FP), and an unmatched ground-truth box is a false negative (FN). Precision (Pr) and recall (Re) are defined in Equations (1) and (2), and the F1 score as their harmonic mean in Equation (3):

$$\mathrm{Pr}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(1)

$$\mathrm{Re}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(2)

$$\mathrm{F}1=2·{\displaystyle \frac{\mathrm{Pr}·\mathrm{Re}}{\mathrm{Pr}+\mathrm{Re}}}$$

(3)

Average precision (AP) for a class is the area under its precision–recall curve, and mean average precision (mAP) is the mean of AP over the N classes, as given in Equation (4):

$$\mathrm{mAP}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\mathrm{AP}}_i$$

(4)

mAP@50 evaluates AP at a fixed intersection-over-union threshold of 0.50, while mAP@50–95 averages AP over thresholds from 0.50 to 0.95 in steps of 0.05 and is therefore the stricter localization criterion. These metrics characterize detection performance on the validation partition and are reported separately from the runtime detection continuity measure defined in Section 3.2.6.

Browning Detection via Computer Vision

A supplementary computer vision routine was implemented on the Raspberry Pi 5 to flag abnormal leaf coloration for operator attention. The routine is an operational alerting mechanism rather than a physiological classifier: its output is a notification threshold on an image-derived color statistic, not a determination of plant health status. The disorder it targets is documented in the postharvest literature for this crop. Marginal browning and tipburn are established physiological disorders in which damaged leaf margins weaken and become susceptible to decay [35], and leaf discoloration correlates negatively with the normalized difference vegetation index, an established indicator of plant health status, under controlled lighting [36]. The threshold was set to meet an operational requirement specified by the collaborating farm, whose growers identified early detection of leaf browning as the point at which corrective action remains effective. At each scheduled capture interval, the webcam acquires an image of the Cos lettuce plant. The YOLO11m model first locates the plant region within the frame, after which the cropped region is converted to the HSV color space. A color-range threshold filter identifies pixels falling within the hue range associated with brown or yellowed leaf tissue, and the browning ratio is computed as the proportion of discolored pixels relative to the total pixels within the detected plant region. Constraining color analysis to the YOLO11m-detected region follows the same principle used to separate crop pixels from a visually similar background in prior detection work, where localization precedes color discrimination to avoid background bias [33]. When the browning ratio exceeds a predefined threshold, the system triggers a user notification and logs the incident including the flagged plant image, current growth stage classification, and sensor data at the time of detection. The logged incident data can subsequently be submitted by the user to the agentic chatbot for further symptom-level diagnostic guidance.

3.2.4. RAG Knowledge Base Construction

The RAG knowledge base underpinning the text advisory path of the agentic chatbot was constructed from a curated collection of domain-specific cultivation documents. Source materials included Cos lettuce growing guides, nutrient management references, common indoor crop pest and disorder identification manuals, and general indoor farming best-practice documents. All documents were processed using the Flowise document loader and chunked into fixed-length text segments with an overlapping window to preserve contextual continuity across chunk boundaries. Each chunk was embedded using the local embedding model within the Ollama deployment on the laptop host and indexed in a PostgreSQL vector database via the pgvector extension for efficient semantic similarity search. At query time, the retriever performed a cosine-similarity search over the vector database to return the top-k most semantically relevant chunks, which were injected into a structured prompt template before being passed to the Mistral 7B inference engine. A system prompt was configured within the Flowise pipeline to instruct the model to analyze visible plant symptoms and environmental conditions, structuring responses around four diagnostic dimensions: observed symptoms, possible causes, recommended actions, and environmental factors to monitor.

3.2.5. Chatbot Advisory Evaluation Protocol

Test Set Design

Chatbot advisory performance was evaluated using a set of 50 plant-related test cases constructed to cover the principal categories of user queries anticipated in the indoor farming context. The test cases spanned six query categories: plant health assessment, disease and browning diagnosis, watering diagnosis, pest detection, root-health assessment, and growth-stage and general knowledge queries. Each test case consisted of a written user query submitted to the chatbot through the web interface under representative operating conditions. For cases requiring symptom interpretation, a reference plant photograph was included alongside the text query to engage the LLaVA 7B multimodal path. All queries were administered by the same evaluator to ensure consistency of phrasing input.

Weighted Ground-Truth Evaluation Method

Chatbot responses were evaluated using a three-level weighted ground-truth scoring scheme. Given the open-ended nature of cultivation advisory queries, this partial-credit approach was adopted in preference to binary accuracy, consistent with weighted scoring methods used in LLM assessment for domain-specific agricultural advisory tasks [19]. Each response was independently assessed against a reference answer prepared from established plant pathology, postharvest, and crop quality documentation for lettuce, comprising the lettuce disease and pest management guidance of Turini [37], the physiological disorder and postharvest quality recommendations for romaine lettuce of Cantwell and Suslow [35], the review of bioactive compounds and pre- and postharvest quality factors in lettuce of Yang et al. [38], and the controlled-lighting growth and quality study of Lee et al. [36]. These documents are external to the retrieval corpus used at inference and therefore constitute an independent basis for scoring rather than a restatement of the system’s own source material. Each response was classified into one of three outcome categories: Correct (score $=1$), if the response fully and accurately addressed the query with no materially misleading information; Partially Correct (score $=0.5$), if the response captured the core intent of the query but lacked precision, omitted key details, or included minor inaccuracies; and Incorrect (score $=0$), if the response failed to address the query or contained factually incorrect information. Weighted accuracy was computed according to Equation (5):

$$\mathrm{Weighted}\mathrm{Accuracy}={\displaystyle \frac{N_{\mathrm{correct}}\times 1+N_{\mathrm{partial}}\times 0.5}{N_{\mathrm{total}}}}$$

(5)

where $N_{\mathrm{correct}}$, $N_{\mathrm{partial}}$, and $N_{\mathrm{total}}$ denote the number of correct, partially correct, and total responses, respectively. This method provides a more nuanced performance measure than binary accuracy alone, acknowledging that partially correct responses retain practical utility for non-expert users even when they fall short of complete precision. This metric quantifies the agreement of each response with established reference guidance under the correct/partial/incorrect rubric; it characterizes the grounding and non-contradiction of the advisory output against the reference documentation rather than diagnostic precision at the pathogen or disorder level.

3.2.6. System Integration and Deployment

Following individual component validation, the complete integrated system was operated under real cultivation conditions throughout a 14-day Cos lettuce deployment period. During this period, all three functional layers: automated sensing and control, YOLO11m plant monitoring, and agentic chatbot interaction, operated concurrently without manual intervention. Detection and sensor telemetry were logged at five-second intervals throughout the experiment, yielding over 100,000 monitoring frames used to quantify runtime detection continuity. Operational continuity was assessed by monitoring runtime cycle counts recorded by the Raspberry Pi, the frequency of watchdog-triggered ESP32 restarts, and the continuity of the irrigation and lighting automation schedules. The experiment provided the dataset used for control performance analysis reported in Section 4 and validated the end-to-end data flow from physical sensors through to the user-facing web application and chatbot interface. Figure 5 illustrates the deployed system alongside the web application dashboard during active cultivation.

Figure 5. Deployed indoor farming system showing the Cos lettuce cultivation enclosure with automated irrigation and grow lighting, alongside the web application dashboard for real-time monitoring and control.

4. Results

4.1. Automated Control Performance

The hysteresis-based irrigation controller maintained soil moisture within the intended operational range throughout the experimental period without manual intervention. Figure 6 presents a representative 24 h soil moisture profile. During the grow-light ON period (0–12 h), the controller held the moisture index within the 70–73% range, while the subsequent dark period (12–24 h) shows steady values at approximately 81–82%. No instance of the index dropping to a critically low level or remaining continuously saturated was recorded, and the controller held it within a narrow band across both photoperiod phases. These values characterize the stability of the closed-loop controller rather than the agronomic adequacy of substrate water status, and indicate consistent automated actuation without oscillation and no sustained excursion toward either drought stress or root waterlogging. The LED grow light operated according to the predefined 12 h on/12 h off photoperiod schedule without deviation across all recorded cycles, with BH1750 lux readings consistently in the range of 55,000 to 56,000 lux during scheduled-on periods. No instance of the ESP32 watchdog triggering an emergency relay cutoff was recorded, confirming uninterrupted serial communication between the ESP32 and Raspberry Pi 5 throughout the operation period. Visible growth progress between transplanting and the end of the observation period (Figure 7) corroborated that the automated photoperiod schedule was adequate to sustain active vegetative growth of Cos lettuce.

Figure 6. Representative 24 h soil moisture profile recorded during the integration experiment.

Figure 7. Cos lettuce plant at transplanting (a) and at approximately 14 days of system-managed cultivation (b).

4.2. YOLO11 Plant Monitoring and Browning Detection

4.2.1. Detection Performance

This subsection reports two separate measurements. Runtime detection continuity describes how consistently the deployed system kept working; validation detection performance describes how accurately the trained model detects plants on the held-out validation partition. The two are reported separately because they support different claims.

• Runtime detection continuity. During the experimental period the deployed model returned a valid detection in 98.8% of over 100,000 frames captured at five-second intervals. This measures how consistently the monitoring loop operated without interruption under the fixed-camera, single-plant configuration of the deployment. It is not a measure of detection accuracy on new plants, camera positions, or cultivation systems, since every frame shows the same plant in the same scene. Figure 8 shows representative detection outputs on unseen test images for all three growth-stage classes, with confidence scores of 0.88, 0.94, and 0.91 for Small, Medium, and Large respectively.

• Validation detection performance. The trained model was evaluated separately on the validation partition of the commercial-greenhouse dataset introduced in Section 3.2 (Section Dataset Collection and Annotation), which contains multiple plants across three growth stages under varied illumination and viewing angles. On that partition the model reached mAP@50 of 0.971, mAP@50–95 of 0.856, precision of 0.948, recall of 0.926, and an F1 score of 0.937 (Table 6).

Figure 8. YOLO11m plant growth stage classification results on unseen test images, showing correct detection across all three classes: (a) small stage (confidence $=0.88$), (b) medium stage (confidence $=0.94$), and (c) large stage (confidence $=0.91$). Each bounding box indicates the detected plant region with its predicted class label and confidence score.

Figure 8. YOLO11m plant growth stage classification results on unseen test images, showing correct detection across all three classes: (a) small stage (confidence $=0.88$), (b) medium stage (confidence $=0.94$), and (c) large stage (confidence $=0.91$). Each bounding box indicates the detected plant region with its predicted class label and confidence score.

Table 6. YOLO11m model performance summary on the Cos lettuce dataset.

Metric	Value	Notes
Runtime detection continuity	98.8%	Operational availability measure over 100,000+ deployment frames; not a generalization metric
mAP@50 (validation set)	0.971	Commercial-greenhouse imagery, multiple plants and growth stages
mAP@50-95 (validation set)	0.856	Stricter localization criterion
Precision	0.948	Low false positive rate
Recall	0.926	High sensitivity across classes
F1 score	0.937	Excellent precision–recall balance

Table 6. YOLO11m model performance summary on the Cos lettuce dataset.

Metric	Value	Notes
Runtime detection continuity	98.8%	Operational availability measure over 100,000+ deployment frames; not a generalization metric
mAP@50 (validation set)	0.971	Commercial-greenhouse imagery, multiple plants and growth stages
mAP@50-95 (validation set)	0.856	Stricter localization criterion
Precision	0.948	Low false positive rate
Recall	0.926	High sensitivity across classes
F1 score	0.937	Excellent precision–recall balance

Per-class results follow the expected pattern. The Large class achieved the highest AP@50 (=$0.995$), consistent with its distinct near-harvest morphology, while Medium and Small reached 0.954 and 0.964 respectively, so all three classes exceeded 0.95. The normalized confusion matrix (Figure 9) illustrates class-level detection behavior at the default operating confidence threshold. The Large class was classified without error (1.00), while the Medium and Small classes reached 0.94 and 0.96 respectively. Residual misclassification occurred almost entirely between the Medium and Small classes, at 0.03 in each direction, with no confusion between non-adjacent growth stages: no Small instance was assigned to the Large class or the reverse. This pattern is consistent with growth stage being a morphological continuum rather than a set of discrete categories. False positive detections, shown in the background column, were assigned predominantly to the Medium and Small classes, the expected behavior in dense canopy imagery where numerous partially occluded plants appear within a single frame. The F1, Precision, Recall, and Precision–Recall curves per growth-stage class are presented in Figure 10. The F1–Confidence curve shows a peak F1 of 0.94 at a confidence threshold of 0.769, and the Precision–Recall curve confirms high area under the curve for all three classes (large 0.995, medium 0.954, small 0.964), collectively yielding an overall mAP@50 of 0.971.

Figure 9. Normalized confusion matrix of the YOLO11m model on the held-out validation set, showing classification performance across Large, Medium, Small, and background classes.

Figure 10. F1, Precision, Recall, and Precision–Recall curves of the YOLO11m model per growth-stage class, with overall mAP@0.5 = 0.971. The F1–Confidence curve shows peak F1 of 0.94 at confidence threshold 0.769, while the Precision–Recall curve demonstrates high area under the curve for all three classes (large = 0.995, medium = 0.954, small = 0.964).

4.2.2. Browning Detection

The HSV-based browning detection module was evaluated on 164 images captured over the observation period of continuous plant monitoring using a leaf-only analysis method, in which browning ratio computation was constrained exclusively to green leaf pixels identified within the YOLO11m-detected plant region, excluding soil, background, and non-leaf elements. Of these, 111 images (67.7%) fell below the alert threshold, with a mean browning ratio of 0.64% (SD $=0.51\%$), while 53 images (32.3%) exceeded it and raised an ALARM condition, with a mean browning ratio of 5.93% (SD $=3.04\%$) and a maximum observed browning ratio of 13.27%. The threshold was set at 2% of the analyzed leaf area during initial setup, in line with the operational requirement identified by the growers at the collaborating farm. These figures characterize the distribution of the browning statistic observed during deployment and the resulting alert frequency; they are not an accuracy measure against an independently established plant health status. The threshold is specific to the illumination configuration and cultivar used here and would require recalibration for other lighting conditions or crops. Figure 11a illustrates a representative result in which the browning statistic remained below threshold, displaying a GOOD status with a browning ratio of 0.6%. Figure 11b illustrates the system response under abnormal conditions, where a discolored pixel ratio of 2.1% exceeded the predefined threshold, triggering an ALARM status with the incident image, growth stage, and sensor data automatically logged for subsequent user review and chatbot-assisted diagnosis.

Figure 11. Browning detection results showing (a) normal plant condition with ‘GOOD’ status output and (b) abnormal condition exceeding the browning threshold triggering an alarm alert.

4.3. Agentic Chatbot Advisory Performance

4.3.1. Weighted Accuracy Results

The agentic chatbot achieved a weighted accuracy of 82.00% across 50 plant-related test cases, comprising 32 fully correct responses (64%), 18 partially correct responses (36%), and zero incorrect responses. The absence of incorrect responses indicates that the pipeline, as deployed, did not produce factually misleading outputs across the test set—a primary safety concern for a system guiding non-expert users. This outcome characterizes the complete pipeline and is not attributed to any single component. To identify where the advisory pipeline performs most and least reliably, the 50 test cases were disaggregated by query category. The per-category distribution of scores, the overall totals, and the reference documentation used to construct the ground-truth answer for each category are given in Table 7, and the complete case-by-case scoring is given in Table 8.

Table 7. Distribution of advisory scores across the 50 test queries by category and in total, with the reference documentation used to construct the ground-truth answer for each category.

Category	Cases	Correct	Partial	Incorrect	Weighted (%)	Reference
Health assessment	10	8	2	0	90.0	[35,36]
Disease/browning diagnosis	12	9	3	0	87.5	[35,37]
Watering diagnosis	8	6	2	0	87.5	[37,38]
Pest detection	5	4	1	0	90.0	[37]
Root-health assessment	7	4	3	0	78.6	[35,37]
Growth stage/knowledge	8	1	7	0	56.3	[36,38]
Overall	50	32	18	0	82.00	—

Table 8. Case-by-case advisory evaluation across the 50 test queries. Modality: T = text-only, M = multimodal (image-bearing). Score: 1 = correct, 0.5 = partially correct. No response was scored incorrect.

#	Query/Task	Modality	Score
Health assessment
1	Cos lettuce in system, healthy or abnormal	M	1
2	Lettuce in greenhouse row, healthy or abnormal	M	1
3	Green leaf lettuce close-up, healthy or abnormal	M	1
4	Basil plant, identify and describe growth condition	M	0.5
5	Young lettuce, is this plant in good condition	M	1
6	Vibrant lettuce, overall health check	M	1
7	Watering-can scene, is lettuce healthy and watered	M	1
8	Potted lettuce, general condition assessment	M	1
9	Hydroponic bed lettuce, health check	M	1
10	Basil growth-condition follow-up detail	T	0.5
Disease/browning diagnosis
11	Yellowing, browning and wilting lettuce, diagnosis	M	1
12	Dark-spot leaf, cause of browning	M	1
13	Severe browning lettuce head, cause analysis	M	1
14	Leaf spot with yellow margin, cause of browning	M	1
15	Edge browning, nutrient vs stress vs disease	M	1
16	Yellow leaves, cause and corrective action	M	1
17	Yellowing potted plants, cause and correction	M	1
18	Yellowing and curling lettuce, cause and correction	M	0.5
19	Dark spots on lettuce, disorder query	M	0.5
20	Brown crispy leaf, general abnormality check	M	1
21	Red-leaf chicory sample, disorder identification	M	0.5
22	Fungal versus bacterial browning, general explanation	T	1
Watering diagnosis
23	Stacked potted plants, over- or underwatered	M	1
24	Wilted lettuce in soil, over- or underwatered	M	1
25	Watering-can and lettuce, over- or underwatered	M	1
26	Wilted plants lying flat, over- or underwatered	M	1
27	Droplets on leaves, recent-watering assessment	M	0.5
28	Soil-moisture guidance for lettuce	T	1
29	Watering frequency for Cos lettuce	T	1
30	Drainage and waterlogging advice	T	0.5
Pest detection
31	Green aphids on leaf, signs of pest damage	M	1
32	Black aphids on leaf, signs of pest damage	M	1
33	Basal soft rot, distinguished as disorder not pest	M	1
34	Aphid control recommendation follow-up	T	1
35	Honeydew and sooty-mold risk explanation	T	0.5
Root-health assessment
36	White firm hydroponic roots, healthy or root rot	M	1
37	Brown thin roots, healthy or root rot	M	0.5
38	White roots with algal growth, health assessment	M	1
39	Dark discolored root mass, root rot check	M	1
40	Root close-up in net cup, condition check	M	0.5
41	Root-rot corrective-action guidance	T	1
42	Algal growth versus rot differentiation follow-up	T	0.5
Growth stage/knowledge
43	Seedling tray, current growth stage	M	0.5
44	Single young lettuce sprout, growth stage	M	0.5
45	Lettuce varieties chart, suitability for hydroponics	M	1
46	Effect of high humidity on AI-based plant monitoring	T	0.5
47	Plant identification, lettuce sample	M	0.5
48	Plant identification, brassica sample	M	0.5
49	Non-plant image, correctly flagged as not a plant	M	0.5
50	Out-of-scope crop query, scope handling	T	0.5

Categories dominated by symptom-based visual interpretation, namely health assessment, disease and browning diagnosis, pest detection, and root-health assessment, achieved weighted accuracy between 78.6% and 90.0%. The growth-stage and general knowledge category, which frequently required exact cultivar naming or precise developmental staging, achieved 56.3%. This disparity accounts for the greater part of the 36% partial-correct rate and reflects a characteristic property of open-domain language models applied to specialized agronomic queries. Four patterns underlie the partial scores: correct symptom identification accompanied by omission of specific treatment dosage or timing; accurate general guidance that did not account for the specific growth stage; qualitative rather than quantitative adjustment recommendations; and a conservative refusal to commit to an exact cultivar or species name despite correctly describing the observable morphology, which was the dominant pattern within the growth-stage and knowledge category. The last of these is a calibrated caution rather than a factual error, and it is consistent with the absence of incorrect responses across the evaluation. Taken together, these patterns indicate that the limiting factors are the granularity of domain-specific information in the retrieval corpus and the model’s caution on exact identification, rather than a failure of reasoning, consistent with findings in the RAG agricultural advisory literature [17,18].

4.3.2. Multimodal Symptom-Level Diagnostic Support

Figure 12 illustrates three representative interaction cases demonstrating the diagnostic capability of the agentic chatbot pipeline. In the first case, a text-only query describing reservoir water with a rotten egg odor and black roots prompted the system to correctly identify potential root rot and anaerobic conditions, providing structured recommendations covering root inspection, reservoir cleaning, and drainage improvement. In the second case, a user-submitted image of a visibly healthy Cos lettuce plant was correctly assessed as showing no signs of disease, with the model identifying vibrant green coloration and well-formed leaves as indicators of adequate light, water, and nutrient supply. In the third case, an image showing yellowing leaf edges with browning and wilting prompted the model to identify nutrient deficiency, watering stress, and potential fungal infection as possible causes, with recommended actions covering soil moisture checking, fertilizer application, and pest inspection.

Figure 12. Representative agentic chatbot responses spanning the query categories of Table 7: (left) a text-only root-health query returning a root rot interpretation with corrective guidance (root-health assessment, score $=1$); (center) an image-bearing query assessing a visibly healthy Cos lettuce plant (health assessment, score $=1$); and (right) an image-bearing query interpreting yellowing and browning leaf symptoms in terms of nutrient deficiency, watering stress, and possible fungal involvement (disease and browning diagnosis, score $=1$).

Across all three cases, the model consistently structured responses around four diagnostic dimensions: observed symptoms, possible causes, recommended actions, and environmental factors to monitor, demonstrating consistent output formatting suitable for non-expert users. The zero incorrect response rate observed across the full 50-case evaluation confirms that the RAG grounding and LLM Checker validation effectively prevented the generation of misleading diagnostic outputs.

4.4. Integrated System Operation

The fully integrated system operated continuously throughout the experimental period without requiring manual intervention. The Raspberry Pi 5 successfully ran all concurrent services including the FastAPI backend, YOLO11m inference, and sensor logging without service crashes. Recoverable software exceptions in a small number of 24 h cycles were handled automatically by the self-monitoring restart mechanism, and the three-strike protection was not triggered, confirming transient rather than persistent failures. The waterproofed electronics compartment exhibited no water ingress, and all actuators remained functional throughout the trial without observed fault.

5. Discussion

5.1. Implications and Positioning

The results indicate that an agentic LLM advisory pipeline can be integrated with threshold-based closed-loop sensing and control and with computer vision-based plant monitoring to provide non-expert-accessible cultivation support in a controlled indoor farming environment, while keeping the two layers architecturally separate. The 82.00% weighted advisory accuracy with zero incorrect responses, the mAP@50 of 0.971 on the validation partition of greenhouse imagery, the 98.8% runtime detection continuity, and the uninterrupted automated actuation observed across the trial period collectively indicate that the integrated architecture is operationally viable at small scale. These are system-level results; whether the platform improves agronomic outcomes relative to conventional management was not evaluated.

Relative to comparable prior work, the present system shares architectural elements with CottonBot [19], which demonstrated an RAG-augmented LLM advisory with agentic tool integration for outdoor irrigation management, and with the multi-agent greenhouse framework of Pardo-Pina et al. [21], which targets large-scale hydroponic facilities with sophisticated multi-agent orchestration. The present system is distinguished by its integration of agentic LLM reasoning, computer vision-based plant health monitoring, and multimodal symptom-level diagnostic support within a single platform on low-cost commodity hardware, without cloud dependency. This combination addresses the practical needs of small-scale and urban indoor farming, where cloud connectivity and infrastructure investment are typically constrained.

5.2. Limitations

Several limitations bound the scope of the present findings and constrain the inferences the reported evidence can support. The system was validated as an integrated engineering platform, and the deployment trial was designed accordingly: a single Cos lettuce plant was monitored under a fixed camera over a 14-day period to demonstrate continuous, uninterrupted operation of the combined control, vision, and advisory layers. The trial therefore characterizes operational continuity rather than agronomic outcome, and quantitative crop-performance evaluation across replicated plants is a separate objective reserved for future agronomic study. Whether the platform improves crop outcomes relative to conventional management was not evaluated and is identified as a primary direction for subsequent work. Detection performance and runtime continuity are reported as two distinct measurements by design: model accuracy is evaluated on the validation partition of commercial-greenhouse imagery spanning multiple plants and growth stages, while the 98.8% runtime figure deliberately quantifies operational continuity of the monitoring loop within the deployed scene, the property relevant to unattended operation. The detection partitions were assigned at the image level rather than grouped by cultivation bed or acquisition session; grouped and external evaluation are identified as directions for future validation, supported by the openly available source images. The browning module is likewise purpose-built as an operator-alerting function, with its threshold set to the deployment illumination in accordance with the operational criterion specified by the collaborating farm; its reported figures accordingly describe alert behavior, which is the intended output, rather than pathological classification. Consistent with this scope, the advisory pipeline is designed to provide grounded symptom-level guidance rather than pathogen-level confirmation, and the retrieval corpus and detection dataset are focused on the validation crop. Extending the platform to additional crops and disorder categories, together with a multi-annotator evaluation and a controlled component ablation, represents the natural next stage of development and is identified as a direction for future work.

5.3. Future Research Directions

Several directions extend the present work. Scaling the sensing and control layer from the current ESP32 to a Programmable Logic Controller would expand the addressable sensors and actuators and improve real-time determinism for larger deployments. Integrating embedded-AI camera systems would support multi-plant monitoring without a fixed overhead webcam, while extending the chatbot with Model Context Protocol tool integration would let the agentic pipeline query live sensor feeds and external data sources beyond static RAG retrieval. Finally, expanding the knowledge base and detection dataset to further crops and symptom categories would broaden applicability.

6. Conclusions

This article has presented the design, implementation, and experimental evaluation of an agentic AI-driven cultivation advisory and symptom-level diagnostic support system for controlled indoor farming. The central contribution is the agentic advisory pipeline: a multi-model architecture that couples a RAG-grounded Mistral 7B language model with a LLaVA 7B vision-language model through condition-based routing, intent classification, multi-step reasoning, and an LLM validation gate, delivering grounded text and image-based cultivation guidance from a single conversational interface. The pipeline is deployed entirely on-device without cloud dependency and integrated with vision-based plant monitoring and a deliberately isolated threshold-based control layer, so that actuation remains governed by deterministic logic while the language model operates on recorded telemetry and imagery. This integration of an agentic LLM pipeline into a working, cloud-free indoor-farming platform on commodity hardware, rather than any individual open-source component, constitutes the contribution of the work. In evaluation, the agentic advisory pipeline achieved a weighted accuracy of 82.00% across 50 queries spanning health assessment, symptom interpretation, watering diagnosis, pest, root-health, and growth-stage categories, scored against established plant pathology and postharvest references, with no incorrect response recorded. These results demonstrate that a locally hosted agentic pipeline combining retrieval-augmented language reasoning with multimodal symptom interpretation can deliver grounded, context-aware cultivation guidance without cloud services, while the supporting monitoring and control subsystems operated continuously throughout the deployment. The architecture directly addresses the practical constraints of small-scale and urban indoor farming, where cloud connectivity and infrastructure investment are typically limited, and provides an on-device foundation for delivering intelligent cultivation support to growers without agronomic training.