Forecasting Individual Dairy Cow Milk Yield via Large Language Model Orchestration

Abstract

Modern dairy farms continuously record individual cow milk yields, yet turning these records into on-demand forecasts requires a multi-step data-to-model pipeline that natural language alone cannot drive. We study large language model (LLM) orchestration over a verified forecasting pipeline, asking not whether it predicts yield but how faithfully it converts requests into correct tool-execution conditions. Against a deterministic Oracle pipeline, we compare a single-LLM function-calling framework and a multi-agent framework that distributes the same workflow across specialized agents. Using Gemini 2.5 Flash, both answer 1085 queries (4340 forecast points) varying in phrasing, horizon, and missingness, drawn from 13 months of commercial-farm records. On execution-matched points (those reproducing the Oracle’s cow, dates, input window, and horizon), the LLM output is statistically equivalent to the Oracle (paired TOST within $\pm 0.5$ kg, $p_{\mathrm{TOST}}<0.001$), so agreement is largely fixed by the deterministic tools. This equivalence is therefore confined to execution-matched points and does not describe overall system reliability. The frameworks differ instead in fidelity: the single-LLM framework reaches only 94.31% completion (zero-filled RMSE 9.70 kg), whereas the multi-agent framework reaches 99.88% at 35% added runtime; the single-LLM losses reflect systematic date-anchoring errors and silent false-accepts, not random noise. In this case study, LLM frameworks can orchestrate verified forecasting tools, and separating responsibilities improves workflow fidelity more than accuracy.

1. Introduction

Forecasting milk yield for individual dairy cows underpins the technologies that drive dairy productivity—precision feeding, health monitoring, breeding and culling decisions, and profitability analysis [1,2]. Individual-cow forecasts guide precision feeding to the expected yield and flag declining production early, which lowers feed costs, reduces disease losses, and sharpens profitability estimates [2,3]. As precision livestock farming spreads, animal-level data such as daily milk yield, somatic cell count, and activity accumulate automatically [4]. Turning these data into on-farm decisions, however, requires an analysis interface that connects forecasting models to the people who use them.

In practice, such interfaces are already provided in part by commercial herd-management software such as DairyComp 305 and PCDART, which centralize records and deliver structured reports and alerts [5,6]. These systems, however, are typically operated through predefined menus, fixed report templates, and query syntaxes that presuppose familiarity with the software, so ad hoc questions still require trained navigation. A natural-language layer complements rather than replaces them: it lets a user pose a free-form question, converts it into the correct sequence of verified tool calls, and returns the result through the same reporting and alerting channels these platforms already expose.

Milk-yield forecasting builds on mathematical models that describe the lactation curve with a few parameters [7,8]. Machine learning and deep learning have since narrowed the unit of prediction from the full lactation curve to the individual cow at the daily level [9,10,11]. The bottleneck for field deployment, however, lies less in the model architecture itself than in the analysis workflow that connects raw data to correct data retrieval, missing-value handling, and forecast calls [3]. In practice, these raw data are often incomplete and inconsistently formatted because of sensor errors, irregular milking schedules, and mismatches between recording systems [4,12]. An accurate forecasting model therefore delivers little value on its own: the surrounding workflow must still turn a user’s request into correct data retrieval, missing-value handling, and forecast calls.

What is needed here is not a large language model (LLM) that estimates the numbers directly, but an orchestration layer that selects verifiable computational tools and preserves their execution conditions. LLM function calling [13,14] and ReAct-style control interleave reasoning with action while observing tool outputs [15]. Rather than collapse a multi-step analysis into one LLM inference, they route each step to an external tool [16]. This design separates the LLM’s natural-language interpretation from the computation and then recombines them, so it suits tasks where reproducibility and verifiability matter. Studies report its effectiveness in autonomous chemical synthesis [17], tool-augmented scientific reasoning [18], and automated clinical calculation [19].

In agriculture and livestock, by contrast, most LLM applications focus on knowledge retrieval, question-answering advice, and retrieval-augmented generation (RAG) domain chatbots [20,21,22,23]. Researchers have recently begun to explore tool-based agents for plant phenotyping and dairy decision support [24,25]. Yet few studies measure how faithfully an LLM preserves a full workflow—from a natural-language query through structured data retrieval, missing-value handling, and forecast-tool execution—or compare that workflow against a reference produced by running the same tools directly.

This study evaluates LLM orchestration frameworks that convert natural-language queries into an individual cow milk-yield forecasting workflow. We compare a single-LLM approach, in which one LLM calls the tools directly, with a multi-agent approach that separates planning, routing, data retrieval, missing-value imputation, and forecasting, running both on the same deterministic tools and data. As a reference, we use an Oracle pipeline that executes the same tools correctly without any LLM. The goal is not to claim the performance of a new forecasting model, but to quantify how faithfully an LLM drives verified tools through natural language. Our contributions are as follows.

• From roughly 13 months of milking records at a single commercial dairy farm, we construct 1085 queries (4340 intended forecast points) that vary in phrasing, input window, forecast horizon, and missingness level, and we quantify workflow fidelity with an execution-matched prediction metric defined by agreement with the Oracle’s execution conditions.
• We show that at execution-matched points both LLM approaches are statistically equivalent to the Oracle, which demonstrates that the deterministic tools fix output agreement and that the systems differ instead in their execution-matched completion rate.
• Through a failure-mode analysis, we establish that the single-LLM’s loss of fidelity stems not from random noise but from systematic date-interpretation errors and silent false-accepts, and we show that separating responsibilities improves workflow fidelity.

2. Related Work

2.1. Advances in Milk Yield Forecasting Methods

Traditional milk yield forecasting rests on mathematical models that describe the lactation curve with a few parameters. Wood’s gamma function [7], Wilmink’s exponential-linear mixture [8], and the polynomial model of Ali and Schaeffer [26] have served as standards for decades, but they do not adequately capture nonlinear disturbances such as between-cow variation, feed changes, and health disorders [9].

The machine learning and deep learning approaches that followed do more than improve accuracy: they expand both the unit of prediction and the input information used. The CNN-based encoder–decoder model of Liseune et al. [9] learns the entire lactation curve in one model and drops the smooth structure that parametric models assume. For national monthly production forecasting, the GRU, LSTM, and Seq2Seq families that Cesarini et al. [27] compare bring the time series’ nonstationarity and long-range dependence directly into the learning objective. The modeling unit also shifts from herd averages to the individual level. The nonlinear regression model of Song et al. [10] provides the temporal resolution needed to detect abnormal milk yield in real time at the daily level, and the Bi-iGRU of Naghashi et al. [11] learns cross-channel dependencies among many dairy variables such as health, management, and milk quality, capturing a multi-signal structure that does not reduce to a single time series.

This trend shows that milk-yield forecasting research increasingly depends not only on model architecture but also on the temporal resolution of the input data, the choice of variables, and the preprocessing conditions. Cleaning the input data, aligning time points, and handling missing values therefore matter more and more. Most existing studies assume that an analyst has already done this preprocessing and that the model input is clean, so cleaning field data and making the tools accessible to users remain a research challenge distinct from the forecasting algorithm itself.

2.2. Data Management Challenges and Decision Support Systems on Dairy Farms

Precision livestock farming systems generate large volumes of animal-level data through sensors and farm management software, yet data collection on its own does not translate directly into decision support. Dairy data come from different sources such as milking machines, activity sensors, breeding records, and feed management systems, and they may differ in time units, identifiers, and missingness patterns. Applying a forecasting model in the field therefore requires reliable data merging, completeness checks, outlier handling, and missing-value imputation before any modeling.

In a systematic review of sensor systems on dairy farms, Rutten et al. [4] point out that many systems stop at data collection and single-indicator interpretation and do not connect adequately to integrated decision support. Schodl et al. [12] propose a procedure for cleaning dairy sensor data that comprises merge validation, data characterization, completeness checks, outlier detection, and screening for technical noise. These works show that field use depends not only on the performance of the forecasting model but also on the reliability of the data preparation that precedes it.

The Dairy Brain project builds an Agricultural Data Hub that collects, integrates, and cleans heterogeneous data from multiple dairy farms, and on this basis provides decision tools at the operational (daily), tactical (medium-term), and strategic (long-term) levels [3]. As a leading example of raising scattered dairy data onto an integrated analytical foundation, it shows that data integration and decision support should be treated as a single continuous pipeline rather than as separate stages. Even so, operating the analysis tools that such a platform provides still requires an analyst to specify the query conditions and select the appropriate tool. The stage after data integration, namely the interface that converts a user’s question into the correct data retrieval and tool execution, remains a separate challenge. The forecasting advances of the previous subsection and the data-management and decision-support work of this one address “what to predict” and “what to integrate,” respectively. Even with clean tools and data in place, however, the layer that links a user’s natural-language query to the correct execution of those tools has received little attention. We address this gap.

2.3. LLM Orchestration and Tool Use

LLM orchestration uses the language model as a control layer that interprets user intent, selects the necessary tools, and decides the next action based on execution results, rather than treating it as the final calculator. From this viewpoint, the core role of the LLM is not to perform numerical computation through its internal parameters, but to connect explicitly defined external tools and data sources in the appropriate order.

A common insight across tool-based LLM research is that computation itself should be separated from execution control. Toolformer [13] and function-calling APIs [14] generalize the structure by which a language model calls external tools, and ReAct [15] frames the sequential control problem of choosing the next action while observing intermediate execution results. The success of tool use therefore depends less on the linguistic quality of a single response than on the ability to select the right tool at the right time and pass the previous execution result to the next step.

This perspective matters even more for quantitative decision tasks. A survey of LLM tool use points to tool selection, call timing, and multi-turn state tracking as the main failure points [16], and the Berkeley Function Calling Leaderboard likewise shows that errors grow in multi-turn agent settings relative to simple function calls [28]. In clinical calculation research as well, error rates are lower when the LLM calls a verified calculation tool than when it computes directly [19].

2.4. LLM Applications in Agriculture and Livestock

LLM applications in agriculture and livestock are expanding rapidly, but most work to date uses the LLM for knowledge retrieval, question answering, and technical advice. Tzachor et al. [20] review the opportunities and risks of LLM-based agricultural extension services, and AgriGPT [21] combines agriculture-specific data with retrieval-augmented generation to improve domain adaptation. BeefBot [22] and AgroLLM [23] likewise focus on delivering livestock and agricultural knowledge to users through RAG-based question answering.

In contrast, recent agent-style research moves toward using the LLM as the control layer of a tool-based analysis pipeline. PhenoAssistant by Chen et al. [24] shows that in plant phenotyping an LLM can orchestrate image processing, visualization, and model training tools through natural-language dialogue, and Liu et al. [25] present an agent-style interface for dairy that drives a literature-grounded chatbot and the MilkBot model through natural language. These studies, however, focus on plant image analysis or on literature question answering with an interface to an existing dairy model. For individual animal livestock performance forecasting, they do not quantitatively compare the numerical fidelity of an LLM-mediated pipeline against the Oracle output produced by running the same tools directly.

Researchers also report limitations when the LLM serves directly as a forecaster on tabular data and time series. Fang et al. [29] note that an LLM can process tabular data by serializing it into text, but gradient-boosted trees and similar methods remain strong on many tabular tasks, and the LLM gains an edge only under specific conditions such as small samples or few-shot settings. This supports the view that, in agricultural and livestock data analysis as well, it may be more appropriate to use the LLM as an orchestrator that calls verified statistical or machine learning tools rather than as the forecasting model itself.

2.5. LLM-Based Multi-Agent Systems

Multi-agent systems distribute subfunctions such as planning, tool selection, execution, and verification across specialized agents. AutoGen by Wu et al. [30] presents a framework for building LLM applications through multi-agent conversation, and Li et al. [31] comprehensively survey the workflows and challenges of LLM-based multi-agent systems. The benefits of role separation are reported for complex tasks that require diverse tool selection and dynamic planning [17,18], but for structured tasks with a fixed processing pipeline, the cost of additional LLM calls and inter-agent communication can offset the performance gain.

A planner decomposes a natural-language query into executable subtasks, and a router assigns each subtask to the appropriate functional module. When routing is a matter of choosing one item from a fixed set of functions rather than open-ended text generation, a sentence-embedding bi-encoder can be used in place of a generative LLM [32]. The perspective of selective prediction can also be applied to flag cases where the selection confidence is low [33].

3. Materials and Methods

3.1. Dataset

This study uses daily milk yield records collected automatically through a DeLaval rotary parlor at a single farm in Dangjin, Chungcheongnam-do, South Korea. The farm milks twice a day, and we define daily milk yield as the sum of the two milking sessions. The raw data are stored in a Microsoft SQL Server database, and we use roughly 13 months of records starting from September 2024 for the experiments.

For this period the database contains daily milking records for 1099 cows, with a mean daily milk yield of 28.15 kg (standard deviation 7.38 kg, range 0–75.9 kg). The queries in this study target 704 distinct cows from this set, so the evaluation is not concentrated on a few animals and covers a range of production levels and lactation stages. The milking records also include some legitimate zero or low-value observations from skipped or abnormal milkings on individual days, which may be formally indistinguishable from missing values. The deterministic interpolation tool shared by all three systems fills only recorded gaps and preserves these legitimate zeros as valid observations (Section 4.5).

3.2. Experimental Design

This experiment compares, under controlled conditions, how three execution modes convert the same natural-language query into tool-execution conditions: the Oracle that runs the tools directly without an LLM, the single LLM, and the multi-agent system. To this end, we construct the evaluation query set by systematically combining the target cow, the base date, the input window, the forecast horizon, and the missingness level within the input window. Every query is fixed to the single task of milk yield forecasting and runs on the same deterministic tools shared by the three systems, so that differences between systems arise only in the control layer that converts a query into tool execution, not in the forecasting model or the input data. The focus of the evaluation is therefore not forecasting accuracy itself but how well each system preserves the execution conditions that the query intends. The following paragraphs describe, in turn, the three axes that define the query set: the missingness level, the combination of input window and forecast horizon, and the diversification of natural-language phrasing.

To assess whether the LLM maintains the same tool-execution procedure even in a workflow that contains missing data, we create 10 missingness levels by removing some observations within the input window. The missingness level is defined as the number of observations removed from the input window, forming 10 steps from level 0 (no missing values) to level 9 (nine missing values). For a fair comparison, the positions removed at each missingness level are fixed identically across the three systems (Oracle, single LLM, multi-agent), so differences between systems arise only from tool-execution control and not from the missingness pattern. Rather than isolate how missing values affect forecasting accuracy, this design checks whether the system runs data retrieval, interpolation, and forecast calls consistently with the Oracle even when a natural-language query contains missing values.

Candidate queries are defined by combining four input windows ending on a base date D (7, 14, 21, and 28 days) with forecast horizons from 1 to 7 days. To avoid the unrealistic condition in which more than half of the input information is lost to missingness, we cap the missing rate at 50%, so a shorter input window allows a lower maximum missingness level (for example, a 7-day window permits at most three missing values). The number of cases and queries follows from these combinations of input window, missingness level, and forecast horizon together with the 50% cap; the detailed derivation is given in Figure 1. In the end we construct 31 cases and 1085 natural-language queries (4340 intended forecast points).

We restrict all queries to a single task type, milk yield forecasting. This design removes task-type differences and isolates how natural-language phrasing alone affects the interpretation of the cow ID, base date, input window, and forecast horizon, and the resulting tool calls. Each query therefore keeps the same semantic structure while varying grammar, vocabulary, and date expressions. For example, the same request is phrased as “Using the past 7 days of records, predict milk yield for cow 817 for the next 3 days starting from 5 December 2024” or “Forecast cow 817’s milk production for the three days following 5 December 2024, based on the previous 7 days of data”. The query-generation procedure and further paraphrase examples are detailed in Appendix A.1.

3.3. System Configuration

We compare three execution modes that share the same data and forecasting tools but differ only in the control layer that calls those tools (Figure 2). All three systems share the same deterministic functions for milking-record extraction, missing-value interpolation, and linear-regression forecasting. The single-LLM and multi-agent frameworks invoke these identical tools through their control layers, while the Oracle calls them directly. Differences between systems therefore arise in natural-language interpretation, tool selection, and parameter passing rather than in the forecasting algorithm. The single-LLM framework and the multi-agent system use Gemini 2.5 Flash without fine-tuning, and, unless otherwise noted, statistics are collected from a single run per query; we separately verify run-to-run stability across 3 independent runs in Section 4.5. The full system instructions and tool-call specifications for both frameworks are given in Appendix A, and the model and decoding settings for every component are summarized in

Table A1. System and decoding settings used for all runs.

Component	Model/Method	Setting
Single-LLM function calling	Gemini 2.5 Flash	temperature 0.7
Planner (multi-agent)	Gemini 2.5 Flash	temperature 0.2; 1–6 subqueries, anchor-first
Extractor (multi-agent)	Gemini 2.5 Flash	temperature 0.0
Router (multi-agent)	Sentence-embedding bi-encoder (cosine)	min-OK score 0.5 (<: re-plan)
Interpolation	Deterministic linear	straight-line between neighbours (shared by all three systems)
Forecaster	Deterministic linear regression	horizon 30 days
Common	—	per-query timeout 180 s; rate-limit exponential backoff

(Appendix B).

Figure 3 shows how an LLM framework connects a natural-language query to the tool-execution units. The framework first interprets the query as execution conditions—the cow ID, base date, input window, and forecast horizon. The milking-record extraction tool (labeled Tabular RAG in Figure 2 and Figure 3) then retrieves the input time series from the raw data, an interpolation step fills in the missing values, and the forecasting tool (labeled LC Predictor in the figures) uses the corrected time series to compute milk yield over the requested horizon. These two names denote roles rather than specific algorithms. “Tabular RAG” refers to retrieval-augmented access to the structured milking-record table—a lookup by cow ID and date range, not a free-text search—and “LC Predictor” names the lactation-curve forecasting slot, which in this study is deliberately instantiated as a simple deterministic linear regression rather than a parametric lactation-curve model; any lactation-curve or deep-learning forecaster can occupy the same slot without architectural change. In the single-LLM framework, one LLM calls the extraction, interpolation, and forecasting tools directly through function calls, whereas in the multi-agent system a planner and a router assign these same tools to dedicated modules.

The Oracle pipeline runs milking-record extraction, linear interpolation, and linear-regression forecasting directly, without an LLM. Here Oracle does not refer to the observed milk yield but to the reference execution produced by running the same deterministic tools directly under the correct conditions.

The single-LLM framework has one LLM interpret the natural-language query and connect the needed milking-record extraction, interpolation, and forecasting functions through function calls. The execution result of each step serves as input for deciding the next action, and the overall flow has a multi-turn structure that repeats observation and action, much like the ReAct pattern [15].

The multi-agent system separates natural-language interpretation, tool selection, information extraction, interpolation, and forecasting into distinct responsibilities. The planner divides the query into subtasks, and the router connects each subtask to one of the milking-record extraction, interpolation, and forecasting functions. The extraction module structures the cow ID, base date, and input window, and the interpolation and forecasting modules call the same deterministic functions as the single-LLM framework.

The router uses a sentence-embedding classifier instead of a generative LLM [32]. Because the subtasks in this experiment are assigned to only one of a small, predefined set of functions, routing is closer to a closed-set classification problem than to free generation. A dedicated classifier reproduces the same routing for the same input, reduces the cost of LLM calls, and flags ambiguous assignments through a confidence threshold [33]. This choice separates tool-selection errors from differences in the wording of a generative response or in the reasoning path.

3.4. Tool Execution and Forecasting Method

When the input window contains missing values, the missing positions are filled by straight-line interpolation between the adjacent observations on either side, in temporal order. On this corrected input window, the forecasting step fits milk yield with a simple linear regression on the date index and extrapolates the fitted line to the forecast horizon requested by each query to produce daily milk yield. This interpolation is a deterministic module in all three systems. Because the extraction, interpolation, and regression procedures are fixed, they return the same output for the same inputs and execution conditions; differences across systems therefore arise solely from how each control layer establishes those inputs and conditions.

We deliberately adopt a simple model as the forecasting tool. The goal of this study is not to build the best-performing milk yield forecasting system but to evaluate whether a natural-language query is correctly converted into an execution sequence of deterministic analysis tools. Only a deterministic forecasting tool lets us separate the parameter-extraction, interpolation, and tool-call errors of the LLM-mediated process from the uncertainty of the forecasting model itself. The framework’s modular design lets a lactation-curve model [7,8,26] or a deep learning model [9,11,27] replace the forecasting component without architectural changes, so the choice of forecaster stays separate from this study’s target of workflow fidelity.

3.5. Evaluation Metrics

We evaluate from three perspectives: execution reliability, agreement with the Oracle, and consistency across conditions. We measure execution reliability by the fraction of all intended forecast points for which a system produces a prediction under the same execution conditions as the Oracle, and by the fraction of natural-language queries from which it extracts the cow ID, base date, and input window correctly.

Even when a numeric prediction exists, we separately check whether it comes from the same forecasting problem as the Oracle. Throughout the paper, we call a case an execution-matched prediction when the cow ID, base date, input window, forecast horizon, and actual forecast dates all match those of the Oracle query. We run the paired and equivalence tests only on the common points that are execution-matched in all three systems.

We define agreement with the Oracle as the Pearson correlation coefficient ($r_{\mathrm{oracle}}$) between the predictions of an LLM-mediated system and those of the Oracle. Because this metric is interpretable only when the two output arrays are compared directly on the same set of query-horizon pairs, we compute it only on the common execution-matched points where all three systems produce an execution-matched prediction. We also report the root mean squared error (RMSE) and mean absolute error (MAE) against the observed milk yield to compare the error magnitude of the Oracle execution and the LLM-mediated execution. Here RMSE and MAE are computed as

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{({\widehat{y}}_i-y_i)}^2},\mathrm{MAE}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}|{\widehat{y}}_i-y_i|,$$

(1)

where ${\widehat{y}}_i$ is the system prediction and $y_i$ is the observed milk yield.

We test differences between systems with a paired t-test after matching the absolute errors computed on the same query-horizon pairs. Because every system produces a value for the same cow, date, and forecast horizon, the observations are paired rather than independent samples, and this test checks whether the LLM-mediated execution produces larger absolute errors than the Oracle execution. To avoid judging workflow differences by statistical significance alone, we also report Cohen’s $d_z$ for paired samples. For the difference value $D_i= |{\widehat{y}}_i^{\mathrm{LLM}}-y_i| - |{\widehat{y}}_i^{\mathrm{Oracle}}-y_i|$, which represents the difference in absolute error between the LLM-mediated and Oracle executions at point i, we define $d_z=\overline{D}/s_D$, where $\overline{D}$ is the sample mean of the difference values $D_i$ and $s_D$ is their sample standard deviation. We interpret $|d_z| <0.2$ as a negligible difference, 0.2–0.5 as small, 0.5–0.8 as medium, and >0.8 as large [34].

Whereas the paired t-test asks whether a difference exists, the paired TOST tests whether that difference falls within a predefined tolerance [35]. To judge whether the LLM orchestration output differs from the Oracle execution enough to matter operationally, we set conservative equivalence margins of 0.5 kg and 1.0 kg. These two margins correspond to about 1.8% and 3.6% of the mean daily milk yield (28.15 kg), respectively.

Because the unit of evaluation is the individual forecast point, the sample size is large, and even a very small mean difference can be statistically significant in a paired t-test. Moreover, forecast points belonging to the same cow or the same query are not independent, so the effective sample size of the test is smaller than the nominal one. For these reasons, we interpret the p-value only as supplementary information about whether a difference exists, and we judge the actual magnitude from Cohen’s $d_z$ and the mean difference. The TOST separately examines whether the observed difference lies within an operationally acceptable range, so that statistical significance and practical equivalence are interpreted together.

4. Results

All analyses below evaluate not the absolute accuracy of the forecasts but how faithfully the LLM-mediated execution preserves the output and execution conditions of the Oracle execution. RMSE and MAE are therefore interpreted as indicators of agreement with the Oracle execution, not as measures of which forecaster performs better.

4.1. Pipeline Reliability

The three systems process the same 1085 queries (4340 intended forecast points) (

Table 1. Execution reliability across the three systems.

Metric	Oracle	Single-LLM	Multi-Agent
Numeric predictions	4340	4195	4339
Execution-matched predictions	4340	4093	4335
Execution-matched completion rate	100%	94.31%	99.88%
Parameter extraction accuracy	—	95.48%	99.72%

).

The single LLM produces a numeric prediction in most cases, but some outputs use a date or parameters that differ from the Oracle and therefore do not count as execution-matched predictions. As a result, its execution-matched completion rate is 94.31%. Under the same criterion, the multi-agent system reaches an execution-matched completion rate of 99.88%.

4.2. Output Agreement with the Oracle Pipeline

We compute output agreement on execution-matched predictions only (

Table 2. Oracle agreement and operational sensitivity.

System	N	RMSE (kg)	MAE (kg)	${\mathit{r}}_{\mathbf{oracle}}$
Oracle	4088	7.46	5.24	—
Single-LLM	4088	7.46	5.24	1.0000
Multi-agent	4088	7.46	5.24	1.0000
Single-LLM, missing-output penalty	4340	9.70	6.42	—
Multi-agent, missing-output penalty	4340	7.45	5.24	—

RMSE, root mean squared error; MAE, mean absolute error.

). Because $r_{\mathrm{oracle}}$ is interpretable only when the Oracle and LLM outputs are matched directly on the same set of query-horizon pairs, it is computed on the common points where all three systems produce an execution-matched prediction. For the operational sensitivity analysis, we replace non-matched LLM outputs with zero and additionally report this as a missing-output penalty.

On execution-matched predictions alone, the RMSE of the three systems is nearly identical. However, when we replace LLM outputs that fail the matching criterion with zero, the RMSE of the single LLM rises to 9.70 kg, whereas the multi-agent system stays at 7.45 kg.

At the common execution-matched points, the correlation between the single-LLM and Oracle predictions is $r_{\mathrm{oracle}}=1.0000$. The mean absolute-error difference between the single LLM and the Oracle is only +0.000 kg, the paired t-test shows no significant difference, and the effect size is negligible ($p=1.000$, Cohen’s $d_z=0.000$). The multi-agent system reproduces the Oracle output exactly ($p=1.000$, $d_z=0.000$).

In the TOST as well, both LLM systems are equivalent to the Oracle at both the 0.5 kg and 1.0 kg margins ($p_{\mathrm{TOST}}<0.001$). The 90% confidence interval for the single LLM is $[+0.000,+0.000]$ kg, and the multi-agent system is numerically identical to the Oracle.

4.3. Output Consistency Across Conditions

RMSE broken down by missingness level and forecast horizon leads to the same conclusion (

Table 3. Conditional RMSE agreement by missingness level and forecast horizon.

Missingness Level					Forecast Horizon
Condition	$\mathit{N}$	GT	Single	Multi	Condition	$\mathit{N}$	GT	Single	Multi
0	505	7.54	7.52	7.54	D+1	1029	5.79	5.79	5.79
1	529	7.74	7.74	7.74	D+2	880	6.34	6.34	6.34
2	545	8.90	8.90	8.90	D+3	734	7.29	7.29	7.29
3	537	9.06	9.06	9.06	D+4	587	8.10	8.10	8.10
4	404	6.13	6.13	6.13	D+5	442	8.81	8.81	8.81
5	377	6.33	6.33	6.33	D+6	291	9.63	9.62	9.63
6	406	5.97	5.97	5.97	D+7	147	10.73	10.73	10.73
7	267	5.82	5.82	5.82
8	266	6.79	6.80	6.79
9	274	6.62	6.62	6.62

GT is the RMSE against the ground truth, and Single and Multi are the RMSE of each system’s output. All RMSE values are in kg, and N is the number of points where all three systems jointly produced an execution-matched prediction.

). At almost every missingness level and forecast horizon, the RMSE of the single LLM and the multi-agent system is numerically identical to the Oracle, and in the few items where a difference appears (for example, missingness levels 0 and 8 and horizon D+6) it stays within rounding error (at most 0.02 kg).

Because the conditional RMSE of the three systems agrees within rounding (at most 0.02 kg) across all missingness levels and forecast horizons (Table 3), the LLM-mediated process creates no new error pattern that grows with missingness level or forecast horizon.

4.4. Component-Level Analysis

Parameter extraction accuracy is the item that differs most between the two LLM systems (

Table 4. Parameter extraction accuracy.

Parameter	Single-LLM	Multi-Agent
Cow ID	97.88%	99.91%
Base date	95.48%	99.72%
Input window	97.88%	99.91%
Overall match	95.48%	99.72%

).

The single LLM’s parameter extraction accuracy is 95.48% by overall match, and its lowest item is base-date extraction (95.48%). Extraction accuracy for the cow ID and the input window is higher, at 97.88% and 97.88%, respectively. The multi-agent system’s overall-match accuracy is 99.72%.

The same straight-line interpolation rule applies to missing inputs before the forecasting step, as a deterministic module in all three systems. We therefore do not report interpolation accuracy in its own table; instead, the execution-matched prediction criterion checks whether the full workflow, interpolation included, preserves the Oracle’s execution conditions.

The multi-agent system’s mean execution time per query is 18.43 s, about 35% longer than the single LLM (13.66 s). The added time is the cost of separating the planner, router, and interpolation modules, and in return the execution-matched completion rate improves.

4.5. Fidelity-Failure Analysis of the Single LLM

To determine where the difference in execution-matched completion rate comes from, we classify, at the execution-log level, the queries for which the single LLM fails to produce an execution-matched prediction. One or more failures appear in 58 of the 1085 queries (5.3%), and the outcomes split clearly into two kinds (

Table 5. Single-LLM failure outcomes (query level, N = 1085).

Outcome Type	Queries	Share	Operational Implication
Silent error (number under wrong conditions)	24	2.2%	Undetectable without verification
Hard failure (forecaster not called, no output)	34	3.1%	Immediately visible to the user
Total (unique queries)	58	5.3%

Shares are relative to all 1085 queries, and the total may not equal the simple sum of the individual items because of rounding.

). One is a silent error, in which the system returns a plausible number under the wrong execution conditions; the other is a hard failure, in which the forecasting function is not called and no output is produced at all. The more dangerous one operationally is the silent error, which goes undetected without a user or a downstream verification step. At the point level, the single LLM false-accepts numbers from execution conditions that differ from the Oracle at 102 forecast points, whereas the multi-agent system does so at only 4.

The dominant cause of these failures is mis-extraction of the base date. The single LLM extracts the base date incorrectly in 26 queries, spanning both silent errors and hard failures, and the pattern is not random but converges on two systematic types. The first is an off-by-one ($-1$ day) error (18 cases), in which the model moves the base date one day earlier because it confuses the inclusive/exclusive boundary in natural language. The second is an error that anchors the base date to the start of the input window (7 cases), in which the extracted base date is shifted from the intended date by exactly the input window length (−7/−14/−21/−28 days). The same type of date error occurs in only 2 cases in the multi-agent system, and most of its failures are planner output-format errors (6 cases), which the system blocks before the forecasting step. This shows the effect of separating the responsibility for date interpretation into a dedicated extraction module.

Because the single LLM delegates interpolation to the same deterministic tool used by the Oracle and multi-agent systems, a legitimate $0.0$ observation from a dry-off or abnormal-milking day is preserved as a valid value rather than treated as missing. Consequently, whenever the cow ID, base date, and input window all match, the single-LLM prediction is numerically identical to the Oracle, and its remaining failures are confined to parameter extraction.

The following cases are representative of each failure type.

• Inclusive/exclusive confusion (silent error). For the query “…starting on 25 February 2025 in 6 days from 7 days of data” (cow 1072), the single LLM extracts the base date as 24 February 2025, forecasts over a window shifted one day earlier, and returns numbers that differ from the Oracle by up to 8.0 kg with no error flag.
• Anchoring to the input-window start (silent error). For the query “…beginning on 12 April 2025 over the next 3 days from 7 days of data” (cow 759), it fixes the base date at 5 April 2025—the start of the input window, seven days before the intended date—and produces forecasts that diverge by up to 40.6 kg.
• Forecaster not called (hard failure). In 34 queries, the single LLM produces no final output at all, most often because the forecasting function is never called. Because this type yields no number, it surfaces immediately, unlike a silent error.

In summary, the single LLM’s loss of execution-matched completion stems not from a limitation of the forecasting model but from failures in natural-language interpretation and tool-call control, and a substantial share of these are silent errors that do not surface without post hoc verification. By separating date interpretation, missing-value handling, and forecast calls, the multi-agent system blocks most of these failures before the forecasting step.

To confirm that these patterns are not an artefact of a single stochastic decoding draw, we repeat the full evaluation over 3 independent runs under the reported decoding settings (single-LLM temperature $0.7$; multi-agent planner $0.2$, extractor $0.0$; Table A1). The orchestration-fidelity metrics stay highly stable across runs: the run-to-run standard deviation of query-level parameter-extraction accuracy is only 0.42 percentage points for the single LLM and below 0.1 percentage points for the multi-agent system. The single LLM’s dominant failures—the off-by-one $-1$-day base-date error and anchoring the base date to the start of the input window—also recur in every run rather than fluctuating as random noise. This confirms that the fidelity gap between the two architectures reflects systematic behaviour rather than a single stochastic draw.

5. Discussion

5.1. Workflow Fidelity

The central question of this study is not whether the LLM forecasts milk yield well on its own, but how accurately it converts a natural-language query into the execution conditions of deterministic analysis tools. At the common execution-matched points, the outputs of the two LLM systems are statistically equivalent to the Oracle, and the conditional RMSE patterns are similar across the three systems. This agreement, however, comes from the deterministic tools that the three systems share, not from the LLM’s forecasting ability, and the LLM-mediated process adds no new bias at execution-matched points across missingness levels or forecast horizons.

Output agreement at execution-matched points alone therefore cannot explain a system’s operational reliability. When outputs that fail the matching criterion are treated as failures, the single LLM’s missing-output penalty increases sharply while the multi-agent system’s stays small, and this gap reveals the substantive difference between the two architectures. The main finding thus lies not in better forecasting accuracy but in how well each architecture keeps a natural-language query a correct tool-execution problem from start to finish.

5.2. Differences Between the Two LLM Frameworks

Both LLM systems preserve the Oracle output at execution-matched points, but the multi-agent system is more stable in execution-matched completion and date interpretation. Missing-value handling illustrates this difference well. The single LLM decides the retrieval, interpolation, and forecasting conditions together in one conversational flow. The multi-agent system instead assigns missing-value handling and forecast calls to separate modules, which makes execution conditions easier to trace and keeps date and parameter errors from propagating to the final forecast.

The choice to use an embedding-based classifier in the router instead of a generative LLM belongs to the same logic. What this study compares is not a specific multi-agent implementation but two orchestration schemes: one in which a single LLM jointly reasons about interpretation, tool selection, and execution conditions, and one that separates these into dedicated modules. The deterministic router and the module separation are therefore not confounders to remove but intrinsic properties of the role-separation scheme. We do not claim that any single component is superior; we claim that separating natural-language interpretation from tool calling better preserves execution fidelity.

5.3. Relation to Forecasting-Model Research

This study does not propose a new milk yield forecasting model. Whereas prior work has raised the expressiveness and accuracy of the forecaster through lactation-curve models, deep learning, and multivariate time-series models [10,11,27], this study addresses whether a user can reliably invoke such a forecaster through natural language.

In a real deployment, the forecasting component can be replaced by a more sophisticated lactation-curve or deep learning model. Even then, the system must still verify the fidelity of natural-language interpretation, data retrieval, missing-value handling, and tool calls. The view of the LLM as a control layer over verified tools rather than as the computing agent recurs in clinical calculation and scientific workflow automation [17,18,19]. This study extends that view to the individual cow forecasting workflow of dairy data analysis.

Our use of linear regression is a deliberate control, not a performance claim: a deterministic, transparent forecaster lets us attribute any discrepancy to orchestration rather than to the model (Section 3.3). Replacing it has two outcomes. If a stronger model, such as an LSTM, can be wrapped as a tool with the same single-series interface, it could improve accuracy at no extra orchestration cost. If it instead needs additional inputs—multivariate signals such as feed intake, activity, and days-in-milk (DIM)—the extraction and routing layers must locate, align, and validate several heterogeneous inputs, each with its own missingness and sampling rate, so the orchestration burden grows and the forecast may become less reliable despite the stronger model. Linear regression is therefore a minimal, functional choice for isolating orchestration fidelity; extending to advanced forecasters is a separate problem for future work.

5.4. Operational Risks of Silent Errors

Among the failure modes we identify, silent errors carry the greatest operational risk because the system returns a plausible forecast under the wrong execution conditions, so no error surfaces to the user. In an automated decision context where forecasts feed farm dashboards or alerts, such errors propagate without any visible signal. A misextracted cow ID, for example, attributes one animal’s yield trend to another and can trigger unnecessary veterinary visits or misdirected feed and management adjustments, raising cost and causing avoidable animal distress; a base-date or input-window error shifts the trend that underlies culling, breeding, and health decisions. Because the downstream tools are deterministic, these consequences stem entirely from orchestration fidelity rather than from the forecasting model. Separating date interpretation, missing-value handling, and forecast calls already reduces silent errors substantially (Section 4.5), but it does not eliminate them.

Beyond role separation, complementary safeguards from the broader LLM literature could further reduce silent errors. The system can re-verify the extracted execution conditions before calling the tools—for example, by requiring agreement across repeated stochastic samples [36]—and can estimate its own confidence to abstain or defer to a human when it is low, in the spirit of selective prediction [33]. Because the dominant silent errors are systematic and field-specific rather than random (Section 4.5), even lightweight verification of the cow ID, base date, input window, and forecast horizon would intercept most of them. We therefore recommend a mandatory verification step for field deployment—human oversight or an automated secondary check—that confirms not only the forecast magnitude but also whether the extracted conditions match the user’s intent before a result is acted upon.

5.5. Practical Deployment and Field Integration

The orchestration layer is designed to sit on top of existing herd-management systems rather than replace them. Its tools are primitives these platforms already implement—record retrieval by cow and date range, and report generation—so a deployment would point the extraction tool at the platform’s data access (for example, a DairyComp query or a PCDART export) while the interpolation and forecasting tools stay unchanged. Because the layer turns a free-form question into an explicit, verifiable sequence of these calls, its output can return as a native report field or raise an alert, with the verification step of Section 4.5 gating it beforehand. Integrating a specific platform’s API and alerting rules is left to future work, but this modular, verified-tool design is what makes such integration tractable.

The same design clarifies how the approach transfers across farms and equipment. The extraction tool (Tabular RAG) exposes a fixed interface—given a cow identifier and a date range, return a daily milk-yield series—decoupled from the storage schema. Porting to another parlor type or vendor (for example, robotic milkers such as Lely or DeLaval VMS) then needs only a thin adapter that maps the target schema’s identifier, timestamp, and yield fields onto this interface and aggregates different logging frequencies, such as the several milkings per day of robotic systems, into the expected daily series. Because the LLM orchestrates against this fixed signature rather than the raw schema, the fidelity behaviour we measure is independent of the data layout; only the adapter changes. We evaluate a single farm with a rotary parlor, so cross-equipment and cross-vendor validation remains future work.

The physiological state of the cow also matters. All cows here are lactating, so forecasting for dry cows—which serves no practical purpose—is out of scope; the legitimate $0.0$ values are isolated dry-off or abnormal-milking days within an active lactation and are valid observations, not missing data. A field deployment should still filter non-lactating animals by gating each forecast on lactation status: where herd-management records exist, it reads the calving and dry-off dates (equivalently, DIM) and forecasts only for cows in lactation; where such metadata are absent, a data-driven fallback flags a cow as non-milking when the input window shows an extended run of consecutive zero yields, rather than an isolated zero, and returns a status message instead of a forecast. Coupling this status check with the extraction step prevents biologically meaningless outputs.

Distinguishing a legitimate zero reading from a sensor fault is a data-quality task upstream of orchestration. The framework assumes curated milking records and does not itself perform sensor-level cleaning; the relevant point is that its architecture can accommodate such a step. The deterministic interpolation tool shared by all three systems fills only recorded gaps and preserves a valid $0.0$ reading as a real observation (Section 4.5), so blind interpolation of legitimate zeros does not occur. The same modular structure lets a deployment add a sensor-status validation tool that, before interpolation, cross-checks each zero or out-of-range reading against sensor-health and milking-event metadata—for example, whether a milking was actually recorded—and labels it as a valid observation, a sensor fault, or a true gap, so only genuine gaps are interpolated. Such a component would harden a field deployment against the data-cleaning errors common in commercial sensor streams [4,12] without altering the orchestration architecture; building and validating it is beyond our scope.

Latency is a clear limitation. Multi-step orchestration of this kind takes seconds to minutes per query—tens of seconds in our multi-agent setting—which hampers real-time, interactive service, and cloud-hosted models add a dependence on stable connectivity that farms may lack. A smaller, locally hosted (quantized) LLM could cut latency and remove that dependence, though whether such a lightweight model can orchestrate reliably requires further study. Fine-tuning is reported to recover much of the accuracy lost to compression [37], but whether this holds for orchestration in the livestock domain remains open.

5.6. Limitations

This study is not a benchmark intended to generalize the operational performance of LLM orchestration; rather, it aims to characterize the fidelity with which a natural-language query is converted into the execution conditions of deterministic analysis tools and to identify its failure modes. The reported completion and error rates are therefore case-based evidence from one data environment and one forecasting task, and we cannot generalize them to claim that the same numbers hold for other farms, other data management systems, or other forecasting models.

In addition, the main quantitative evaluation uses a single LLM backbone (Gemini 2.5 Flash). Although the reported orchestration-fidelity metrics remain stable across 3 independent runs under the reported decoding settings (Section 4.5), so the conclusions do not hinge on a single stochastic draw, their absolute values may still shift with the model version. Cross-validating across multiple LLM backbones therefore remains outside the scope of this study and should be addressed in future work.

This study focuses on a single structured task, milk yield forecasting. For tasks that combine multiple data sources and decision criteria, such as health-record lookup, feed-efficiency analysis, and cow-to-cow comparison, the difficulty of planning and routing may change. Future work should extend the same matching evaluation to multiple farms, diverse LLM backbones, and the substitution of high-performance forecasting models.

6. Conclusions

This study measures how faithfully a natural-language query is converted into the correct execution conditions of a deterministic milk-yield forecasting pipeline—milking-record extraction, missing-value imputation, and linear-regression forecasting—rather than improving the forecaster itself. Across 1085 queries, both LLM frameworks reproduce the Oracle’s numeric output at execution-matched points (TOST within $\pm 0.5$ kg), confirming that, once a query is executed under the correct conditions, the shared deterministic tools fix the agreement. The frameworks diverge instead in how reliably they reach those conditions: the single-LLM framework attains 94.31% execution-matched completion, whereas separating planning, routing, extraction, interpolation, and forecasting into dedicated modules raises this to 99.88%. Because the equivalence holds only at matched points, this difference—not the numeric agreement—governs overall reliability: under a zero-fill penalty the single-LLM RMSE rises to 9.70 kg, while the multi-agent system stays near the Oracle at 7.45 kg.

Systematic silent errors dominate the single-LLM losses—off-by-one and input-window-anchoring date extraction—and recur across 3 independent runs rather than varying as random noise, indicating that the fidelity gap is architectural rather than a stochastic artefact. The practical implication is that connecting a natural-language interface to farm decision-making requires not only the forecast value but also verification that the system produced it for the correct cow, base date, input window, and horizon; role separation supplies much of this fidelity, and a dedicated verification step closes the remainder. Extending the same execution-matched evaluation to additional farms, task types, and LLM backbones is a natural next step.