Ranking Soil Quality Indicators Using the SMART Criteria, AHP Method and Chatbots

Abstract

Soil is a complex environmental component characterized by numerous physical, chemical, and biological attributes that determine its capacity to provide ecosystem services. While composite soil health indices are traditionally derived from static literature reviews or costly expert panels, this study evaluates the potential of general-purpose large language models as a rapid, low-cost aid for AI-driven environmental decision support. We systematically evaluate the conceptual reliability and mathematical consistency of four distinct Large Language Models (LLMs): ChatGPT, Gemini, Claude, and Copilot, in executing an Analytic Hierarchy Process (AHP) matrix integrated with the SMART criteria. The evaluation prioritized seven indicators: soil organic matter (SOM), earthworm presence, aggregate stability, electrical conductivity, available nutrients, pH, and water infiltration capacity. All platforms generated mathematically consistent AHP matrices, with consistency ratios below accepted thresholds. Across ten independent runs per platform, soil organic matter received the highest mean global weight (~0.22) and earthworms the lowest (~0.08); inter-platform agreement was statistically significant (Kendall’s W = 0.75, p = 0.007), yet single-run variability reached ~50%, showing that repeated prompting is required. This approach offers a practical framework for rapid, preliminary indicator screening under resource-constrained conditions, while revealing model-specific divergences and possible training-data biases that warrant further investigation. However, our findings suggest AI-generated AHP outputs should be treated as preliminary decision-support results and should be verified through cross-platform comparison, repeated prompting, expert review, and site-specific validation.

1. Introduction

Assessing soil health under resource-constrained conditions requires identifying a prioritized set of measurable parameters—a challenge shared by agricultural producers, environmental restoration specialists, and land-use planners alike. To mitigate excessive labor and financial expenditures, it is necessary to determine which quality indicators most accurately and comprehensively represent local conditions [1,2]. However, the selection of such indicators remains partly dependent on expert judgment, local experience, and the availability of analytical resources. This creates a need for transparent decision-support approaches that can help structure and compare alternative indicators.

The burgeoning integration of Artificial Intelligence (AI) across diverse scientific domains has raised questions about whether AI platforms can reliably prioritize soil quality parameters [3,4]. Specifically, the consistency and accuracy of AI-generated rankings for soil health indicators have not yet been rigorously evaluated [4]. This issue is particularly relevant for multicriteria decision-making methods, such as the Analytic Hierarchy Process (AHP), where pairwise comparisons and consistency checks are central to the reliability of the final ranking. Furthermore, since soils must be analyzed quickly and accurately for rational use within a sustainability framework—or even for remediation—the development of studies like this one yields products that help meet this demand.

This study therefore evaluates the efficacy of AI platforms in developing hierarchical indicator rankings within an Analytic Hierarchy Process (AHP) framework. The goal is to establish a robust hierarchy of indicators that optimize resource allocation and improve the accuracy of available data on soil health, thereby enhancing sustainability efforts in soil analysis and land-use processes. More specifically, the study examines whether different AI platforms generate consistent AHP-based rankings, whether their outputs differ across systems, and whether the resulting priorities are consistent with established soil quality literature.

1.1. Concepts of Soil, Soil Health and Soil Quality Indicators

Soil is the uppermost layer of the Earth’s surface layer, an essential natural capital formed through the weathering of rocks and the decomposition of organic matter over time, under the influence of climate, topography, microorganisms and more recently human activity. It performs multiple critical functions: supporting plant growth, providing habitat for living organisms, enabling water infiltration and storage, neutralizing harmful chemicals, and serving as a foundation for infrastructure. Its composition, including mineral particles, organic matter, water and air, varies considerably across regions and land-use types [5,6].

Soil health reflects its ability to function within the sustainable limits of the ecosystem and land use. When healthy, soil can support essential biological activities, to maintain environmental quality, and to promote the health of plants and animals. Thus, soil health also reflects its resilience, or how the soil responds to environmental stress [6,7]. Since many soil functions cannot be directly measured, evaluation efforts have largely focused upon the selection of key indicators [8].

An indicator is a measurable attribute used to monitor and evaluate performance, status, or progress. In soil science, researchers have developed numerous quality indicators categorized into physical, chemical, and biological domains [2,9]. Many indicators overlap across categories or interplay with other environmental elements, such as water, and their measurement requirements range from simple field procedures to complex, costly, and labor-intensive laboratory analyses [10,11,12].

The scientific community recognizes at least thirty indicators across three categories: physical, chemical and biological. Although some indicators have an independent mode of functioning, most interact dynamically, changing in response to fluctuations in others [1]. This high volume and complex interconnectedness can obscure the identification of truly representative attributes [2,10].

Recognizing a prioritized, manageable subset of indicators is therefore critical for professionals who require soil quality data but operate under time and resource constraints. Decision-support methods that synthesize multidimensional information into actionable rankings have consequently gained increasing prominence in applied soil science [3,10].

1.2. The AHP Method and SMART Criteria

Multicriteria decision-making methods such as the Analytic Hierarchy Process (AHP) were developed to support complex decisions involving multiple quantitative and qualitative factors. AHP decomposes a decision problem into criteria, sub-criteria, and alternatives, and then uses pairwise comparisons to derive relative priorities [13,14]. In the context of soil quality assessment, this structure is useful because indicators differ not only in scientific relevance, but also in measurability, cost, feasibility, and practical interpretability.

In short, AHP offers a theory and methodology for relative measurement. Rather than seeking exact numerical values, relative measurement focuses on the proportional relationships among alternatives or criteria. The central idea is to simplify the analysis of complex systems by reducing them to a series of pairwise comparisons. The method rests on three core principles [15,16]:

Hierarchy construction: Complex problems often demand organizing criteria into a hierarchical structure, which reflects a natural pattern of human reasoning. AHP facilitates this structuring process, most commonly using a tree-like hierarchy in which the main goal is decomposed into criteria and lower-level alternatives.

Priority setting: The method establishes priorities through pairwise comparisons of elements with respect to a given criterion.

Logical consistency: AHP incorporates indices that permit analysts to evaluate the consistency of establishment of priority, ensuring that the relative rankings do not violate logical coherence. The consistency of the results is often measured by the Consistency Ratio (CR), calculated by Equations (1) and (2):

$$\mathrm{C}\mathrm{R}={\displaystyle \frac{\mathrm{C}\mathrm{I}}{\mathrm{I}\mathrm{R}}}$$

(1)

$$\mathrm{C}\mathrm{I}={\displaystyle \frac{{\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}}-\mathrm{n}}{\mathrm{n}-1}}$$

(2)

where CR is the consistency ratio, CI is the consistency index, IR is the Randomness Index (or Random Inconsistency Index) based on the matrix dimension, λ_max is the maximum eigenvalue and “n” is the matrix size.

The CR is a fundamental metric for validating the consistency of pairwise judgments made by decision-makers. According to Saaty [14], a CR of 0.10 or lower indicates an acceptable level of internal consistency; values above 0.10 suggest that the pairwise comparisons should be reviewed. We stress, however, that a CR at or below 0.10 reflects only the internal logical coherence of the pairwise-comparison matrix in Saaty’s sense and does not guarantee scientific reliability or ecological validity.

In practice, people often find it easier to express their preferences through verbal judgments rather than numerical scales. To make this tendency achievable, AHP links specific linguistic terms of numerical values, thereby assisting decision-makers in articulating priorities. One way is integrating the SMART criterion with AHP. Integrating the SMART criteria (Specific, Measurable, Achievable, Relevant, Time-bound) with the AHP (Analytic Hierarchy Process) method is a powerful strategy for multi-criteria decision-making, combining clarity in goal setting with rigorous mathematical structuring for choosing the best alternative [4,17].

This methodological framework converts broad intentions into clear, actionable strategies by specifying desired outcomes and the metrics used to assess progress. Each component of the SMART framework is briefly outlined below [17]:

Specific: The criterion precisely identifies the element under evaluation and indicates whether the direction of change aligns with the intended management outcomes.

Measurable: The criterion should be associated with quantifiable indicators. These indicators should be accessible through direct measurement, sampling, or systematic monitoring.

Achievable: The criterion considers the spatial and temporal context of the system being evaluated, ensuring that the expected outcomes are realistic given the system’s characteristics and monitoring history.

Relevant: The criterion evaluates whether the focus of assessment is aligned with the overarching goals of the evaluation framework.

Time-bound: The criterion establishes a clear timeframe for achieving the intended goal, thereby creating a sense of urgency and guiding the sequencing of actions.

1.3. Linking AHP and SMART Criteria with Soil Indicators

To evaluate soil quality, soil indicators must be selected in a way that reflects both scientific relevance and practical applicability. These indicators should be assessed according to their importance for ecosystem services and sustainability, their sensitivity to changes in soil functioning, and their relevance to key soil processes [1,7,10]. Despite the growing number of studies proposing soil quality indicators and composite soil health indices, the process of selecting a prioritized subset of indicators under limited financial and analytical resources remains largely dependent on expert judgment and context-specific decision frameworks [8,16].

Decision-making methods like the Analytic Hierarchy Process (AHP) are often and successfully used in environmental assessments [16]. However, their implementation usually requires expert knowledge, which may introduce variability in judgments, increase the time required for analysis, and limit the reproducibility of the procedure [16]. Simultaneously, recent advances in artificial intelligence, particularly large language models, have created new opportunities to support complex decision-making and rapid creation of hierarchical evaluation schemes. Nevertheless, there is currently a lack of systematic studies examining whether AI platforms can consistently generate logically coherent AHP matrices, how their prioritization of soil quality indicators compares across systems, and to what extent such AI-assisted rankings align with established soil science knowledge.

To the best of our knowledge, this is an original study that operationalizes a hybrid AHP–SMART framework within LLM architectures to prioritize soil quality attributes within a sustainability-oriented decision-support context.

The contribution of this research is twofold: first, it applies a controlled repeated-query design to compare the stability, mathematical consistency, and cross-platform agreement of several general-purpose LLMs performing the same AHP–SMART task, demonstrating that internal mathematical consistency (consistency ratio ≤ 0.10) can coexist with unstable single-run weightings and occasional logic inversions. Second, it shows that AI-generated priorities may reflect patterns present in the underlying scientific literature, such as the comparatively lower weighting of the biological indicator considered here (earthworms). Because only one biological indicator was included and training-data content was not analyzed directly, this is presented as a hypothesis requiring broader testing rather than as a demonstrated effect.

Therefore, this work extends beyond a simple ranking exercise by establishing a methodological baseline for evaluating the stability, consistency, and limitations of generative AI in environmental multicriteria decision support.

2. Material and Methods

2.1. Selection of Soil Quality Indicators

To implement the Analytic Hierarchy Process (AHP), we used the S.M.A.R.T. criteria as the main evaluation framework [17]. In this study, the S.M.A.R.T. criteria were applied not as management objectives, but as criteria for assessing the practical suitability of soil quality indicators. To implement the Analytic Hierarchy Process (AHP), a standardized set of seven soil quality attributes was selected to serve as the experimental baseline (Figure 1). To select a representative baseline of soil quality indicators, a systematic search of the literature indexed in the Web of Science Core Collection was performed for records published up to May 2026. The search query string utilized was: “soil quality” OR “soil health” AND “indicator” OR “quality index”, which yielded a total of 5055 initial candidate records.

The selection of indicators followed a multi-stage screening process: (i) Initial identification: records retrieved from the database were evaluated for relevance to multi-criteria soil assessment. (ii) Inclusion/Exclusion criteria: studies were included if they provided empirical or conceptual frameworks for soil health evaluation across physical, chemical, or biological domains and detailed specific measurement protocols. Studies focusing exclusively on non-soil environmental matrices, highly localized heavy metal contamination, or single-crop response trials without generalizable indicator frameworks were excluded. (iii) Candidate indicator mapping and final selection for the AHP baseline: Seven core indicators were selected. From the indicators across the physical, chemical, and biological domains identified from the eligible literature and supplemented by standard institutional monitoring guidelines [1,2,10,18]: Soil Organic Matter (SOM), Soil pH, Available Nutrients, Electrical Conductivity (EC), Aggregate Stability, Water Infiltration and Retention Capacity, and Earthworm Presence.

The inclusion of these seven indicators was justified by two key criteria: (i) their universal recognition and frequency of occurrence in consensus soil-health literature across physical, chemical, and biological domains, and (ii) their feasibility for assessment via standard field or laboratory protocols [1,2,10,11].

Restricting the baseline set to seven indicators was necessary to preserve a parsimonious, computationally manageable matrix and prevent decision fatigue or excessive noise in the Large Language Model (LLM) evaluations. Furthermore, while earthworms were selected as the primary macro-biological indicator due to their straightforward field quantification methods, microbial and enzymatic metrics typically require more complex, labor-intensive laboratory protocols and display higher micro-scale spatial variability. Their exclusion does not indicate lower ecological significance; these indicators might be incorporated into future, after more comprehensive evaluations. A brief conceptualization and methods of analysis of each attribute is provided in Appendix A.

2.2. Selection of AI Platforms

Four widely used, publicly available general-purpose Large Language Models (LLMs), developed by OpenAI, Google, Anthropic, and Microsoft, were selected to facilitate an exploratory cross-platform comparison. Rather than offering a statistically representative or exhaustive sample of the global LLM landscape, this selection provides a representative standard of systems commonly deployed for scientific assistance and decision support. The interface details are: OpenAI GPT-5.5 Instant; Gemini Flash 3, Claude Sonnet 5 with Medium reasoning and Opus 4.8, Microsoft Copilot version 365 with dynamic model selection. All platforms operated under fixed default hyperparameters (e.g., temperature) that could not be modified through the web GUI, with access restricted to a public web interface.

All tests were carried out via the official web user interfaces in May and June 2026 using default platform settings. We always consider the default configurations of each platform, with this operational limitation explicitly noted.

2.3. Experimental Procedure, Prompt Design and Data Analysis

To evaluate output stability, each AI platform was queried ten independent times using an identical prompt. Every run was performed in a newly initiated conversation without conversational memory or additional contextual information. The publicly available version of each platform available during the study period was used throughout the experiment. In one case, an additional instruction was required to complete the requested AHP procedure, as discussed below.

The standardized prompt is presented below. Pairwise-comparison matrices were extracted directly from the completed response sequence without post hoc modification of the numerical values. ChatGPT required one confirmation prompt to continue from the SMART-criteria matrix to the soil-quality-indicator analysis; no other follow-up interaction occurred. Because large language models may generate different outputs even when presented with identical instructions, the results should be interpreted as exploratory observations of platform behavior rather than as deterministic or fully reproducible outputs. This design enabled inter-platform agreement and within-platform variability to be evaluated under comparable experimental conditions.

The AI platforms provided numerical values for the Consistency Index (CI) and Consistency Ratio (CR), calculated using Equations (1) and (2). These values were used to verify whether the generated AHP matrices satisfied the commonly accepted consistency threshold.

Descriptive statistics (mean, median, mode, standard deviation, and coefficient of variation) together with graphical summaries were used to characterize the generated rankings. Because only four AI platforms were compared, inter-platform agreement was assessed using Kendall’s coefficient of concordance (W) and pairwise Spearman rank correlations based on the mean rankings obtained from the ten independent runs. Within-platform variability was quantified using coefficients of variation (CV).

Prompt submitted to the AI platforms:

Hello, I’m working on a project involving the Analytic Hierarchy Process (AHP).

The goal is to establish a hierarchical system of parameters related to soil quality indicators.

As criteria, I’m considering those of the SMART system (specific, measurable, achievable, relevant, and time-bound).

As parameters, I’m considering pH, organic matter, aggregate stability, electrical conductivity, available nutrients, earthworms, and water infiltration and retention capacity.

Assuming you are acting as a soil scientist, could you generate matrices with suggestive AHP values in this context?

I also need you to present, in detail, that is, with explanations and numerical values, the final values for the consistency index and the consistency ratio.

We considered the non-parametric test Kendall’s W (coefficient of concordance) to evaluate the significance and the degree of agreement among the platforms that ranked the same set of soil attributes.

3. Results

3.1. Ranking the S.M.A.R.T. Criteria and Consistency Analysis

The Relevant criterion received the highest mean weight (0.375) and was ranked first by all four platforms. Time-bound received the lowest mean weight (0.076). Inter-platform variability differed considerably among the SMART criteria: it was highest for Specific (CV = 31.7%) and Time-bound (CV = 27.3%), whereas Achievable showed the lowest variability (CV = 4.8%). Thus, although the platforms agreed on the highest-ranked criterion, they differed substantially in the relative weighting of the remaining SMART dimensions (

Table 1. Weights of the SMART criteria assigned by each AI platform.

S.M.A.R.T. Factor ↓ \A.I. Platform →	Claude	ChatGPT	Copilot	Gemini	Average	CV (%)
Relevant	0.363	0.426	0.307	0.406	0.375	14.0
Measurable	0.270	0.251	0.232	0.270	0.256	7.1
Specific	0.164	0.172	0.287	0.157	0.195	31.7
Achievable	0.098	0.094	0.104	0.095	0.098	4.8
Time-bound	0.105	0.057	0.069	0.072	0.076	27.3
Coefficient of Variation—CV (%)	57.7	73.3	53.9	69.1	----

Where dark green cells indicate the highest values within each platform and red cells indicate the lowest values.

).

Despite the observed variation, all platforms produced Consistency Index and Consistency Ratio values below and compatible with the accepted threshold, indicating internal consistency of the pairwise comparison matrices rather than validation of the results themselves (

Table 2. Maximum eigenvalues (λ_max), Consistency Indexes, and Consistency Ratios for SMART factors yielded by tested AI platforms.

	Claude	ChatGPT	Copilot	Gemini
λmax	5.33	5.11	5.12	5.06
Consistency Index	0.083	0.027	0.030	0.015
Consistency Ratio	0.074	0.024	0.027	0.013

).

3.2. Ranking of Soil Quality Indicators and Consistency Analysis

The AI platforms produced divergent results regarding indicator prioritization (Figure 2 and

Table 3. Average final weights across the four AI platforms. In this heat map, cells dark green-painted correspond to the highest values on each AI platform, and cells painted in orange correspond to the lowest values.

Soil Quality Indicator	Claude	ChatGPT	Copilot	Gemini	Average	CV (%)
Soil Organic Matter	0.189 ±0.019	0.224 ±0.042	0.229 ±0.068	0.226 ±0.029	0.217	8.8
pH	0.164 ±0.013	0.162 ±0.045	0.137 ±0.050	0.201 ±0.042	0.166	16.0
Available Nutrients	0.136 ±0.024	0.145 ±0.023	0.163 ±0.045	0.140 ±0.028	0.146	8.3
Water Infiltration and Retention	0.131 ±0.014	0.144 ±0.024	0.144 ±0.044	0.119 ±0.019	0.135	8.8
Aggregate Stability	0.119 ±0.014	0.155 ±0.057	0.149 ±0.045	0.102 ±0.036	0.131	19.3
Electrical Conductivity	0.155 ±0.020	0.113 ±0.059	0.092 ±0.047	0.150 ±0.025	0.128	23.6
Earthworms	0.107 ±0.019	0.057 ±0.012	0.085 ±0.028	0.061 ±0.011	0.078	30.1
Coefficient of Variation (%)	19.6	35.6	33.6	39.6

Where green cells indicate the highest values within each platform (column) and red cells the lowest; the bottom row is each platform’s coefficient of variation across the criteria.

). Evaluating the platform over ten independent runs revealed that soil organic matter earned the highest average global weight (~0.22), whereas earthworms received the lowest (~0.08) (Table 3). While there was significant overall agreement across platforms (Kendall’s W = 0.75, p = 0.007), single-run values varied by up to 50%, demonstrating that reliable results depend on repeated prompting.

Like the results of the SMART approach factors, the consistency index (CI) and consistency ratio (CR) calculation for the soil quality indicator parameters also showed low values, confirming the internal consistency of the generated pairwise-comparison matrices (

Table 4. Maximum eigenvalues (λ_max), Consistency Indexes, and Consistency Ratios for Soil Quality Indicators yielded by the four AI platforms.

	Claude	ChatGPT	Copilot	Gemini
λmax	7.35	7.57	7.45	7.22
Consistency Index	0.058	0.084	0.075	0.056
Consistency Ratio	0.044	0.065	0.057	0.023

). This fact was reported in the analysis of the four AI platforms.

Averaging over the ten runs, the four platforms showed strong and statistically significant agreement in their indicator rankings (Kendall’s W = 0.75, χ² (6) = 18.4, p = 0.007; mean pairwise Spearman ρ = +0.68). At the same time, within-platform variability was high: mean coefficients of variation ranged from about 13% (Claude) to about 34% (Copilot), and individual indicators varied by up to ~50% between runs. The platforms’ averaged rankings therefore converge, whereas single outputs are unstable, confirming that a single query is an unreliable estimate, and that repeated prompting is required.

To test version dependence directly, the same protocol was run on two Claude builds—Sonnet 5 and Opus 4.8 (n = 10 each). The two versions produced almost identical rankings (Spearman ρ = 0.96) but differed in stability and dispersion: Opus 4.8 was markedly more consistent across runs (mean intra-version CV ~5% vs. ~13% for Sonnet 5) and produced flatter weight distributions. Model version thus affected the confidence and granularity of the AI-generated priorities more than their ordinal ranking, empirically confirming that these outputs are version-dependent.

3.3. Performance of the AI Platforms

All four platforms returned numerical AHP matrices accompanied by explanations that were generally internally coherent and directly responsive to the prompt. However, notable differences were observed in the depth of interpretation and the degree of autonomous task completion across platforms (

Table 5. Interpretative explanations provided by AI platforms.

AI Platform	Interpretative Explanations Provided by AI Platforms
Claude Sonnet 5	Among the soil parameters, Available Nutrients and Organic Matter consistently receive top global scores because they excel across all SMART dimensions: they are specific (quantifiable in mg.kg−1 or %), highly measurable (lab and field kits), relevant to crop performance, and respond to management within 1–3 growing seasons. Earthworms, while a powerful biotic indicator, rank lower primarily due to lower measurability and attainability scores, counting protocols are labor-intensive and subject to seasonal variability.
ChatGPT OpenAI GPT-5.5 Instant	Organic Matter and Water Infiltration dominate the Relevance criterion because they influence structure, nutrient cycling, and resilience. The pH, EC, and Available Nutrients score highest in Measurability due to standardized laboratory methods. Earthworms rank lower in Attainability and Measurability because biological sampling introduces variability. Most influential indicators—Organic Matter, Water Infiltration and Retention, Aggregate Stability (representing soil structure), water regulation, and carbon cycling, which are central to soil quality. Moderately important—Available Nutrients, pH. Lower weight indicators: Electrical Conductivity, Earthworms (due to measurement variability.
Copilot 365	The platform did not provide any interpretative explanations.
Gemini Flash 3	Organic Matter—High relevance outweighs lower “Time-bound” scores. Available Nutrients—Balanced performance across all SMART criteria. pH—Carried by extremely high Measurability/Attainability. Water Capacity—Essential for crop health; moderate ease of measurement. Electrical Conductivity—High ease of use, but lower specific relevance to all soils. Aggregate Stability—Critical physical indicator, but harder to measure (Attainability). Earthworms—High biological relevance but lack “Time-bound” precision.

).

ChatGPT was the only platform that did not complete the full analysis autonomously. After generating the SMART criteria matrix, it paused and asked whether the user wished to proceed with the soil quality indicator analysis. Upon confirmation, it completed the remaining matrices. This behavior suggests that ChatGPT’s default configuration may divide complex analytical requests into sequential steps, possibly as a safeguard against generating excessively long outputs in a single response.

Copilot completed both analyses but was the only platform that did not provide any qualitative interpretation of its numerical outputs (Table 5). This absence of interpretative commentary limits the utility of Copilot’s outputs in contexts where methodological justification is required alongside numerical results.

Claude, ChatGPT, and Gemini each provided qualitative explanations for their AHP value assignments, linking indicator scores to specific SMART dimensions (Table 5). Although the mean results were broadly coherent, isolated runs occasionally produced divergent weighting patterns. These deviations did not persist across repetitions and therefore illustrate run-to-run instability rather than a stable platform-specific preference.

Across all platforms, response times were sufficiently rapid to suggest that AI-assisted AHP construction could meaningfully accelerate the early stages of multicriteria environmental assessments. However, such acceleration should be understood as support for preliminary structuring and screening, not as a substitute for expert-based assessment or independent validation.

4. Discussion

4.1. Selection and Representativeness of AI Platforms

In the context of sustainability, continuous monitoring of soil quality indicators prevents erosion and degradation, guiding regenerative management. Utilizing data reduces reliance on chemical inputs and promotes carbon sequestration. It is, therefore, an indispensable strategy for balancing soil use with long-term environmental conservation. The quantitative and qualitative insights generated across the four AI platforms provide a unique perspective on digital soil governance and management. The fact that Soil Organic Matter (SOM) achieved the highest overall priority (≈0.22) demonstrates that LLMs can accurately capture foundational soil paradigms. However, the real innovative value of this cross-platform analysis lies in capturing the high inter-platform variability (e.g., a CV of 31.7% for the ‘Specific’ criterion), proving that AI platforms cannot yet be treated as deterministic or interchangeable expert systems.

The four platforms selected for this study: ChatGPT, Gemini, Claude, and Copilot, represent widely used and freely accessible AI systems as accessed in July 2026, and their selection is broadly consistent with independent popularity data reported by Forbes Brasil [19]. Minor discrepancies between our platform selection and the Forbes ranking, notably the inclusion of Copilot in our study and its absence from that list, may reflect differences in the methodology used to assess platform popularity and the rapid pace of change in the AI landscape. Therefore, the platform selection should be interpreted as a sensible and exploratory sampling strategy rather than as a definitive ranking of global AI platform usage.

A functionally relevant distinction emerged between platforms: while ChatGPT, Claude, and Gemini provided both numerical outputs and qualitative interpretations, Copilot delivered only numerical matrices without accompanying explanation (Table 5). This difference has practical implications for applied environmental assessments, where interpretative transparency is as important as numerical precision. Numerical AHP outputs without explanatory justification may be less useful for expert review, methodological auditing, and decision-making contexts requiring traceable reasoning.

4.2. Ranking the S.M.A.R.T. Criteria and Consistency Analysis

Across all evaluated platforms, the consistency indices and consistency ratios consistently remained below the 0.10 threshold established by Saaty [14], thereby demonstrating the internal logical coherence of the AI-generated pairwise comparison matrices (Table 2). However, this mathematical consistency should not be misinterpreted as empirical validation of scientific accuracy. All four platforms allocated the highest priority vector to the “Relevant” criterion (mean weight: 0.375), underscoring its foundational role in linking specific soil quality indicators to broader land management objectives [17]. This criterion, alternatively operationalized as “Realistic” within variant SMART frameworks, evaluates the degree to which selected metrics align with the overarching objectives of the assessment, rendering it a predictable priority within a decision-support context [4,17].

In one isolated run, the weighting pattern produced by Claude diverged substantially from the remaining outputs: rather than assigning the highest weight to “Relevant,” Claude ranked it lowest (0.05), instead prioritizing “Time-bound” (0.39). Such an isolated divergence should not be interpreted as evidence of hallucination, because no formal hallucination-detection procedure was applied. It is more appropriately described as an unstable or domain-misaligned weighting decision that did not persist across repeated runs, a known limitation of large language models, whose outputs may be internally consistent yet occasionally misaligned with domain knowledge [3,4,20,21]. This finding reinforces the methodological recommendation that AI-generated AHP outputs should be cross-validated across multiple platforms and reviewed by domain experts before use in applied assessments.

Inter-platform variability differed substantially among the SMART criteria, ranging from 4.8% for Achievable to 31.7% for Specific. This pattern indicates that the platforms agreed relatively closely on some dimensions but interpreted others, particularly Specific and Time-bound, less consistently. This variability may reflect differences in model design, training data, response generation procedures, and stochastic output variability, rather than genuine disagreement about soil science principles.

4.3. Ranking of Soil Quality Indicators and Consistency Analysis

The AI-generated rankings of soil quality indicators showed partial alignment with expert-based assessments reported in the literature, suggesting that AI-assisted AHP may be useful as an exploratory tool for preliminary indicator screening [1,10,21].

Soil Organic Matter (SOM) received the highest average priority weight across platforms (≈0.22), consistent with its established status as a primary soil quality indicator [10,11]. This result is also broadly consistent with expert-based AHP assessments: Çelik and Sürücü [22] reported SOM weights of approximately 0.24 for orchard and arable soils, and 0.37 for pastures, values of a comparable order of magnitude to those generated here. The convergence between AI outputs and expert-derived rankings for SOM.

pH and Available Nutrients ranked second and third overall, although their average weights were very similar. Available Nutrients showed notably low inter-platform variability (CV = 8.3%), indicating strong agreement across AI systems regarding its importance. This stability may reflect the direct connection between nutrient availability and soil fertility, plant growth, and land management decisions.

Earthworm presence received the lowest average weight (0.08) and the highest inter-platform variability among indicators (CV = 30.1%), reflecting the well-documented challenges associated with biological indicators in soil quality assessment [23,24,25]. Earthworms are highly sensitive to edaphic conditions, climate, land use, and seasonal fluctuations, and their quantification requires labour-intensive field protocols that are difficult to standardize [23,26]. These characteristics may reduce their measurability and achievability scores within the SMART framework, which likely contributed to their low average ranking and high variability across platforms.

Within the context of ecological restoration monitoring, the secondary priority assigned to earthworm presence does not necessarily denote negligible ecological significance; [23,24]. Rather, it reflects the pragmatic constraints of deploying this metric in rapid, resource-limited assessments. Although biological indicators are highly efficacious for evaluating ecosystem trajectory and recovery [9,18,24,27], their robust interpretation typically requires rigorous seasonal controls, longitudinal sampling designs, and specialized taxonomic expertise. Consequently, earthworm presence is optimized as a complementary indicator within high-resolution monitoring frameworks, rather than as a primary screening metric in initial assessments.

4.4. Interpretative Capacity of AI Platforms

The qualitative interpretations provided by ChatGPT, Claude, and Gemini (Table 5) were generally technically coherent and broadly aligned with established soil science knowledge. This suggests that these platforms may be able not only to generate numerical AHP matrices, but also to provide domain-relevant justifications for their outputs. This interpretative capacity is particularly valuable in resource-limited assessment contexts, where decision-makers may need transparent explanations to support the evaluation of numerical rankings. However, such explanations should not be treated as independent validation, because plausible reasoning may still accompany questionable or biased weighting decisions.

The lower average ranking of biological indicators across AI-generated outputs is broadly consistent with trends reported in the soil science literature. Schweng et al. [28] note that digital soil mapping efforts have focused predominantly on organic carbon and chemical properties using remote sensing, largely omitting biological attributes. Through a bibliometric analysis Gatica-Saavedra et al. [25] found that among commonly used soil health indicators, chemical indicators appear in 76% of studies, physical indicators in 58%, and biological indicators in 55%, with earthworms specifically ranked below soil enzymes and microbial respiration in terms of frequency of use. The lower prevalence of biological indicators in applied assessments is attributed to their high spatiotemporal variability, the time required for data collection, and the need for specialized methods and expertise [25,27,28].

Taken together, these findings suggest that AI platforms may reflect, and in some cases could reinforce, the existing biases of the scientific literature on which they were trained. This has implications for the use of AI in environmental decision-making: outputs should be interpreted not only in terms of their internal consistency, but also in relation to the scope, balance, and representativeness of the knowledge base they draw upon. For this reason, AI-generated rankings should be considered preliminary decision-support outputs rather than definitive expert judgments.

4.5. Mechanisms Underlying Inter-Platform Differences

The observed differences may arise from a combination of model-specific factors and stochastic generation, including differences in training corpora, alignment procedures, default inference configurations, and the way each model interprets the prompt. Because these mechanisms were not experimentally isolated, they should be considered plausible explanations rather than demonstrated causes. Soil organic matter (SOM) consistently received the highest overall ranking because it is widely recognized as one of the most comprehensive indicators of soil quality and ecosystem functioning [1,10]. SOM plays a crucial role in climate change mitigation by acting as a valuable carbon reservoir [27,28]. Maintaining and rebuilding SOM is, therefore, essential to ensuring long-term food security and environmental balance. SOM directly influences numerous physical, chemical and biological soil properties, including nutrient cycling, aggregate stability, water retention, microbial activity, carbon sequestration and crop productivity [10,11]. Its dominant position is therefore consistent with decades of soil science research and with previous expert-based soil quality assessment frameworks, suggesting that the AI platforms reproduced the broad scientific consensus regarding the central role of soil organic matter.

In contrast, biological indicators, particularly earthworm abundance, exhibited substantially greater variability among platforms. Earthworms are widely accepted as valuable indicators of soil biological quality because they strongly influence soil structure, organic-matter decomposition, nutrient cycling and ecosystem functioning [29,30]. However, their practical use is considerably more context-dependent than that of most physicochemical indicators: earthworm populations are strongly affected by soil type, climate, moisture conditions, agricultural management, land use and seasonal variability, and their assessment requires standardized field sampling protocols that are not routinely included in many soil monitoring programs [29]. The greater variability observed among AI platforms may therefore reflect the broader diversity of scientific perspectives represented in the

The platforms also differed in how they expressed their judgements. Copilot combined the highest run-to-run variability with the absence of any qualitative justification, whereas Claude assigned comparatively greater importance to biological indicators such as earthworm presence, indicating a broader interpretation of ecological soil functioning. These differences should not be interpreted as evidence that one platform is scientifically superior to another; rather, they demonstrate that different large language models may assign different relative importance to the same evidence because they are developed using different training corpora, optimization procedures and alignment strategies [19,20]. Comparable variability has been reported in recent comparative evaluations of LLM performance across scientific and decision-support tasks [28,30].

From a practical perspective, these findings indicate that AI platforms can produce internally consistent AHP matrices and may substantially accelerate the preliminary stages of multicriteria environmental assessment. Nevertheless, mathematical consistency alone does not guarantee ecological validity or scientific correctness. AI-generated rankings should therefore be regarded as structured preliminary decision-support tools rather than substitutes for expert judgement, and cross-platform comparison, repeated prompting, transparent reporting of model versions and independent expert verification remain essential before AI-generated AHP rankings are incorporated into practical soil quality assessment or environmental management [13,14].

5. Final Remarks

Through this study we evaluated the operational viability and constraints of utilizing Large Language Models (LLMs) within a hybrid AHP–SMART framework for soil quality indicator prioritization. The major conclusions are as follows:

(i) Mathematical Consistency vs. Scientific Validity: While all tested LLM platforms generated mathematically consistent pairwise comparison matrices (CR ≤ 0.10), internal logical coherence does not inherently guarantee ecological validity or domain accuracy.

(ii) Output Instability and Stochasticity: Single-run LLM outputs are inherently unstable, showing within-platform coefficients of variation up to ~34% and occasional logic inversions. Consequently, separate queries are unreliable, and consistent decision-support requires a protocol based on repeated prompting and cross-platform benchmarking.

(iii) Replication of Literature Biases: The AI-generated rankings prioritized physicochemical attributes, led by SOM (global weight ≈ 0.22), while consistently assigning lower weights to biological metrics like earthworms (≈0.08). This reflects and potentially reinforces existing biases in published literature regarding the measurement feasibility of biological indicators.

(iv) Practical Guidance: Generative AI should be viewed strictly as a rapid, low-cost screening tool for initial model structuring rather than an autonomous expert system. Operational deployment in environmental decision-making should ask for expert confirmation, site-specific calibration, and explicit reporting of model versions.

Limitations: A methodological limitation of this study is that inference parameters were governed by the default settings of each platform’s web interface and could not be manually constrained or extracted. While this mirrors the typical user experience of practitioners relying on standard web GUIs for environmental decision support, future studies utilizing API access should explicitly control temperature to further isolate model variance. Although each platform was queried ten times, the four systems remain a convenience sample of mainstream free-tier platforms and do not cover the full AI landscape. Outputs are version- and configuration-dependent and temporally unstable, so the results describe the specific builds accessed in this study. The analysis relies on AI-generated pairwise comparisons without an independent expert-derived benchmark; agreement with the published literature and the internal cross-platform concordance therefore serve as indirect validity checks rather than a formal ground truth. Finally, the analysis is generic rather than site-specific.