From Data Quality to Quality of Agentic Data Use: A Conceptual Framework for Agentic Data Engineering

Featured Application

The proposed framework can guide the design of enterprise data agents that generate and execute SQL or data pipelines while enforcing task-specific authorization, canonical business semantics, runtime validation, provenance, and governance controls.

Abstract

Large language models and AI agents are extending data-engineering automation beyond isolated artifact generation toward end-to-end processes in which agents interpret requirements, select data, generate transformations, invoke tools, validate results, and communicate analytical outputs. This shift introduces risks that conventional notions of data quality and execution success do not fully capture. A dataset may satisfy established quality standards, and a generated query may execute without technical errors, while the agent still selects an incorrect metric, combines incompatible analytical grains, accesses unauthorized data, or draws conclusions that are insufficiently supported by evidence. This paper develops a conceptual framework for Agentic Data Engineering centered on Quality of Agentic Data Use, defined as the extent to which an agent uses and communicates data in accordance with task, semantic, quality, security, governance, and provenance requirements. An evidence-informed analysis of Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance shows that these foundations provide essential but fragmented capabilities. The proposed framework integrates and extends them through four core artifacts: Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. It also introduces an execution lifecycle, a reference architecture, a failure taxonomy, and a multidimensional evaluation framework. A governed sales-analysis scenario illustrates how the proposed artifacts interact throughout an agent-mediated data process. In addition, a controlled Databricks prototype and a complementary benchmark comprising 10 cases and 40 executions demonstrate the framework’s technical feasibility and support the independent computation of enforcement indicators. The benchmark highlights the value of separating generation from validation while also showing that the current validation and automated-repair mechanisms require further calibration. These preliminary findings do not establish generalized improvements in safety, correctness, or reliability. Rather, they provide an operational foundation for broader empirical evaluation of trustworthy agent-mediated data-engineering processes.

1. Introduction

Data engineering has evolved in response to the need to collect, integrate, transform, store, and distribute increasing volumes of data. Data warehousing established architectures for integrating operational sources, while dimensional modeling organized analytical data around business processes, declared grain, dimensions, and facts [1]. Data quality approaches later introduced mechanisms to detect incomplete, inconsistent, invalid, or outdated data [2,3,4]. More recently, DataOps, data products, and Data Mesh have emphasized automation, domain ownership, federated governance, and explicit agreements between data producers and consumers [5,6,7,8].

Despite these changes, requirements interpretation and key design decisions have remained largely under human responsibility. Data engineers, architects, analysts, and data stewards determine authoritative sources, analytical grain, official metrics, acceptable transformations, and conditions requiring human review [1,9,10]. Data platforms then execute SQL queries, pipelines, quality tests, access controls, and orchestration workflows designed or approved by these professionals [1,9].

Generative artificial intelligence is changing this distribution of work. Large language models can generate SQL and transformation code, interpret schemas, produce documentation and tests, enrich metadata, and assist in diagnosing execution errors. Agent architectures extend these capabilities through planning, memory, tool use, environmental observation, and iterative correction [11]. Recent proposals characterize data agents as systems capable of reasoning over data, queries, environments, and tools while coordinating multiple stages of a data workflow [12]. Similarly, emerging benchmarks assess whether agents can configure ingestion tools, generate transformations, and construct complete ELT pipelines rather than isolated SQL statements [13].

An agent may receive a natural-language business requirement and perform several activities throughout the analytical process. It can infer user intent, inspect metadata, catalogs, schemas, or semantic models, and identify candidate datasets, tables, attributes, relationships, and business concepts [12,14]. It can then generate SQL or transformation code, invoke data tools, execute artifacts, inspect results, and iteratively refine the solution when errors or inconsistencies are detected [13,14]. Finally, it can synthesize and communicate the result through tables, reports, visualizations, or narrative explanations [15,16]. Thus, the agent participates not only in code generation but also in task interpretation, data selection, tool use, validation, and result communication.

These capabilities create substantial opportunities. Natural-language interfaces can reduce barriers to accessing enterprise data, while agentic systems can automate data-engineering tasks and iteratively diagnose and repair generated artifacts [12,13,17]. However, they also introduce risks not captured by conventional data-quality checks or execution success. Text-to-SQL systems may generate semantically valid but semantically incorrect queries, including incorrect joins, filters, or aggregations [18,19]. Agents may also select non-canonical metrics, combine incompatible grains, use data that are not suitable for the declared purpose, or execute transformations that violate business rules [1,10,20]. In addition, they may produce plausible results without preserving the artifacts, evidence, and validation records necessary for reproducibility and auditability [21].

These failures expose a fundamental distinction: data quality does not guaranty the quality of an agent’s use of data. Current evaluations of data agents support this distinction. Although Text-to-SQL systems have improved considerably, production-oriented studies continue to report ambiguous intent, hallucinated columns, incorrect joins, inappropriate aggregations, and values inconsistent with database content [14,22,23,24,25]. End-to-end data engineering is also more difficult than isolated SQL generation. ELT-Bench showed that the best evaluated configuration correctly generated only a limited subset of target data models, despite stronger performance in extraction and loading [13]. These results indicate that executable code and successful tool calls are insufficient indicators of reliable data use.

In this paper, we adopt a broad enterprise-data interpretation of Agentic Data Engineering as the engineering activities required to specify, generate, execute, validate, govern, and audit data processes in which AI agents act as producers, transformers, operators, validators, or consumers of governed data artifacts. This interpretation integrates concepts from Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, Design by Contract, and Data Provenance. The central problem is how these foundations should be connected when a probabilistic agent operates on enterprise data under delegated authority.

The term Agentic Data Engineering has recently appeared with different scopes. Luo et al. use Autonomous Agentic Data Engineering to describe an LLM agent that executes an end-to-end training-data curation loop for model specialization, including planning, synthetic data generation, validation, repair, and iterative optimization based on downstream model performance [26]. Their work shows that agents can perform non-trivial data-engineering activities while exhibiting failures related to invalid submissions, distribution shift, semantic inconsistency, and insufficient quality assurance. Our perspective is broader and enterprise-oriented, focusing on how agents access, select, interpret, transform, combine, validate, and communicate governed data under delegated authority. Accordingly, our contribution is not the introduction of the term, but a conceptual framework for specifying, governing, validating, and auditing agent-mediated data processes.

This paper makes five contributions. First, it defines Quality of Agentic Data Use as a quality concept centered on the interaction among an agent, a task, and enterprise data. Second, it analyzes six existing foundations and identifies both their capabilities and the remaining gaps. Third, it proposes four complementary artifacts: Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. Fourth, it introduces an execution lifecycle, a reference architecture, and a failure taxonomy. Finally, it proposes evaluation dimensions and a research agenda for future empirical work.

Figure 1 illustrates the progressive evolution of data engineering according to the degree of autonomy assigned to AI systems. The process begins with human-specified workflows and progresses through AI-assisted and delegated stages toward agentic and supervised-autonomous data processes.

Figure 1. Evolution from human-specified data engineering to AI-assisted, delegated, agentic, and supervised-autonomous data processes.

The remainder of this paper is organized as follows. Section 2 introduces the main concepts underlying data engineering, AI agents, autonomy, and Quality of Agentic Data Use. Section 3 describes the evidence-informed research approach and conceptual synthesis used to develop the framework. Section 4 examines Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance as complementary foundations for trustworthy agent-mediated data processes. Section 5 presents the proposed framework, formalizes Quality of Agentic Data Use, and defines its four core artifacts: Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. Section 6 describes the execution lifecycle and reference architecture for controlled agentic data processes. Section 7 illustrates the framework through a governed quarterly sales-analysis scenario. Section 8 presents a taxonomy of agentic data failures, while Section 9 introduces a multidimensional evaluation framework. Section 10 discusses the contribution boundaries, research findings, practical adoption, professional implications, and the limitations of the evidence. Finally, Section 11 summarizes the conclusions and outlines directions for future work.

2. Background

This section explains the concepts required to understand the proposed framework. First, the evolution of data engineering is described. Second, the characteristics of AI agents are presented. Finally, the relationship between data quality and the quality of agentic data use is discussed.

2.1. Data Engineering and Its Main Artifacts

Data engineering encompasses the design and operation of systems that collect, integrate, transform, store, and deliver data to operational, analytical, and machine-learning consumers [9,10]. Its core artifacts include data models, ETL/ELT pipelines, data-quality rules, metadata catalogs, semantic models, access-control policies, lineage and provenance records, tests, and observability mechanisms [9,10,21]. Together, these artifacts support data accessibility, reliability, traceability, security, and fitness for operational and analytical use [9,10].

Dimensional modeling illustrates the importance of prior semantic and design decisions. Its design process requires selecting a business process, declaring the grain, identifying dimensions, and defining facts [1]. Consequently, a technically valid query is not necessarily analytically correct; correctness also depends on the appropriate selection of process, grain, dimensions, and measures. This principle remains critical when SQL queries and analytical workflows are generated autonomously by AI agents.

Data quality extends these concerns through dimensions such as completeness, validity, consistency, uniqueness, accuracy, and timeliness [2]. Modern data systems operationalize these dimensions as executable checks capable of detecting schema changes, anomalous distributions, and missing values at scale [3,4]. Data Contracts further formalize producer–consumer agreements through specifications covering schema, quality, ownership, versioning, and service-level objectives [7,8]. Semantic models and ontologies complement these mechanisms by providing consistent representations of business concepts and their relationships [27].

These mechanisms were primarily designed for data products, pipelines, and human-governed applications. Although they remain essential in agentic settings, their control boundaries do not necessarily capture the agent’s task-specific decisions and behavior during data access, transformation, validation, and communication.

2.2. AI Agents in Data Processes

An AI agent is a software system capable of interpreting an objective, planning a sequence of actions, invoking tools, observing execution results, and adapting its behavior accordingly. ReAct is an early approach that integrates reasoning and acting through iterative interaction with an environment [11]. More recent data-oriented architectures extend this paradigm to data ecosystems. The Data Agent architecture, for instance, combines understanding of data, queries, tools, and execution environments with workflow planning and execution [12]. Evidence from dynamic agent-evaluation environments further shows that successful execution does not necessarily imply reliable numerical reasoning. In the Agent Trading Arena, LLM-based agents operated in a closed-loop financial environment in which their decisions influenced subsequent market states. The study reported persistent difficulties in interpreting percentage changes, relational numerical patterns, and long-term dependencies, supporting the need to evaluate observable agent behavior throughout the interaction rather than only the plausibility of the final response [28].

In data engineering, an agent may assume multiple roles within the same workflow. It can consume data through retrieval and interpretation; produce derived datasets, analytical outputs, or reports; transform data by generating SQL queries, mappings, or pipelines; operate platforms through tool selection and invocation; validate results by detecting execution errors, inconsistencies, or anomalous outputs; and communicate findings through visualizations, explanations, or recommendations [12,14,15,29]. These roles increasingly coexist because data agents integrate semantic interpretation, planning, tool use, execution, validation, and result synthesis within a unified process [12,29]. This integration increases system capability but also enlarges the set of decisions and actions that require control, traceability, and auditability.

Different levels of autonomy must also be distinguished. Existing work characterizes autonomy as a continuum determined by the degree of user control, the actions an agent is authorized to perform, and the oversight or approval mechanisms required [30,31]. Based on this distinction, this work defines four operational levels for data-engineering processes. An AI-assisted system generates recommendations that are reviewed and executed by a human. A delegated system executes a bounded task within an explicitly constrained environment. An agentic system can plan, invoke tools, validate intermediate results, and adapt its actions while remaining subject to authorization and policy constraints. Finally, a supervised-autonomous system can operate continuously, while preserving human accountability, runtime monitoring, escalation, auditability, and revocation mechanisms [20,31]. Table 1 summarizes these levels of autonomy in data engineering.

Table 1. Levels of autonomy in data engineering.

Level	System Capability	Human Role	Typical Execution Authority
AI-assisted	Generates suggestions or candidate artifacts.	Reviews, modifies, and executes.	None or local preview.
Delegated	Executes a bounded task in a controlled environment.	Defines the scope and approves exceptions.	Read-only or sandboxed.
Agentic	Plans, selects tools, executes, validates, and adapts.	Supervises execution and manages escalations.	Policy-bounded multi-step execution.
Supervised-autonomous	Operates continuously within defined policies and objectives.	Maintains accountability, audits outcomes, and intervenes when required.	Continuous but revocable authority.

The table should not be interpreted as a maturity path that must end in unrestricted autonomy. In high-impact processes, supervised autonomy can be more appropriate because it combines adaptation with human accountability.

2.3. Data Quality and Quality of Agentic Data Use

Wang and Strong argue that data quality should be evaluated in relation to data-consumer requirements rather than reduced to accuracy alone [2]. This perspective is particularly relevant to agentic systems because the agent introduces an additional decision layer between governed data and the final output. A dataset may satisfy established quality criteria while the agent selects an inappropriate source, misinterprets a business concept, or produces a conclusion unsupported by the available evidence.

In this paper, we define Quality of Agentic Data Use as the degree to which an AI agent accesses, selects, interprets, transforms, combines, and communicates data in accordance with declared task, semantic, quality, security, governance, and provenance requirements. The construct integrates concerns established in data quality, data governance, semantic modeling, AI risk management, and provenance research [1,2,10,20].

The construct comprises six dimensions. Task alignment assesses whether the agent’s actions and outputs are consistent with the declared objective and intended data use [20]. Data authorization assesses whether accessed assets, records, and attributes are permitted for the user, task, and purpose [10,20]. Semantic correctness evaluates whether metrics, dimensions, analytical grain, joins, aggregations, and temporal interpretations conform to their defined business semantics [1,10,18]. Action validity evaluates whether transformations, tool invocations, and side effects comply with technical constraints and business rules [9,20]. Evidence grounding assesses whether claims and recommendations are supported by retrieved data, execution results, and validation evidence [32]. Finally, auditability assesses whether sufficient information about inputs, transformations, tool actions, decisions, and outputs is preserved to support reconstruction and review [10,21].

Table 2 distinguishes this concept from adjacent forms of quality.

Table 2. Different forms of quality in data and AI systems.

Quality Concept	Principal Unit	Main Question
Data Quality	Dataset or data item	Do the data satisfy the expected properties?
Pipeline Quality	Transformation process	Does the process execute and transform data correctly?
Model Quality	Statistical or AI model	Does the model satisfy predictive and behavioral criteria?
Output Quality	Returned artifact	Does the output satisfy accuracy, format, and usability requirements?
Quality of Agentic Data Use	Agent–data–task interaction	Did the agent use and communicate the data correctly and under the authorized conditions?

These types of quality are complementary. Agentic data use cannot be trustworthy if the input data are defective. However, high-quality data are not sufficient when the agent can make incorrect semantic or operational decisions.

3. Research Approach

This study follows an evidence-informed conceptual research approach. Its purpose is to analyze established concepts, standards, architectures, and recent studies in order to identify the requirements introduced when AI agents participate actively in enterprise data processes. The study is not presented as a systematic literature review, and its principal objective is conceptual development rather than statistical or experimental validation.

The analysis was guided by four research questions:

• RQ1: Which tasks, decisions, and artifacts of data engineering are being transformed by large language models and AI agents?
• RQ2: To what extent do Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance support trustworthy agent-mediated data processes?
• RQ3: Which additional artifacts are required to specify, validate, govern, and audit the use of enterprise data by an AI agent?
• RQ4: How should agentic data processes be evaluated beyond syntactic validity and execution success?

3.1. Evidence Identification and Selection

A structured, multi-source literature search was conducted during June and July 2026 to support the conceptual analysis. The search combined AnswerThis-assisted retrieval from OpenAlex and Semantic Scholar with manual searches in IEEE Xplore, the ACM Digital Library, and MDPI. The search covered Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, Data Provenance, data agents, Text-to-SQL systems, autonomous data pipelines, human oversight, and agent evaluation.

Priority was given to peer-reviewed articles, established conference proceedings, standards, and primary technical sources. Recent preprints and industrial documentation were included when addressing emerging mechanisms for which peer-reviewed evidence remains limited. Secondary sources and LLM-assisted retrieval were used only to identify candidate publications; claims incorporated into the manuscript were verified against the original sources.

Publications were retained when they contributed at least one of the following: a relevant conceptual definition; an architecture, standard, or machine-readable mechanism; an implementation or evaluation of an agent-mediated data process; or evidence concerning semantic, authorization, tool-use, governance, provenance, or human-oversight failures. Foundational publications predating the main search period were retained when necessary to establish core concepts.

The objective was to achieve sufficient conceptual coverage rather than an exhaustive bibliographic census. Publications that informed the motivation or terminology but did not directly support a specific claim were not necessarily cited in the final manuscript. Appendix B details the search sources, selection criteria, LLM-assisted organization, and author-review process.

3.2. Conceptual Analysis

The evidence was organized into six foundations: Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance. These foundations were selected because they provide complementary structural, semantic, behavioral, organizational, and traceability control mechanisms. They are not intended to exhaust the relevant research areas, but to provide a consistent analytical basis for examining existing mechanisms and deriving agent-specific requirements.

Each foundation was analyzed according to its scope, primary unit of control, implementation artifacts, enforcement mechanisms, contribution to agent-mediated data processes, and remaining limitations. The resulting comparison is therefore a qualitative conceptual assessment of the mechanisms reported in the literature and should not be interpreted as an empirical ranking or as evidence of their relative effectiveness.

3.3. Framework Derivation

The framework was derived by consolidating the limitations identified across the six foundations into recurrent requirements concerning task purpose, authorization, semantic correctness, permitted operations, behavioral validation, evidence, accountability, and auditability. These requirements were then organized into four engineering functions:

• specification of the conditions governing delegated data use;
• behavioral validation of observable agent actions and outputs;
• traceability of tasks, artifacts, tool calls, validations, interventions, and outcomes;
• governance of authority, ownership, approval, escalation, and lifecycle control.

These functions resulted in Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. Quality of Agentic Data Use was introduced as the integrating construct for evaluating the complete agent–task–data interaction. The lifecycle, reference architecture, failure taxonomy, and evaluation framework were subsequently derived by organizing these functions around an end-to-end agent-mediated data process.

3.4. Methodological Scope

The approach enables a structured conceptual comparison and the derivation of theoretically grounded artifacts, but does not establish their effectiveness across operational contexts. The illustrative scenario demonstrates the internal coherence of the framework, while the controlled Databricks prototype provides preliminary evidence that selected authorization and policy violations can be detected under the implemented conditions. Neither constitutes evidence of generalized improvements in safety, correctness, or reliability.

Broader validation will require reproducible task corpora, implemented contracts and expectations, heterogeneous data domains and failure conditions, multiple evaluators, inter-rater agreement, and controlled comparisons with alternative configurations. The implications and limitations of the conceptual and prototype-based evidence are discussed in Section 10.

4. Analysis of Existing Foundations

This section analyzes six foundations relevant to trustworthy agent-mediated data processes: Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance. Following the dimensions defined in Section 3.2, the analysis examines their principal units of control, contributions, applicability to agentic systems, and remaining limitations. The objective is to identify which existing mechanisms can be reused and where task-bound integration or agent-specific extensions are required.

4.1. Data Contracts

Data Contracts formalize producer–consumer agreements by specifying schemas, ownership, quality requirements, service-level objectives, versioning, and responsibilities [7,8]. Their primary unit of control is the data product or producer–consumer interface. Because they are machine-readable and versioned, they support discovery, automated validation, testing, and cross-domain governance [7].

However, conventional Data Contracts primarily specify what a data product provides rather than how an agent may use it within a particular task. The validity of an attribute, for example, does not determine whether an agent is authorized to use it for a given purpose, combine it with other data, or transmit derived results through an external tool. Agent-mediated processes therefore require task-specific extensions covering purpose, authorization, semantics, permitted transformations and tools, validation requirements, evidence, and escalation conditions. Data Contracts provide a structural foundation for these controls, but do not by themselves govern delegated data use.

4.2. Semantic Layers

Semantic Layers encode governed metrics, dimensions, relationships, aggregation rules, temporal roles, and business terminology independently of physical data structures [33,34]. Ontologies provide a related machine-interpretable representation of domain concepts, relationships, and constraints [27]. Their primary unit of control is therefore the business concept or semantic relationship.

Semantic Layers help agents map natural-language requests to approved definitions, calculation rules, join paths, and data sources, reducing ambiguity and improving schema linking [17,33,34,35]. However, semantic availability does not ensure semantic compliance. An agent may access a governed definition yet still select an incorrect metric, omit required filters, use an incompatible join path, or apply an invalid temporal interpretation [18,19]. Independent behavioral validation is therefore required to verify that governed semantics are both selected and implemented correctly.

4.3. Data Quality

Data Quality evaluates properties of datasets and pipelines, including completeness, accuracy, consistency, validity, uniqueness, timeliness, and distributional conformity [2,3,4,10]. Frameworks such as Deequ and TensorFlow Data Validation operationalize these properties through profiling, constraints, anomaly detection, and executable rules that can be applied before and after agent execution [3,4,36].

However, data quality is distinct from correctness of use. A valid and timely dataset may still be inappropriate for a specific task; multiple valid sources may conflict; and valid measures or timestamps may be applied at an incorrect grain or temporal role. This distinction is consistent with the established view of data quality as contextual fitness for use [2,37]. Agentic systems therefore require complementary behavioral measures, including metric-selection accuracy, grain and temporal-role compliance, authorization compliance, evidence coverage, and provenance completeness [20,32].

4.4. Guardrails

Guardrails constrain AI-system inputs, outputs, tool calls, and actions. They can sanitize prompts, validate structured outputs, restrict tool invocations, enforce permissions, and block unsafe or unauthorized operations [38,39,40]. Their primary unit of control is the runtime interaction point at which a rule or approval condition can be enforced. Model-independent frameworks such as NeMo Guardrails make these policies explicit, inspectable, and reviewable [20,38].

However, general-purpose guardrails do not inherently capture enterprise-data semantics. They may prevent a destructive SQL operation while failing to detect a read-only join that duplicates measures, the use of a non-canonical metric, or a privacy risk arising from the combination of individually authorized attributes [18,19,41,42]. Data-aware guardrails therefore require contextual rules derived from contracts, Semantic Layers, quality systems, authorization policies, and provenance records. Declarative semantic constraints can operationalize this context as independently enforceable rules over metrics, sources, joins, grain, temporal roles, purpose, and outputs [20,41].

4.5. AI Governance

AI Governance encompasses the policies, roles, responsibilities, and controls used to manage AI-system risks throughout the lifecycle. Its primary unit of control is the AI system within its organizational and risk context. Frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 address accountability, oversight, documentation, autonomy, monitoring, incident management, and change control [20,30,43].

However, these frameworks generally operate at a higher level of abstraction than individual data actions. They may require transparency or human oversight without specifying how an agent should validate a metric, join path, grain, temporal role, or source-precedence rule, or which execution events should trigger intervention. Agentic Data Governance must therefore link organizational policies to task-specific controls, ownership, approved tools, autonomy levels, escalation conditions, exceptions, monitoring, rollback, override, and revocation mechanisms [44,45,46].

4.6. Data Provenance

Data Provenance records the origin, generation, use, and transformation of data artifacts. W3C PROV models entities, activities, agents, and their relationships, providing an interoperable basis for reproducibility, debugging, compliance, and process reconstruction [21,47,48]. In agentic processes, provenance can capture datasets, generated artifacts, tool calls, validation results, outputs, and the applicable versions of agents, models, contracts, policies, validators, and execution environments [49,50].

Conventional lineage may identify the data and transformations involved without capturing the full decision context, including available alternatives, selection constraints, authorizing contracts, repair attempts, or human interventions. Agentic Data Provenance therefore extends lineage with observable records of tasks, actions, validations, evidence, interventions, and outcomes. It does not require private chain-of-thought, but rather externally inspectable evidence sufficient to reconstruct the process and assess compliance [20,49,50].

4.7. Integrated Comparison

Table 3 compares the principal unit of control, contribution, and remaining limitation of each foundation.

Table 3. Comparison of foundations for agentic data systems.

Foundation	Principal Unit	Main Contribution	What Is Not Fully Addressed
Data Contracts	Producer–consumer agreement or data product	Schema, quality, ownership, versioning, and service-level objectives.	Task-bound use, tool permissions, interpretation, and execution evidence.
Semantic Layers	Business metric, dimension, or concept	Governed definitions, hierarchies, relationships, and aggregation rules.	Verification that the agent selected and implemented the intended meaning.
Data Quality	Dataset, field, or pipeline stage	Property validation, anomaly detection, profiling, and drift monitoring.	Correctness of the agent’s interpretation, contextual use, and decision.
Guardrails	Input, output, tool call, or action	Runtime restrictions, validation, and policy enforcement.	Domain-specific data semantics, analytical grain, and metric correctness.
AI Governance	AI system and risk context	Accountability, oversight, documentation, monitoring, and lifecycle control.	Operational validation rules for a specific data action.
Data Provenance	Data artifact and transformation history	Origin, lineage, responsibility, traceability, and process reconstruction.	Complete task, alternative-selection, and decision context for agent actions.

The six foundations address complementary units of control. Data Contracts and Semantic Layers provide explicit structure and governed semantics; Data Quality and Guardrails enable executable validation and runtime enforcement; and AI Governance and Data Provenance support accountability, lifecycle control, traceability, and process reconstruction.

Their main limitation is fragmentation. Task purpose, agent identity, data authorization, business semantics, tool permissions, validation outcomes, evidence, and accountability are typically distributed across separate mechanisms and repositories. No single foundation integrates these elements into a task-specific assurance process covering the complete agent–task–data interaction. This gap motivates the integrated framework introduced in the following section.

5. Proposed Framework for Agentic Data Engineering

The proposed framework organizes existing mechanisms around the concept of Quality of Agentic Data Use and introduces four complementary artifacts: Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. Each artifact serves a distinct engineering function. Agentic Data Contracts specify the authorized and expected use of data; Agentic Expectations assess whether observable agent behavior complies with those specifications; Agentic Data Provenance records execution events, validations, evidence, and externally relevant decisions; and Agentic Data Governance defines authority, accountability, ownership, approval, escalation, and lifecycle controls.

The framework does not replace the six foundations examined in the previous section. Rather, it integrates and extends their capabilities around a different unit of engineering concern: the interaction among an agent, a task, governed data, semantic definitions, tools, and evidence. This perspective is necessary because a dataset, pipeline, or generated artifact may satisfy its individual technical requirements while the agent’s overall use of data remains semantically incorrect, unauthorized, insufficiently supported, or unauditable.

5.1. Quality of Agentic Data Use

Definition 1

(Operational definition of Quality of Agentic Data Use). Let (t) denote a task with declared purpose (p), and let (a) denote an agent operating under a contract (C(t,a)). The contract binds the task and agent to a set of authorized data assets (D), governed semantic definitions (S), permitted operations and tools (O), and evidence obligations (E).

An agent-mediated data process achieves a given level of Quality of Agentic Data Use to the extent that the observable behavior of (a), including its data selections, generated artifacts, transformations, tool invocations, validation responses, and communicated results, satisfies the requirements represented by:

$$[\mathrm{Q}(\mathrm{t},\mathrm{a})=(\mathrm{p},\mathrm{D},\mathrm{S},\mathrm{O},\mathrm{E}).]$$

The dimensions of the construct correspond to verifiable subsets of these requirements:

• Task alignment evaluates whether the agent’s actions and outputs are consistent with the declared purpose (p).
• Data authorization evaluates whether the selected assets, records, and attributes are permitted under (D).
• Semantic correctness evaluates whether metrics, dimensions, analytical grain, joins, aggregations, and temporal interpretations conform to (S).
• Action validity evaluates whether transformations, tool invocations, execution environments, and side effects comply with the constraints represented by (O).
• Evidence grounding and auditability evaluate whether the communicated result is supported by traceable evidence and whether the process satisfies the recording and reconstruction obligations represented by (E).

This definition makes the construct operationalizable and empirically assessable by linking each dimension to requirements encoded in versioned and reviewable artifacts. It does not prescribe a universal aggregate score, weighting scheme, or acceptance threshold, because these depend on the domain, task, impact, risk, and authorized level of autonomy. Instead, it identifies the requirements that must be represented and the observable actions and outcomes that must be evaluated.

Figure 2 summarizes how the six foundations support the proposed artifacts. Data Contracts provide explicit obligations and operational constraints; Semantic Layers define governed business semantics; Data Quality contributes executable validation rules; Guardrails constrain tools and actions at runtime; AI Governance establishes accountability and oversight; and Data Provenance records execution events and supporting evidence. Together, these mechanisms provide the basis for controlled, semantically compliant, and auditable agentic data use.

Figure 2. Integration of six foundational mechanisms into an agentic data process. Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance provide complementary constraints, semantic context, validation, control, accountability, and traceability, thereby supporting controlled and auditable data use by AI agents.

5.2. Agentic Data Contracts

An Agentic Data Contract is a machine-readable specification that defines how an authorized agent may access, interpret, transform, combine, execute, and communicate data for a declared task. It extends a conventional Data Contract by binding a data product to the task, agent identity, governed semantics, permitted operations, validation requirements, evidence obligations, and governance conditions applicable to a specific execution.

The contract includes identification and versioning information, such as its identifier, schema version, owner, steward, status, and validity period. It also specifies the task class, declared purpose, expected output, impact level, authorized autonomy level, agent class, model restrictions, execution identity, operating environment, and applicable human role.

Its data-authorization section defines permitted and prohibited assets, row- and column-level restrictions, sensitivity classifications, external sources, and purpose limitations. The semantic section specifies official metrics, dimensions, analytical grain, valid join paths, aggregation rules, temporal roles, source-precedence rules, and acceptable alternatives. The operations section defines read and write permissions, allowed transformations, approved tools and execution environments, side-effect restrictions, and computational budgets.

Finally, the contract specifies the expectations to be evaluated, conditions requiring clarification or human review, provenance events to be recorded, and evidence-retention requirements. Table 4 summarizes the main sections of an Agentic Data Contract.

Table 4. Main sections of an Agentic Data Contract.

Section	Main Elements
Identification	Contract identifier, version, owner, steward, status, and validity period.
Task	Task class, declared purpose, expected output, impact level, and autonomy level.
Agent	Authorized agent class, model constraints, execution identity, and operating environment.
Data authorization	Allowed and prohibited assets, sensitivity classification, and row- and column-level restrictions.
Semantics	Official metrics, dimensions, analytical grain, valid joins, aggregation rules, and temporal interpretation.
Operations and tools	Read/write mode, permitted transformations, approved tools, and allowed side effects.
Validation	Required expectations, thresholds, sandbox requirements, and human-review triggers.
Provenance and evidence	Execution events, artifact versions, evidence links, retention rules, and approvals.

The following YAML fragment illustrates how these sections can be operationalized in a machine-readable contract for quarterly sales analysis. The example defines a delegated analytical task, identifies the authorized agent and execution environment, restricts the usable data assets, specifies the canonical metric, analytical grain, temporal role, and valid join paths, limits the permitted operations, and declares the expectations and review conditions that apply to execution.

The structure of an Agentic Data Contract for the quarterly sales-analysis scenario is presented in Box 1.

Box 1. Example of an Agentic Data Contract for the quarterly sales-analysis scenario.

• agentic_data_contract:
•   id: quarterly_sales_analysis
•   schema_version: 1.0
•   version: 2.1
•   owner: commercial_data_office

•   task:
•     class: analytical_query
•     purpose: quarterly_sales_performance
•     impact: medium
•     autonomy: delegated
•     output:
•       type: table_and_narrative
•       required_dimensions:
•         - quarter
•         - region

•   agent:
•     allowed_classes:
•       - governed_analytics_agent
•     execution_identity: analytics_agent_readonly
•     environment: production_read_replica

•   data_assets:
•     allowed:
•       - gold.fact_sales
•       - gold.dim_product
•       - gold.dim_date
•       - gold.dim_store
•     prohibited:
•       - restricted.customer_sensitive_attributes

•   prohibited_attributes:
•     - customer_income
•     - customer_gender

•   semantics:
•     fact_sales_grain: one_row_per_order_line

•     canonical_metric:
•       name: net_sales
•       expression: >
•         gross_sales - discount_amount - return_amount

•     valid_join_paths:
•       - fact_sales.product_key -> dim_product.product_key
•       - fact_sales.date_key -> dim_date.date_key
•       - fact_sales.store_key -> dim_store.store_key

•     temporal:
•       role: fiscal_quarter
•       period: previous_quarter

•   operations:
•     read_only: true
•     allowed:
•       - select
•       - aggregate
•       - inner_join
•       - left_join
•     prohibited:
•       - update
•       - delete
•       - external_export

•   expectations:
•     required:
•       - authorized_assets_only
•       - canonical_metric_required
•       - grain_compatibility_required
•       - read_only_sql_required
•       - prohibited_attributes_excluded
•       - temporal_role_required
•       - evidence_required
•       - provenance_complete

•   human_review:
•     triggers:
•       - unresolved_metric_ambiguity
•       - restricted_data_request
•       - confidence_below_threshold
•       - unapproved_external_tool

The example demonstrates that the contract extends beyond conventional schema or data-quality specifications by integrating task purpose, authorization, semantic constraints, operational permissions, validation requirements, evidence obligations, and escalation conditions within a task-specific artifact.

Specifically, the contract requires the agent to use the canonical metric net_sales, preserve the order-line grain of the sales fact table, apply the fiscal-quarter temporal role, follow approved join paths, exclude restricted customer attributes, and operate in read-only mode. It also specifies that unresolved semantic ambiguity, restricted-data requests, low-confidence conditions, or attempts to use unapproved external tools must trigger review or blocking.

The contract is not intended to replicate all governance rules within a single file. Instead, it may reference definitions maintained in catalogs, Semantic Layers, policy repositories, identity systems, or access-control services. Its role is to bind the applicable versions of these definitions and policies to a specific task, agent, and execution context.

5.3. Agentic Expectations

An Agentic Expectation is a declarative, verifiable rule that assesses whether an agent’s observable behavior complies with the contractual, semantic, security, governance, and provenance requirements associated with a task. Whereas conventional expectations typically evaluate properties of datasets or pipelines, Agentic Expectations evaluate the agent–task–data interaction.

Agentic Expectations can be organized into several categories. Asset authorization verifies that the agent accesses only permitted tables, columns, rows, files, APIs, or external sources. SQL and code safety verifies that generated artifacts comply with the authorized operation mode and exclude prohibited instructions. Metric governance verifies that the selected measure and its implementation conform to the canonical definition and version. Grain safety verifies that joins and aggregations preserve the declared analytical level of detail.

Privacy expectations assess masking, suppression, minimum-group, disclosure, and composition constraints. Tool authorization verifies the use of approved tools, functions, environments, and parameters. Evidence-grounding expectations assess whether analytical claims are supported by executed artifacts and observable results. Provenance expectations verify that required task, artifact, validation, execution, approval, and outcome events have been recorded. Finally, uncertainty and human-review expectations assess whether ambiguity, low confidence, policy exceptions, and high-impact actions are handled according to contractual requirements.

Table 5 summarizes the main categories of Agentic Expectations.

Table 5. Categories of Agentic Expectations.

Category	Example Question
Asset authorization	Did the agent use only the data assets permitted by the contract?
SQL and code safety	Is the generated artifact read-only and free of prohibited operations?
Metric governance	Does the query implement the canonical metric and its approved version?
Grain safety	Are the joined datasets and aggregations compatible with the declared analytical grain?
Privacy	Does the result satisfy masking, suppression, and minimum-group requirements?
Tool authorization	Did the agent use only approved tools, functions, and parameters?
Evidence grounding	Are the analytical claims supported by traceable execution results and validation evidence?
Provenance completeness	Were the required task, artifact, execution, validation, and approval events recorded?
Uncertainty management	Did the agent identify ambiguity and handle it according to the contract?
Human-review compliance	Was the task escalated when a human-review condition was met?

An expectation may allow an action, issue a warning, request clarification, trigger automatic repair or regeneration, block execution, or escalate the case for human review. The appropriate response depends on the violation type and severity, action reversibility, task impact, and autonomy level defined by the active contract.

Low-risk, reversible technical failures may be repaired automatically. Semantic ambiguity should trigger clarification when no canonical interpretation can be established. Authorization, privacy, and prohibited-tool violations should block execution, whereas high-impact or irreversible operations should require explicit approval even when technical validations succeed.

Some expectations can be enforced deterministically. A SQL parser can detect prohibited DELETE statements, a catalog can verify asset authorization, and a policy engine can validate tool permissions. Others require semantic, probabilistic, or expert assessment, such as determining whether a conclusion is sufficiently supported or whether two business concepts are compatible.

Probabilistic validators should not be treated as infallible secondary models. Their calibration, false approvals, false blocks, and correlated failures with the primary agent must also be monitored. Deterministic and independently enforceable checks should therefore be preferred whenever requirements can be represented explicitly.

Section 6 describes how contracts and expectations are resolved and enforced during execution, while Appendix A documents their implementation in the controlled Databricks prototype.

5.4. Agentic Data Provenance

Agentic Data Provenance is a structured, auditable record linking a task to its governing contract, agent configuration, selected data, generated artifacts, tool calls, validation outcomes, approvals, evidence, interventions, and final output.

The provenance model can reuse the entities, activities, agents, and relationships defined by W3C PROV [21,47]. However, agentic data processes require an application profile that also captures task context, version bindings, generated alternatives, validation events, repair attempts, tool invocations, evidence, governance decisions, and communicated outcomes.

Core provenance objects include the user request, declared purpose, contract version, agent and model versions, metadata context, semantic definitions, selected assets, generated SQL or code, tool invocations, execution environment, validation events, execution results, evidence artifacts, human review, and final response.

A key component is the observable decision record. Rather than exposing private model reasoning or hidden chain-of-thought, it captures the externally relevant information required for review: the governed options available to the agent, the selected option, the applicable contractual or policy basis, the validation outcome, and supporting evidence.

An example of an observable metric-selection decision record is presented in Box 2.

Box 2. Observable decision record for canonical metric selection.

{

  "decision_type": "metric_selection",

  "task_id": "task-2026-0419",

  "available_options": [

    "gross_sales",

    "net_sales"

  ],

  "selected_option": "net_sales",

  "selection_basis":

    "canonical metric in contract v2.1",

  "validator":

    "canonical_metric_required",

  "validation_result": "passed"

}

Provenance depth should be proportional to risk. A low-risk exploratory query may record the task, data assets, query, tool invocation, and result. Medium-risk processes may additionally capture contract versions, validators, repair events, and evidence links. High-risk or irreversible actions may require complete tool parameters, policy versions, artifact hashes, approvals, reviewer decisions, and retention controls.

Provenance completeness should not imply indiscriminate logging. Records should preserve the information required for reconstruction, accountability, and compliance while minimizing unnecessary sensitive content, storage overhead, and exposure of internal implementation details.

5.5. Agentic Data Governance

Agentic Data Governance is the organizational and technical system of policies, roles, responsibilities, controls, and evidence that governs how agents act on enterprise data. It operationally connects Data Governance and AI Governance by translating organizational requirements into task-specific, enforceable controls.

Its responsibilities include approving agent classes, models, tools, execution environments, and autonomy levels; maintaining contracts and expectations; assigning ownership and stewardship; managing exceptions; defining review and escalation procedures; monitoring control effectiveness; and investigating incidents.

It also governs how contracts, policies, semantic definitions, expectations, identities, and provenance records are versioned, approved, activated, superseded, and retained. This lifecycle control is essential because executions cannot be reliably reconstructed without the applicable artifact versions.

Several roles participate in this governance model. Data owners define legitimate purposes and approve data use; data stewards maintain business definitions, quality rules, valid sources, and source-precedence policies; agent developers implement runtime logic and validators; platform administrators manage identities, environments, credentials, and tools; risk or compliance roles define approval, evidence, retention, and escalation requirements; and human reviewers resolve ambiguous, exceptional, or high-impact cases.

Governance operates as a control plane across the lifecycle and should extend beyond policy documents or post hoc audits. It must integrate identity management, contract binding, runtime enforcement, tool authorization, validation, evidence collection, monitoring, human intervention, override, rollback, and revocation.

5.6. Relationship Among the Artifacts

The four artifacts are complementary and form a closed assurance loop. The Agentic Data Contract defines what the agent is authorized and expected to do; Agentic Expectations verify compliance with those requirements; Agentic Data Provenance records artifacts, actions, validations, evidence, interventions, and outcomes; and Agentic Data Governance defines ownership, approval, version control, and the handling of violations and exceptions.

The artifacts operate at different stages but remain interdependent. The contract is bound before planning and execution; expectations are evaluated before, during, and after execution; provenance is accumulated throughout the process; and governance applies across the lifecycle, defining the controls and evidence required for each task and autonomy level.

Trustworthy execution also requires explicit version binding. Each task should reference the active contract, semantic definitions, policies, expectations, agent and model configuration, tool versions, and execution environment. Without these bindings, agents may apply incompatible definitions or policies, and subsequent audits may be unable to determine which controls were active during execution.

The relationship can be summarized as follows:

• the contract defines the expected and authorized behavior;
• the expectations evaluate the observed behavior;
• provenance records the behavior and its evidence;
• governance controls the authority and lifecycle of the other three artifacts.

Together, the four artifacts operationalize Quality of Agentic Data Use. They do not guarantee correctness by themselves, but they establish the specifications, control points, evidence, and accountability mechanisms required to evaluate and improve an agent-mediated data process.

Figure 3 presents a reference information model for the main entities and relationships in the proposed framework. The Agentic Data Contract binds the task, authorized agent, data assets, permitted tools, and required expectations. Agentic Expectations evaluate observable execution behavior and produce outcomes that may permit execution, trigger repair, block an action, or require human review. Execution and validation results generate provenance events, while governance policies define ownership and control conditions across the artifacts.

Figure 3. Reference class model for the proposed framework artifacts and their implementation status in the controlled Databricks prototype.

The stereotypes distinguish components implemented in the controlled Databricks prototype from those partially implemented or specified but not enforced. Task and agent bindings, data-asset restrictions, expectation abstractions, validation results, execution outcomes, and blocking mechanisms were implemented. Execution-level provenance and governance were only partially implemented, while human-review triggers were defined but not enforced. Detailed examples of contracts, expectations, provenance records, validation rules, and prototype evidence are provided in Section 5.2, Section 5.3, Section 5.4 and Section 5.5 and Section 7, and Appendix A.

6. Lifecycle and Reference Architecture

The four artifacts introduced in the previous section form an integrated assurance process for agent-mediated data operations. This section organizes Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance into an end-to-end lifecycle and presents a reference architecture linking business requests, contracts, metadata, semantic definitions, agent execution, validation, governed tools, provenance, and organizational oversight.

The lifecycle and architecture are technology-independent and define the main control points required when an AI agent accesses, interprets, transforms, validates, or communicates enterprise data under delegated authority. Depending on task risk, these controls may combine deterministic services, semantic validators, probabilistic evaluators, and human review.

6.1. Lifecycle of an Agentic Data Process

The lifecycle begins when a user, application, or automated process submits a task and ends when the result, supporting evidence, and execution history are recorded for audit and improvement. It comprises eight principal stages.

6.2. Task Specification and Risk Classification

The process begins by defining the objective, task class, declared purpose, expected output, affected business process, anticipated side effects, and required autonomy level.

An initial risk classification is then assigned. Read-only analysis over aggregated, non-sensitive data may be classified as low or medium risk, whereas tasks that modify production data, export information, access sensitive attributes, or affect regulated decisions require stricter controls, additional evidence, or human approval.

This specification provides the context for subsequent decisions on data selection, semantic interpretation, tool use, validation, execution, and escalation.

6.3. Contract Discovery and Binding

The platform identifies the Agentic Data Contract applicable to the task, agent, requester, and execution environment. Contract binding associates the execution with authorized data assets, governed semantics, analytical grain, temporal roles, permitted tools, operational constraints, required expectations, provenance obligations, and review conditions.

The binding must preserve the exact versions of the contract and referenced artifacts, including semantic definitions, quality rules, access-control policies, validators, tool configurations, and governance policies. Version binding is required because these elements may change after execution.

If no applicable contract exists or the request cannot be mapped to a declared purpose, automatic execution should stop. The system should request clarification, apply a restricted fallback policy, or escalate the task to an authorized reviewer.

6.4. Context and Semantic Resolution

After contract binding, the platform retrieves the task-specific metadata and semantic context, including schemas, catalog descriptions, ownership, sensitivity classifications, canonical metrics, dimensions, valid join paths, analytical grain, temporal roles, source-precedence rules, quality status, and applicable policies.

This context defines the governed alternatives available to the agent but does not ensure their correct selection or implementation. Compliance is subsequently assessed through Agentic Expectations.

The retrieved context should remain task-scoped. Exposing unrestricted catalogs or complete enterprise schemas may increase ambiguity, reveal unnecessary information, and expand the risk of unauthorized or semantically inappropriate selections.

6.5. Planning and Artifact Generation

The agent constructs a task plan covering relevant datasets, metrics, dimensions, filters, transformations, tools, validation steps, and output formats.

It then generates one or more artifacts, such as SQL queries, transformation code, pipeline configurations, API requests, reports, visualizations, or narrative explanations. Each artifact should have a unique identifier linked to the task, contract version, agent and model configuration, and generation timestamp.

Planning and generation remain constrained by the active contract. A generated artifact is not inherently authorized, semantically correct, or safe to execute and must therefore satisfy the applicable pre-execution controls.

6.6. Pre-Execution Validation

Before execution, Agentic Expectations evaluate the aspects of the generated artifact that can be verified without producing the final result. Typical pre-execution checks include:

• authorization of assets, records, and attributes;
• use of canonical metrics and approved semantic definitions;
• analytical-grain and join compatibility;
• valid temporal roles and filtering conditions;
• read-only or write permissions;
• approved tools, functions, parameters, and environments;
• prohibited operations and side effects;
• privacy, masking, and disclosure restrictions;
• required human-review or approval conditions.

A validation may permit execution, issue a warning, request clarification, trigger repair, block the action, or escalate the task. The appropriate response depends on the violation type, severity, reversibility, impact, risk, and autonomy level defined by the active contract.

Deterministic controls should be preferred whenever requirements can be expressed explicitly. Semantic or probabilistic validators may be necessary for requirements that cannot be reduced to structural rules, but their uncertainty, calibration, false approvals, and false blocks must also be monitored.

6.7. Repair, Clarification, or Escalation

When validation fails, the process follows a controlled response path. A reversible technical defect, such as an invalid column reference, may trigger automatic regeneration. A semantic violation may instead produce structured feedback identifying the expected metric, grain, temporal role, source, or business rule.

If the issue cannot be resolved through an explicit rule, the system should request clarification or escalate the task. Typical cases include conflicting authoritative sources, unresolved business terminology, uncertain purpose, privacy exceptions, or high-impact operations.

Repair attempts must remain bounded. The active contract or governance policy should define the maximum number of retries, permitted repair strategies, timeouts, cost limits, and termination conditions. Failed and repaired artifacts should remain preserved in the provenance record.

6.8. Controlled Execution

After the pre-execution expectations are satisfied, the artifact is submitted to an approved tool within a controlled environment. The execution identity should be separated from the user’s personal credentials and granted only the permissions required for the task.

The execution layer enforces access controls, environment restrictions, budgets, timeouts, network policies, and side-effect constraints. It should record the tool, parameters, environment, execution identifier, timestamps, status, affected assets, and generated results.

Successful execution only confirms that the artifact was processed by the tool. It does not establish semantic correctness, authorization, completeness, or sufficient evidential support, which require post-execution validation.

6.9. Post-Execution Validation and Evidence Reconciliation

Post-execution validation evaluates the result against the active contract, executed artifact, and available evidence. Checks may include output-schema validation, cardinality, aggregation consistency, privacy constraints, numerical consistency, evidence coverage, and correspondence between the generated narrative and execution results.

This stage is critical when the agent communicates analytical conclusions. A technically valid query may still support an incorrect or unsupported explanation. Important claims, numerical values, and recommendations should therefore be linked to identifiable execution results or validation evidence.

Post-execution validation also completes expectations that cannot be assessed beforehand, including evidence availability, provenance completeness, reproducibility, and communication grounding.

6.10. Provenance Capture, Delivery, and Improvement

Agentic Data Provenance is accumulated throughout the lifecycle rather than generated only at its conclusion. Before delivery, the system verifies that the required task, contract, artifact, validation, execution, evidence, repair, approval, and outcome records are available.

The governed response should include the requested business result and, when appropriate, a concise control summary identifying the canonical metric, contract version, execution identifier, validation status, evidence location, provenance status, and any repair or human review.

After delivery, the lifecycle continues through monitoring, audit, and improvement. Organizations should analyze execution failures, repair rates, validator errors, false blocks, policy violations, reviewer interventions, latency, cost, provenance gaps, and user feedback.

These observations can refine contracts, semantic definitions, expectations, policies, tools, autonomy levels, and benchmark cases. Recurrent semantic failures may indicate incomplete contracts, ambiguous Semantic Layers, or insufficient validators, whereas high false-block rates may reveal overly restrictive or poorly calibrated controls.

The lifecycle therefore forms a closed control loop: task requirements are translated into a contract; the contract constrains planning and generation; expectations evaluate observable behavior; approved tools execute compliant artifacts; post-execution checks reconcile results and evidence; provenance records the execution history; and governance supports audit, intervention, and continuous improvement.

Figure 4 summarizes the lifecycle. Although the stages need not correspond to separate software components, they represent distinct control points that should remain observable, versioned, and auditable throughout an agent-mediated data process.

Figure 4. Lifecycle for specifying, generating, validating, executing, and auditing an agentic data process. The lifecycle is supported by Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance.

6.11. Reference Architecture

The reference architecture organizes the lifecycle into interacting functional layers. It is modular and can be implemented using different data platforms, catalogs, semantic technologies, policy engines, agent frameworks, validation services, and provenance systems.

6.12. Interaction and Task Interpretation

The interaction layer receives requests from users, applications, APIs, or scheduled processes. It authenticates the requester and captures the business objective, expected output, declared purpose, urgency, and relevant contextual information.

A task-interpretation service transforms the request into a structured representation that separates the business objective from implementation instructions and identifies unresolved ambiguities before the task is passed to the agent runtime.

6.13. Contract and Policy Control Plane

The control plane contains Agentic Data Contracts, governance policies, identity rules, tool permissions, review conditions, and autonomy constraints. It determines the applicable contract and binds the corresponding artifact versions to the execution.

This layer is authoritative. Although a prompt may include a contract summary, the prompt should not constitute the enforcement boundary. Access control, tool authorization, blocking, approval, and evidence requirements should be enforced by services external to the generative model.

6.14. Enterprise Metadata and Semantic Services

This layer provides schemas, catalogs, ownership information, lineage, classifications, quality status, Semantic Layer definitions, ontologies, business glossaries, valid join paths, analytical grain, temporal roles, and source-precedence rules.

The same governed definitions should be available to both the agent runtime and the validation mechanisms. The agent uses them during planning and artifact generation, while Agentic Expectations evaluate whether the resulting behavior complies with the active contract.

6.15. Agent Runtime

The agent runtime performs task decomposition, planning, context selection, artifact generation, tool selection, execution observation, and controlled repair. It may comprise a single model, multiple specialized agents, memory components, planners, and tool adapters.

The runtime does not operate with unrestricted authority. Its accessible data, tools, parameters, execution environments, retry limits, budgets, and permitted side effects are constrained by the active contract and governance policy. The agent proposes and coordinates actions, while independent services enforce the applicable boundaries.

6.16. Agentic Expectations and Data-Aware Guardrails

This layer evaluates generated artifacts, tool calls, execution results, and communicated outputs through deterministic validators, semantic checks, policy evaluation, privacy controls, evidence reconciliation, and human-review triggers.

Data-aware guardrails derive constraints from contracts, Semantic Layers, catalogs, quality systems, authorization services, and governance policies. Their function extends beyond blocking generally unsafe actions to detecting domain-specific violations, including incorrect metrics, incompatible grains, invalid temporal roles, prohibited data combinations, inappropriate sources, and unsupported claims.

Validation outcomes should be represented in a structured form that can be consumed by the agent runtime, governance services, and provenance store. A validation result may include the violated rule, observed and expected values, severity, recommended response, and supporting evidence.

6.17. Governed Tools and Controlled Execution

The governed tool layer exposes the capabilities available to the agent, including SQL engines, transformation platforms, orchestration services, catalogs, visualization tools, machine-learning platforms, APIs, and controlled external services.

Each tool adapter should enforce identity, permissions, parameters, environment restrictions, network policies, budgets, and side-effect constraints. It should also generate standardized execution records for validation and provenance capture.

Separating tool access from the agent runtime reduces reliance on prompt instructions and makes execution controls observable, enforceable, and revocable.

6.18. Evidence, Provenance, and Audit Services

This layer records the task, purpose, contract version, agent and model configuration, semantic context, generated artifacts, validation outcomes, repairs, tool calls, execution results, evidence, approvals, interventions, and final response.

The provenance store should support reconstruction, querying, impact analysis, incident investigation, and compliance review. It should also enforce retention, access, minimization, and integrity requirements because provenance records may contain sensitive operational information.

Audit services can use these records to measure trace completeness, evidence coverage, reproducibility, audit reconstruction time, repair rates, and reviewer workload.

6.19. Governance and Human Oversight

Agentic Data Governance operates across all architectural components. It assigns ownership, approves contracts, tools, models, autonomy levels, and validators, and defines escalation, override, revocation, retention, and incident-management procedures.

Human oversight should be event-driven rather than treated as a generic final step. The architecture should specify which conditions require review, which role receives the task, what evidence is provided, how the decision is recorded, and whether execution resumes or terminates.

Governance must also assess the architecture’s effectiveness over time by reviewing policy changes, model updates, validator performance, new data assets, tool modifications, observed failures, and changes in organizational risk.

6.20. Deployment Patterns and Incremental Adoption

The reference architecture supports incremental adoption rather than requiring full implementation from the outset. An initial deployment may consist of a read-only analytical agent operating on a limited set of approved datasets, using a simplified Agentic Data Contract, deterministic SQL safety checks, asset authorization, and basic execution provenance.

A second stage can introduce canonical metrics, analytical-grain rules, temporal roles, evidence grounding, and structured repair, requiring closer integration with catalogs and Semantic Layers.

A third stage can support pipeline generation or controlled write operations in sandboxed environments, with stricter tool permissions, isolation, approval gates, rollback mechanisms, and richer provenance.

Supervised-autonomous deployments may operate continuously within defined policies but require runtime monitoring, revocable credentials, behavioral objectives, incident procedures, human escalation, and periodic reauthorization.

These patterns illustrate that Agentic Data Engineering can be adopted progressively, with validation, evidence, and governance increasing according to task authority, impact, irreversibility, and risk.

7. Illustrative Scenario: Governed Quarterly Sales Analysis

This section illustrates the proposed Agentic Data Engineering framework as an integrated process. The scenario is explanatory and is not intended as an empirical evaluation or benchmark. Its purpose is to demonstrate how Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance interact during the execution of a realistic enterprise-data task.

The scenario also shows how the four proposed artifacts reuse and extend the six foundations discussed previously. Data Contracts provide structural agreements; Semantic Layers define governed business semantics; Data Quality contributes executable validation mechanisms; Guardrails constrain runtime actions; AI Governance establishes organizational accountability; and Data Provenance supports traceability and process reconstruction. The proposed framework coordinates these capabilities around a task executed by an agent under delegated authority.

Figure 5 provides a high-level view of the foundations, orchestration mechanisms, and governed interaction instantiated in the illustrative scenario.

Figure 5. Layered view of the governed agentic data process. The proposed framework connects existing data and governance foundations with task-specific specification, validation, execution, provenance, and oversight mechanisms.

7.1. Business Request and Analytical Context

A business analyst submits the following request:

“What were the net sales of premium products during the previous quarter by sales region?”

Although the request appears simple, it requires several engineering, semantic, and governance decisions. The system must identify the official sales metric, select authorized datasets, preserve the valid analytical grain, apply the correct temporal role, use a permitted execution tool, exclude prohibited attributes, generate a valid query, preserve supporting evidence, and return a result that can be reconstructed and audited.

For this scenario, the enterprise data platform contains the following governed assets:

• gold.fact_sales, with one row per order line;
• gold.dim_product, containing product categories;
• gold.dim_store, containing sales regions;
• gold.dim_date, containing calendar and fiscal periods.

The organization defines net_sales, rather than gross_sales, as the canonical metric for quarterly commercial reporting. It also defines fiscal_quarter as the applicable temporal role for this task.

7.2. Contract Binding

Before the agent generates an artifact, the platform binds the request to the applicable Agentic Data Contract. The contract defines the purpose, authorized assets, official metric, analytical grain, temporal role, permitted tool, prohibited attributes, required expectations, and provenance obligations.

The main elements of the contract bound to the illustrative scenario are summarized in Box 3.

Box 3. Active Agentic Data Contract bound to the illustrative sales-analysis scenario.

• task:
    purpose: quarterly_sales_analysis
    autonomy: delegated

• authorized_assets:
    - gold.fact_sales
    - gold.dim_product
    - gold.dim_store
    - gold.dim_date

• semantics:
    canonical_metric:
      name: net_sales

•   grain:
      fact_sales: one_row_per_order_line

•   temporal:
      role: fiscal_quarter
      period: previous_quarter

• allowed_tools:
    - sql_executor

• prohibited_attributes:
    - customer_income
    - customer_gender

• required_expectations:
    - authorized_assets_only
    - canonical_metric_required
    - grain_compatibility_required
    - read_only_sql_required
    - prohibited_attributes_excluded
    - temporal_role_required
    - evidence_required
    - provenance_complete

At this stage, no SQL has been executed. The contract establishes the semantic, operational, authorization, validation, and evidence boundaries within which the agent may act. The eight required expectations correspond to the requirements that must be satisfied before the process is considered complete. Some can be evaluated before execution, while others require the execution result and provenance record.

7.3. Agent Planning and Initial SQL Generation

The agent interprets the request and constructs an initial plan. The plan identifies the required metric, product classification, temporal filter, regional dimension, and aggregation operation:

• identify the canonical sales metric;
• select the governed data assets required by the task;
• filter products classified as premium;
• restrict the analysis to the previous fiscal quarter;
• join sales with product, store, and date dimensions;
• aggregate the result by sales region;
• return a table and a narrative explanation.

The agent then produces the initial SQL statement shown in Box 4.

Box 4. Initial SQL statement generated by the agent.

SELECT

    s.sales_region,

    SUM(f.gross_sales) AS quarterly_sales

FROM gold.fact_sales AS f

JOIN gold.dim_product AS p

    ON f.product_key = p.product_key

JOIN gold.dim_store AS s

    ON f.store_key = s.store_key

JOIN gold.dim_date AS d

    ON f.date_key = d.date_key

WHERE p.product_category = ‘Premium’

  AND d.fiscal_quarter = :previous_quarter

GROUP BY s.sales_region;

The query is syntactically valid and uses existing tables and join keys. However, it violates the semantic requirements of the active contract because it uses gross_sales instead of the canonical net_sales metric. This error illustrates why successful parsing or execution is insufficient to establish trustworthy data use.

7.4. Expectation Failure and Repair

Before execution, the generated query is evaluated by the Agentic Expectations engine. The first relevant expectation, canonical_metric_required, compares the metric implemented in the generated artifact with the canonical metric bound by the active contract.

The validation produces the structured result shown in Box 5.

Box 5. Validation result identifying a canonical metric failure.

expectation: canonical_metric_required

artifact: sql-v1

status: FAIL

observed_metric: gross_sales

required_metric: net_sales

action: block_and_regenerate

Because the violation concerns enterprise semantics, the query is blocked before execution. The agent receives structured feedback identifying the violated expectation, the observed metric, and the required canonical definition.

Following the validation feedback, the agent produces the corrected SQL artifact shown in Box 6.

Box 6. Corrected SQL statement generated after validation.

SELECT

    s.sales_region,

    SUM(f.net_sales) AS quarterly_net_sales

FROM gold.fact_sales AS f

JOIN gold.dim_product AS p

    ON f.product_key = p.product_key

JOIN gold.dim_store AS s

    ON f.store_key = s.store_key

JOIN gold.dim_date AS d

    ON f.date_key = d.date_key

WHERE p.product_category = ‘Premium’

  AND d.fiscal_quarter = :previous_quarter

GROUP BY s.sales_region;

The regenerated query is evaluated against the complete set of eight contractual expectations. Metric correctness, asset authorization, grain compatibility, read-only execution, prohibited-attribute exclusion, and temporal-role compliance can be evaluated before execution. Evidence availability and provenance completeness are finalized after the result and execution records have been produced.

Table 6 reports the final status of the eight expectations declared in the active contract. It therefore combines pre-execution and post-execution validation outcomes.

Table 6. Final validation trace for the illustrative scenario.

Expectation	Result	Evidence
canonical_metric_required	Pass	The corrected query uses the contract-defined net_sales metric.
authorized_assets_only	Pass	All referenced tables are authorized by the active contract.
grain_compatibility_required	Pass	The joins preserve the order-line grain before regional aggregation.
read_only_sql_required	Pass	The artifact contains only a SELECT statement.
prohibited_attributes_excluded	Pass	No prohibited customer attribute is accessed or returned.
temporal_role_required	Pass	The query uses the fiscal-quarter temporal role and previous-quarter period specified in the contract.
evidence_required	Pass	The final response is linked to the executed query and result set.
provenance_complete	Pass	The task, contract, artifacts, validations, execution, and output are recorded.

Figure 6 summarizes the complete flow from the business request to the governed response, including the initial semantic failure, repair loop, controlled execution, provenance capture, and governance controls.

Figure 6. Illustrative execution of a governed agentic data process. The initial query fails the canonical-metric expectation, is repaired before execution, and subsequently produces a validated, traceable, and governed business response.

7.5. Controlled Execution and Result Verification

After the six pre-execution expectations pass, the corrected query is submitted to the approved sql_executor tool. The tool executes the query against a read-only production replica. The execution environment records the query identifier, execution timestamp, contract version, agent identity, tool parameters, environment, and result artifact.

A simplified result is presented in Table 7.

Table 7. Simplified result produced by the corrected SQL statement.

Sales Region	Quarterly Net Sales
North	2,300,000
South	1,800,000
Central	1,250,000

Post-execution validation verifies the output schema, result cardinality, absence of prohibited attributes, aggregation consistency, evidence availability, and provenance completeness. The last two contractual expectations are considered satisfied only after the executed artifact, result set, and associated records are available.

7.6. Provenance Trace for the Illustrative Scenario

Agentic Data Provenance records the complete observable history of the process. The trace links the initial task with the active contract, both SQL versions, the failed expectation, the repaired artifact, the executed tool call, the result set, and the final response.

A simplified Agentic Data Provenance record for the illustrative scenario is presented in Box 7.

Box 7. Simplified Agentic Data Provenance record for the illustrative scenario.

• task_id: task-2026-0419
  request: quarterly premium-product sales by region

• contract:
    id: quarterly_sales_analysis
    version: 2.1

• agent:
    id: governed_analytics_agent
    model_version: model-v4.3

• generated_artifacts:
    - artifact_id: sql-v1
      metric: gross_sales
      status: rejected

•   - artifact_id: sql-v2
      metric: net_sales
      status: executed

• validation_events:
    - expectation: canonical_metric_required
      artifact: sql-v1
      result: FAIL

•   - expectation: canonical_metric_required
      artifact: sql-v2
      result: PASS

•   - expectation: authorized_assets_only
      artifact: sql-v2
      result: PASS

•   - expectation: grain_compatibility_required
      artifact: sql-v2
      result: PASS

•   - expectation: read_only_sql_required
      artifact: sql-v2
      result: PASS

•   - expectation: prohibited_attributes_excluded
      artifact: sql-v2
      result: PASS

•   - expectation: temporal_role_required
      artifact: sql-v2
      result: PASS

•   - expectation: evidence_required
      artifact: sql-v2
      result: PASS

•   - expectation: provenance_complete
      artifact: sql-v2
      result: PASS

• tool_calls:
    - tool: sql_executor
      execution_id: SQL-8AF3
      environment: production_read_replica

• output_artifact:
    result_id: RESULT-8AF3

• final_validation_summary:
    required_expectations: 8
    passed_expectations: 8
    failed_expectations: 0

• provenance_status: complete

The provenance record contains nine validation events associated with eight required expectations. The difference occurs because canonical_metric_required is evaluated twice: it first fails for sql-v1 and subsequently passes after the agent produces sql-v2. The other seven expectations produce one passing event each.

The final validation summary therefore reports eight satisfied expectations, while the provenance record preserves the complete failure-and-repair history. A final pass status alone would not reveal that the first generated artifact violated the active contract.

This trace differs from conventional lineage because it records not only which data and transformations produced the result, but also the declared task, governing contract, generated alternatives, validation outcomes, repair event, execution environment, tool invocation, and supporting evidence. These elements allow the process to be reconstructed without requiring access to the model’s private reasoning.

7.7. Agentic Data Governance

Agentic Data Governance operates across the complete scenario. It determines which agent class is authorized, which contract and policy versions apply, which tools and environments can be used, which expectations are mandatory, and which conditions require clarification, blocking, or human review.

For this medium-risk, read-only analytical task, governance permits automatic execution after the required deterministic and semantic pre-execution expectations pass. A human reviewer is not required because the corrected request contains no unresolved ambiguity, restricted-data access, privacy exception, unapproved external tool, or irreversible side effect.

The governance record includes:

• responsible agent and execution identity;
• active contract, semantic, and policy versions;
• assigned risk and autonomy levels;
• required and completed validations;
• repair, approval, and escalation status;
• provenance-retention requirement;
• accountable data owner and data steward.

If the agent had requested a prohibited customer attribute, selected an unapproved external tool, encountered unresolved metric ambiguity, or attempted an operation outside the delegated read-only authority, the governance policy would have blocked execution or transferred the case to a designated reviewer.

7.8. Governed Business Response

After execution and post-execution validation, the analyst receives the requested result together with the evidence and control summary presented in Box 8.

Box 8. Final governed business response and validation summary.

• Premium-product net sales during the previous quarter:

• North:  $2.30 M
  South:  $1.80 M
  Central: $1.25 M

• Metric:
    net_sales

• Contract:
    quarterly_sales_analysis v2.1

• Execution:
    SQL-8AF3

• Required expectations:
    8

• Final validation:
    8 passed

• Validation events:
    9, including one repaired failure

• Provenance:
    complete

The response separates the analytical result from its supporting metadata. This allows the user to identify the metric used, the governing contract, the executed artifact, the validation status, and the existence of a prior repair. The business result is therefore accompanied by sufficient evidence to support review and reconstruction.

7.9. Controlled Prototype Validation

A controlled prototype validation was implemented in Databricks to examine whether the proposed artifacts could operationally detect, repair, block, and record different classes of agentic data failure. The prototype used a Sales Analytics domain containing two fact tables at different grains and four dimensions. Three deliberately selected questions represented metric mismatch, grain incompatibility, and unauthorized use of a sensitive attribute.

Each question was evaluated under three configurations: Prompt Only, with neither contract context nor independent validation; Contract Only, with task and semantic context but no active enforcement; and Full Framework, with an Agentic Data Contract, pre-execution Agentic Expectations, repair, blocking, and provenance capture. This design produced nine controlled executions. The prototype evaluates the operational behavior of the control mechanisms rather than the general SQL-generation accuracy of a particular LLM.

The Prompt Only configuration executed all three syntactically valid but semantically or governmentally incorrect artifacts. Contract context was sufficient to guide canonical metric selection in the first case, but it did not prevent the grain or authorization failures. Under the Full Framework, the metric error was repaired and executed, the incompatible-grain query was blocked after unsuccessful repair, and the sensitive-attribute query was blocked immediately.

All nine execution traces were stored in the prototype provenance table, including the initial SQL, validation results, actions, repaired SQL when applicable, revalidation results, and final outcome. These results provide a small operational proof of concept. Because the questions and failure artifacts were deliberately selected and no repeated stochastic trials were conducted, the results do not establish general effectiveness or support statistical inference. Appendix A provides the implementation and execution details.

8. Taxonomy of Agentic Data Failures

A taxonomy is useful for organizing possible failures, identifying detection mechanisms, and selecting mitigation controls. The proposed taxonomy covers syntactic, structural, semantic, authorization, privacy, tool, evidence, provenance, ambiguity, and escalation failures.

The Databricks prototype scenarios reported in Table 8 provide concrete examples of three categories in this taxonomy: metric mismatch, grain incompatibility, and unauthorized use of a sensitive attribute. The remaining categories define additional failure modes for subsequent empirical evaluation.

Table 8. Comparative outcomes of the Databricks prototype validation.

Failure Scenario	Prompt Only	Contract Only	Full Framework
Metric mismatch	Executed using the non-canonical gross_sales metric.	Used the canonical net_sales metric, but without independent validation.	Detected the non-canonical metric, repaired the artifact, revalidated it, and executed.
Grain incompatibility	Executed a direct fact-to-fact join that could produce fan-out.	Retained the incompatible join despite receiving contract context.	Detected the incompatible grain, attempted repair, and blocked execution when the revised artifact remained invalid.
Sensitive attribute	Executed a query using the contract-prohibited gender attribute.	Retained the prohibited attribute because no active enforcement was present.	Detected the authorization violation and blocked the query before execution.

8.1. Syntactic and Structural Failures

A syntactic failure occurs when the generated SQL, code, or configuration is invalid. These failures are generally easy to detect through parsers, compilers, or execution errors. A structural failure occurs when the artifact uses a column, key, relationship, or schema element incorrectly. Schema validation and catalog checks can identify many of these problems.

8.2. Metric and Aggregation Failures

A metric failure occurs when the agent selects or calculates a measure that does not correspond to the official business definition. An aggregation failure appears when a measure is aggregated incorrectly, such as summing a non-additive measure or omitting a required filter. These failures can execute successfully and therefore require semantic validation. The metric-mismatch scenario in Table 8 illustrates this failure through the use of gross_sales instead of the contract-defined net_sales metric.

8.3. Grain and Join Failures

A grain failure occurs when the agent combines data at incompatible levels of detail. For example, joining an order-line fact with a daily product aggregate can duplicate values. A join can be technically valid because the keys exist, but analytically incorrect because the multiplicity was not considered.

8.4. Temporal Interpretation Failures

Enterprise data can include transaction date, registration date, delivery date, fiscal period, and event time. The agent can select the wrong temporal role or ignore time-zone and fiscal-calendar rules. The contract should define the valid interpretation for the task.

8.5. Authorization and Purpose Failures

An authorization failure occurs when the agent accesses an asset, attribute, record, or tool that is not permitted. A purpose failure is more subtle. The agent may have technical access to the data but use them for a purpose that was not approved. Task-scoped authorization is required to distinguish these cases.

8.6. Privacy Composition Failures

Two datasets or attributes can be individually permitted but create a privacy risk when combined. An agent can also produce groups that are too small or include sensitive quasi-identifiers. These failures require rules about composition, aggregation, suppression, and output disclosure.

8.7. Tool and Side-Effect Failures

An agent can invoke a tool with inappropriate parameters, use a production environment instead of a sandbox, export data to an external service, or perform an irreversible operation. Tool governance must consider both the capability and the side effect.

8.8. Agentic Security Threats

Agentic data systems also introduce security threats that extend beyond conventional data-access violations. These threats arise because retrieved content can influence agent behavior, agents can invoke external tools using delegated credentials, and technically authorized actions can produce unauthorized side effects. Table 9 relates six representative threats to the corresponding framework controls and evaluation mechanisms.

Table 9. Agentic security threats, mitigations, and evaluation mechanisms.

Threat	Agentic Failure	Principal Mitigations	Evaluation Mechanism
Prompt injection	A user attempts to override task, authorization, or tool restrictions through adversarial instructions.	Separate system instructions from user content; treat the active contract and policies as authoritative; apply input validation and independently enforced expectations.	Execute adversarial prompt suites and measure attack success, false approvals, false blocks, and policy-compliant outcomes.
Indirect prompt injection	Retrieved documents or data contain instructions that the agent interprets as commands.	Treat retrieved content as untrusted data; validate source and authorization; sanitize context; separate data from instructions; restrict available tools and validate outputs.	Insert controlled malicious instructions into retrieved documents and verify whether they influence tool calls, data access, or the final response.
Tool abuse	The agent invokes an approved tool using prohibited functions, parameters, targets, or execution frequency.	Use tool and function allowlists, parameter constraints, execution budgets, rate limits, sandboxing, and approval gates.	Test malformed, excessive, prohibited, and out-of-scope tool calls and measure correct blocking and over-blocking.
Privilege escalation	The agent attempts to use broader permissions than those delegated for the task or inherits an overprivileged workload identity.	Bind user, agent, purpose, and execution identity; enforce least privilege, task-scoped capabilities, short-lived credentials, and independent access control.	Attempt cross-role, cross-domain, and prohibited-asset access and measure unauthorized executions and correctly blocked requests.
Compromised external tool	An approved external service returns manipulated results, performs unexpected actions, or is replaced by an untrusted version.	Use approved endpoints and versions, integrity verification, output validation, restricted credentials, isolation, monitoring, revocation, and circuit breakers.	Simulate malformed responses, integrity failures, unavailable services, and unexpected side effects; verify rejection, isolation, and escalation.
Unauthorized side effect	A technically valid request produces an unapproved write, deletion, export, notification, transaction, or irreversible external action.	Declare permitted side effects in the contract; use read-only defaults, transaction boundaries, dry runs, human approval, rollback, and execution blocking.	Submit destructive, irreversible, and external-export actions and verify that approval, blocking, rollback, and provenance requirements are enforced.

Within the proposed framework, Agentic Data Contracts define the permitted identities, assets, tools, parameters, environments, and side effects for a task. Agentic Expectations independently evaluate generated artifacts and attempted actions before and after execution. Agentic Data Governance approves tools, autonomy levels, exceptions, and revocation conditions, while Agentic Data Provenance records the retrieved context, policy versions, tool calls, validation decisions, approvals, and outcomes. The effectiveness of these controls should be evaluated through adversarial test suites containing both prohibited and legitimate requests, allowing false approvals, false blocks, attack success, repair, escalation, and audit reconstruction to be measured. The present prototype evaluates selected authorization, metric, and grain violations; comprehensive adversarial evaluation of all six security threats remains future work.

8.9. Evidence and Quantitative Hallucination Failures

An unsupported claim occurs when the output includes a conclusion that is not linked to the execution result. A quantitative hallucination occurs when the agent reports a number that was not produced by the query or calculation. Evidence expectations should connect each important claim with a traceable artifact.

8.10. Source Conflict and Ambiguity Failures

A source conflict failure occurs when several valid sources contain different values and the agent has no policy for selecting or reconciling them. An ambiguity failure occurs when the request has several interpretations and the agent chooses one without clarification or documented policy.

8.11. Provenance and Escalation Failures

A provenance failure occurs when the process cannot be reconstructed because the contract, artifact, tool call, validation, or approval was not recorded. An escalation failure occurs when the agent continues a task that should have been transferred to a person.

Table 10 summarizes the proposed taxonomy, illustrative examples, principal detection mechanisms, and possible mitigations.

Table 10. Main categories of agentic data failures.

Category	Example	Main Detection Mechanism	Possible Mitigation
Syntactic	Invalid SQL statement.	Parser validation or execution feedback.	Automatic repair followed by revalidation.
Structural	Nonexistent column or incorrect schema element.	Schema, metadata, and catalog validation.	Schema grounding and structural constraints.
Metric and aggregation	Gross sales used instead of net sales, or a non-additive measure summed across time.	Canonical-metric, formula, filter, and aggregation validation.	Versioned metric definitions and aggregation rules in the contract and Semantic Layer.
Grain and join	Facts duplicated after joining datasets at incompatible grains.	Grain, cardinality, join-path, and multiplicity analysis.	Approved join paths, declared grains, and required pre-aggregation.
Temporal interpretation	Registration date interpreted as event date or calendar quarter used instead of fiscal quarter.	Semantic and temporal-role validation.	Explicit temporal roles, calendars, time zones, and valid periods.
Authorization	Access to a contract-prohibited table, record, or sensitive attribute.	Access-control and contract-compliance validation.	Task-scoped permissions, asset restrictions, and least-privilege execution identities.
Purpose	Technically permitted data used for an unapproved objective.	Purpose-policy and intended-use evaluation.	Purpose binding, approval conditions, and human review.
Privacy composition	Individually permitted attributes combined to infer sensitive information or produce excessively small groups.	Privacy, disclosure-risk, composition, and minimum-group validation.	Masking, suppression, aggregation, minimization, and minimum-group rules.
Tool and side effect	External export, destructive operation, or production tool invoked without approval.	Tool-call interception, parameter validation, and policy enforcement.	Tool and operation allowlists, sandboxing, approval gates, and rollback.
Evidence and quantitative hallucination	A conclusion or numerical value reported without support in the executed result.	Evidence-coverage, result-reconciliation, and grounding validation.	Mandatory links between claims, executed artifacts, results, and validation evidence.
Source conflict	The agent selects one of several conflicting sources without applying a precedence or reconciliation policy.	Source comparison, precedence-rule, and conflict detection.	Declared source precedence, reconciliation rules, clarification, or human review.
Ambiguity	Arbitrary interpretation of a concept such as active customer.	Ambiguity, semantic-equivalence, and confidence assessment.	Clarification request, documented interpretation, or escalation.
Provenance	Missing contract, artifact, validation, tool-call, or policy version.	Provenance-completeness and version-binding validation.	Mandatory event logging, artifact retention, and version binding.
Escalation	A high-risk or unresolved action proceeds without required review.	Review-trigger, risk-threshold, and approval validation.	Human-approval workflow, execution blocking, and recorded intervention.

The failures can also be classified according to impact, detectability, reversibility, scope, latency, and control point. These dimensions allow organizations to select controls proportionally to risk. For example, a reversible read-only query may tolerate a controlled repair loop, whereas an irreversible production action should require stricter pre-execution validation and explicit approval. The Databricks benchmark operationalizes selected metric, grain, authorization, blocking, and repair failures, while the remaining categories define cases for broader future evaluation.

9. Evaluation Framework

The evaluation of Agentic Data Engineering should not be limited to execution accuracy. It must determine whether the agent completed the task, used the correct semantics, respected authorization, produced sufficient evidence, and left a complete audit record.

9.1. Functional Metrics

Functional evaluation includes syntactic validity, code compilation, task completion, tool-call success, result accuracy, latency, computational cost, and number of repair attempts. These metrics are already common in Text-to-SQL and ELT benchmarks [13,14,22,23,24,25]. They remain necessary because an agentic process cannot be useful when its artifact does not execute.

9.2. Semantic Metrics

Semantic evaluation includes canonical metric selection, equivalence between the generated expression and the official formula, grain compliance, join correctness, aggregation correctness, temporal interpretation, source selection, and conflict-resolution compliance. These metrics require benchmark cases that contain more than a reference SQL query. The benchmark must include the task purpose, metric version, analytical grain, valid joins, source precedence, and acceptable alternatives.

9.3. Governance and Safety Metrics

Governance metrics include unauthorized asset-use rate, purpose-policy violations, prohibited operations, unauthorized tool calls, privacy compliance, human-review compliance, successful blocking rate, false approval rate, false blocking rate, and policy-evaluation overhead. These measures indicate whether the system is both safe and usable.

9.4. Evidence and Provenance Metrics

The evaluation should measure provenance completeness, artifact-version traceability, evidence coverage, reproducibility, validation traceability, approval traceability, and audit-reconstruction time. A complete trace is valuable only if it can be interpreted efficiently by reviewers.

9.5. Operational Metrics

Operational metrics include repair success, number of repair attempts, escalation precision and recall, reviewer workload, mean time to resolution, latency overhead, cost of evidence capture, and maintenance effort for contracts and expectations. These metrics are necessary to determine whether the control system is practical in production.

Table 11 summarizes the proposed evaluation dimensions for agentic data processes.

Table 11. Evaluation dimensions for agentic data processes.

Dimension	Example Metrics
Functional	Execution success rate, task-completion rate, latency, and computational cost.
Semantic	Metric-selection accuracy, grain compliance, join correctness, aggregation correctness, and temporal-role accuracy.
Governance and safety	Unauthorized-access rate, purpose-policy violations, tool-compliance rate, blocking recall, false-approval rate, false-block rate, and escalation compliance.
Evidence and provenance	Evidence coverage, reproducibility rate, trace completeness, artifact-version completeness, and audit-reconstruction time.
Operational	Repair coverage, governed repair success, escalation quality, reviewer workload, control overhead, and recovery time.

Evaluation can be automatic, manual, or hybrid, depending on whether the applicable requirement can be represented explicitly. Schema, metric, grain, authorization, tool, and provenance requirements can be evaluated automatically against versioned contracts, Semantic Layers, policies, and execution records. Ambiguous business concepts, alternative valid interpretations, evidence sufficiency, and high-risk exceptions may require expert assessment. Pre-execution evaluation applies to generated artifacts, authorization, semantics, tools, and anticipated side effects. Runtime evaluation monitors tool calls, policy state, resource use, and intermediate actions. Post-execution evaluation examines results, evidence, provenance completeness, repair outcomes, and auditability.

Section 9.6 defines the acceptance and aggregation procedure, while Appendix C demonstrates its independent application to a controlled benchmark.

9.6. Operational Indicators and Acceptance Procedure

Quality of Agentic Data Use can be assessed through execution-level indicators associated with the applicable contractual expectations. Relevant indicators include governed-decision accuracy, critical-violation blocking recall, safe-query specificity, false approvals, false blocks, repair coverage, governed repair success, evidence coverage, and provenance completeness. These indicators should be evaluated by validators that are operationally separated from the agent generating the artifact and against a predefined gold standard or versioned contractual rules.

The framework does not prescribe a universal weighted score because compensating across dimensions could produce unsafe decisions. For example, high evidence or provenance coverage cannot compensate for a critical authorization or privacy violation. Therefore, an execution e is accepted only if every mandatory applicable expectation returns $\mathrm{Pass}$ and no unresolved critical violation remains after validation or repair:

$$\mathrm{Accept}\left(e\right)=1\iff \left(\underset{i\in M\left(e\right)}{\bigwedge }{E}_i\left(e\right)=\mathrm{Pass}\right)\wedge \left(C\left(e\right)=⌀\right).$$

Here, $\mathrm{Accept}\left(e\right)\in \{0,1\}$ indicates whether execution e is accepted; $M\left(e\right)$ is the set of mandatory expectations applicable to e; $E_i\left(e\right)$ is the validation result of expectation i for e; $\mathrm{Pass}$ denotes a successful validation result; and $C\left(e\right)$ is the set of unresolved critical violations. Accordingly, $\mathrm{Accept}\left(e\right)=1$ only when all mandatory expectations pass and $C\left(e\right)=⌀$.

A descriptive compliance rate can also be calculated as follows:

$$\mathrm{ComplianceRate}\left(e\right)={\displaystyle \frac{{\displaystyle \sum _{i\in A\left(e\right)}\mathbf{1}\left[E_i\left(e\right)=\mathrm{Pass}\right]}}{\left|A\right(e\left)\right|}},$$

where $A\left(e\right)$ is the set of all expectations applicable to execution e, and $\mathbf{1}[·]$ is the indicator function, which equals 1 when the enclosed condition is true and 0 otherwise. The compliance rate provides a descriptive summary of validation performance but does not override the acceptance rule. A single unresolved critical violation results in rejection, regardless of the aggregate compliance rate. Appendix C applies these indicators to a controlled benchmark comprising 10 cases and 40 executions.

9.7. Experimental Design

A comprehensive evaluation can compare five conditions. Condition A uses a free-form prompt and unrestricted schema context. Condition B adds a conventional Data Contract. Condition C uses an Agentic Data Contract with task and semantic restrictions. Condition D adds independent Agentic Expectations. Condition E applies the complete framework with governance binding and Agentic Data Provenance. The controlled benchmark reported in Appendix C implements a preliminary four-condition subset of this broader design: Prompt Only, Contract Only, Validation Without Repair, and Full Framework. In the Contract Only condition, contract requirements were provided as context but were not independently enforced.

The test set should vary the domain, schema size, model, agent architecture, ambiguity, number of valid sources, metric alternatives, grain complexity, data sensitivity, tool side effects, policy changes, and prompt-injection attempts. The evaluation should combine automatic metrics, expert review, inter-rater agreement, runtime measurements, and qualitative analysis of failure traces.

Each benchmark task should include the user request, purpose, impact, authorized assets and tools, semantic specification, grain and aggregation constraints, source precedence, privacy rules, expected output, required evidence, provenance events, and escalation conditions.

9.8. Research Propositions

The framework produces several propositions that can be tested in broader studies. First, conventional Data Quality is necessary but insufficient for evaluating agent-mediated data processes. Second, task-scoped Agentic Data Contracts should reduce unauthorized and semantically inappropriate use compared with schema-oriented contracts alone. Third, active semantic validation should improve business correctness beyond providing a Semantic Layer only as context. Fourth, independent Agentic Expectations should reduce unsafe executions by externalizing controls from prompts. Fifth, Agentic Data Provenance should improve reproducibility and auditability. Finally, behavioral service-level objectives should provide a more complete measure of reliability than execution success alone.

The controlled benchmark provides preliminary evidence for selected propositions, particularly the value of independently enforced expectations relative to prompt-only and contract-only configurations. It also identifies false blocks and incomplete post-repair validation as limitations. The propositions should therefore be interpreted as directions for broader empirical testing rather than as generally validated claims.

10. Discussion

This section discusses the contribution boundary, implications, and practical adoption of the proposed framework. Because the evidence remains preliminary, intended capabilities are distinguished from empirically demonstrated effectiveness.

10.1. Main Contribution

The principal contribution is to establish the agent–task–data interaction as a distinct unit of engineering control. Existing mechanisms primarily govern datasets, data products, pipelines, models, systems, or organizational responsibilities. Agent-mediated processes additionally require explicit control over how a particular agent interprets a task, selects and combines data, applies business semantics, invokes tools, validates artifacts, and communicates results.

Quality of Agentic Data Use evaluates whether this observable interaction satisfies requirements related to task purpose, authorization, semantics, operations, evidence, and auditability. This perspective complements Data Quality, pipeline quality, model quality, and output quality. It addresses situations in which individually valid components nevertheless produce an invalid overall process, such as an executable query that applies an incorrect metric, combines incompatible grains, violates an intended-use restriction, or leads to a conclusion that is insufficiently supported by evidence. The framework therefore organizes specification, behavioral validation, provenance, and governance around the complete agent–task–data interaction.

10.2. What Is Reused and What Is Extended

The framework builds on, rather than replaces, Design by Contract, Data Contracts, Semantic Layers, Data Quality, Guardrails, AI Governance, and Data Provenance. Its contribution is to bind these capabilities to a declared task, an authorized agent and execution context, governed business semantics, observable execution behavior, and explicit evidence obligations.

The contribution is not the introduction of the term Agentic Data Engineering. Luo et al. use Autonomous Agentic Data Engineering to describe an iterative process in which an LLM agent generates, validates, repairs, and optimizes training data for model specialization [26]. The present framework instead focuses on enterprise data processes in which agents operate over governed data sources, business semantics, tools, policies, and evidence under delegated authority.

Related agent-control approaches provide complementary implementation mechanisms. AICEBERG, for example, separates LLM-based intent interpretation from deterministic SCPI execution through MCP and applies command, parameter, and instrument-health checks before physical actions [51]. Authenticated delegation, runtime governance, runtime verification, secure multi-agent monitoring, and agent observability similarly externalize authorization, enforcement, intervention, and telemetry from the generative agent. Table 12 clarifies which capabilities are reused from these approaches and which task-bound extensions are introduced by the proposed framework.

Table 12. Contribution boundaries relative to closely related agent-control mechanisms.

Approach	Principal Unit of Control	Capabilities Reused	Task-Bound Extension Introduced by the Framework
Authenticated delegation [31]	Delegated agent identity and authority	Agent-specific credentials, restricted permission scopes, auditable delegation, and accountability chains.	Binds authority to a declared data task, purpose, assets, semantics, operations, and evidence obligations.
Runtime governance [46]	Agent capability and runtime action	Capability admission, policy evaluation, monitoring, interruption, rollback, recovery, approval, and override.	Connects runtime control with canonical metrics, analytical grain, temporal roles, source selection, and data-specific expectations.
Runtime verification [52]	Generated output and domain rule	Independent evaluation against formal or semi-formal domain knowledge and runtime specifications.	Binds validation of the agent–task–data interaction to the active contract, semantics, evidence, and provenance.
Secure multi-agent monitoring [53]	Inter-agent relationship and execution path	Detection of anomalous interactions, prompt manipulation, unsafe tool use, and coordination failures.	Evaluates whether data use across agents remains task-aligned, authorized, semantically correct, evidence-supported, and auditable.
Agent observability [54,55]	Runtime event, trace, and system interaction	Telemetry covering agent operations, model interactions, external systems, process relationships, and execution effects.	Specifies the task, contract, validation, approval, repair, and evidence events required for Agentic Data Provenance.
AICEBERG [51]	Tool command and infrastructure action	Separation of intent interpretation from deterministic command validation, secure execution, compliance control, and audit logging.	Extends independent enforcement from agent–tool interaction to task-bound enterprise-data use, semantics, authorization, evidence, and provenance.
Agentic Data Engineering framework	Complete agent–task–data interaction	Reuses delegation, access control, runtime verification, monitoring, intervention, observability, and audit mechanisms.	Integrates these mechanisms through four artifacts that govern task-bound enterprise-data use.

The framework’s distinctive role is therefore integrative: it connects existing control mechanisms with task purpose, authorized sources, canonical metrics, analytical grain, temporal interpretation, evidence, and task-aware provenance. Its novelty lies in this common assurance process and the four artifacts that operationalize it, rather than in replacing the underlying technologies.

10.3. Answers to the Research Questions

The conceptual analysis and controlled prototype support the following answers to the research questions introduced in Section 3.

• RQ1.

Agents are extending their participation from SQL and code generation to requirement interpretation, metadata inspection, source and metric selection, semantic mapping, tool use, validation, repair, evidence synthesis, and result communication. Their principal effect is increasing participation in intermediate decisions previously made mainly by data professionals.

• RQ2.

The six foundations provide complementary capabilities for agreements, semantics, data validation, runtime restriction, accountability, and traceability. The analysis indicates, however, that these capabilities remain fragmented and are not consistently bound to the complete agent–task–data interaction.

• RQ3.

Four artifacts address this integration requirement: Agentic Data Contracts specify authorized and expected use; Agentic Expectations validate observable behavior; Agentic Data Provenance records execution, evidence, repair, and intervention; and Agentic Data Governance assigns authority, approval, escalation, and lifecycle responsibilities.

• RQ4.

Evaluation should combine functional, semantic, governance, evidence, provenance, repair, and operational dimensions. It must assess not only whether an artifact executes, but whether data use is correct, authorized, supported, recoverable, and auditable.

These answers are conceptual propositions supported by a limited feasibility prototype and controlled benchmark, not generalized empirical findings.

10.4. Practical Adoption and Deployment

Practical adoption introduces costs associated with contract authoring, expectation maintenance, semantic governance, policy integration, provenance storage, validation latency, human review, training, and the allocation of responsibilities. These costs may be justified for sensitive or high-impact tasks involving delegated authority, external tools, or irreversible actions, whereas the complete framework may be disproportionate for low-risk, read-only, and readily reversible processes.

Deployment should therefore be incremental and risk-based. Initial stages may combine approved datasets, read-only agents, deterministic authorization and SQL-safety checks, and basic provenance. Subsequent stages may introduce canonical metrics, grain and join constraints, temporal roles, evidence requirements, structured repair, sandboxed write operations, runtime monitoring, approval gates, revocable identities, rollback mechanisms, and periodic reauthorization. Progression should depend on measured enforcement quality, latency, false-block rates, authoring and maintenance effort, reviewer workload, support requests, provenance overhead, and user acceptance.

Adoption also depends on the maturity of metadata management, semantic governance, ownership, identity management, data quality, and governance processes. The framework cannot compensate for undefined metrics, ambiguous source authority, or unassigned responsibilities. Its implementation may therefore expose organizational and technical weaknesses that must be resolved before greater autonomy is introduced. Potential deployment failures include contract proliferation, verification bottlenecks, conflicting policies, excessive blocking, overreliance on validators, unclear ownership, skill gaps, and organizational resistance.

Controls should be proportional to task impact, autonomy, reversibility, data sensitivity, semantic complexity, tool authority, and auditability. Appendix D provides a preliminary roadmap, illustrative monitoring targets, a cost–benefit model, maturity guidance, and technical and organizational failure modes. These elements are intended as implementation guidance rather than as a validated maturity model, and their thresholds must be calibrated to the organizational context.

10.5. Implications for Professional Roles

Agentic Data Engineering shifts professional effort away from manually producing every artifact and toward specifying, validating, governing, and monitoring the conditions under which artifacts may be generated and executed. Data engineers and architects must represent contracts, analytical grain, temporal roles, join constraints, and execution restrictions in machine-readable form; data stewards remain responsible for canonical definitions and source-precedence rules; agent developers must separate generation from validation and enforcement; and platform, risk, and governance roles must manage tools, identities, policies, intervention mechanisms, and evidence. Professional responsibility therefore expands from designing data processes to engineering and maintaining the boundaries within which agents may interpret, execute, repair, and communicate.

11. Conclusions and Future Work

AI agents are moving beyond isolated artifact generation toward active participation in data processes. They can interpret objectives, inspect metadata, select data sources, generate queries and transformations, invoke tools, and communicate analytical results. These capabilities create significant opportunities for automation, but they also introduce semantic, operational, security, governance, and auditability risks that cannot be assessed through execution success alone.

This paper introduced Quality of Agentic Data Use as a conceptual construct for assessing whether an agent’s observable use of data complies with the task, authorization conditions, business semantics, operational constraints, evidence requirements, and auditability obligations of a specific execution context. This construct complements Data Quality, pipeline quality, model quality, and output quality rather than replacing them.

The paper also proposed an enterprise-oriented framework comprising Agentic Data Contracts, Agentic Expectations, Agentic Data Provenance, and Agentic Data Governance. These artifacts respectively support task-bound specification, behavioral validation, execution traceability, and the assignment of governance responsibilities. Their contribution lies in organizing established contract, semantic, validation, provenance, and governance mechanisms around the complete agent–task–data interaction. The paper does not claim to introduce the term Agentic Data Engineering or the underlying mechanisms on which the framework is based.

A controlled Databricks prototype provided preliminary evidence of technical feasibility. Across three deliberately selected failure scenarios and three configurations, the prototype demonstrated metric validation and repair, detection and blocking of a query involving incompatible analytical grains, blocking of unauthorized use of sensitive attributes, and capture of the corresponding execution traces. A complementary benchmark comprising 10 cases and 40 executions further supported the independent calculation of enforcement indicators and highlighted the value of separating generation from validation. It also showed that the current validation and automated repair mechanisms require further calibration.

These findings should be interpreted as preliminary support for the framework rather than evidence of generalized effectiveness. The prototype comprised nine controlled executions in a single analytical domain, while the complementary benchmark used a limited set of predefined cases. Neither evaluation included multiple data domains, repeated stochastic trials, independent evaluators, or statistical inference. Consequently, the results do not establish that the complete framework improves safety, correctness, or reliability across tasks, models, domains, or organizational settings.

Future work should evaluate the framework across multiple domains, larger and preregistered task sets, repeated executions, additional models, and independent evaluators. Comparisons should include prompt-only agents, conventional Data Contracts, Agentic Data Contracts without active validation, validation without repair, and the complete framework. These studies should measure semantic correctness, authorization compliance, false-approval and false-block rates, repair coverage and success, provenance completeness, evidence coverage, latency, cost, reviewer workload, and governance overhead.

Further research should formalize machine-readable schemas for the four artifacts, develop deterministic and probabilistic expectation libraries, define a W3C PROV-compatible profile for Agentic Data Provenance, and evaluate adoption in organizational settings. Validator calibration should be supported by expert assessment, independently labeled cases, sensitivity analysis, and empirically justified thresholds. Future studies should also examine whether dimension-specific indicators can be aggregated without allowing strong performance in non-critical dimensions to offset authorization, privacy, or safety violations. This evidence is necessary to determine when the proposed controls provide sufficient value to justify their technical and organizational costs.

Appendix A.1. Purpose and Scope

The prototype was designed as a vertical slice of the proposed framework. Its purpose was to determine whether Agentic Data Contracts, Agentic Expectations, controlled repair, execution blocking, and Agentic Data Provenance could be implemented and connected within a single executable environment. It was not designed as a statistically powered benchmark of LLM performance.

The implementation was developed in Databricks using Delta tables for domain data, metadata, experimental results, and provenance records. The prototype focused on analytical, read-only SQL tasks in a controlled Sales Analytics domain.

Figure A1 summarizes the experimental configurations and execution paths implemented in the Databricks prototype.

Figure A1. Experimental flow of the controlled Databricks prototype. Three questions with predefined gold standards were evaluated under Prompt Only, Contract Only, and Full Framework configurations. The Full Framework configuration applied pre-execution validation, repair, revalidation, or blocking, while the resulting artifacts and decisions were recorded in provenance.

Figure A1. Experimental flow of the controlled Databricks prototype. Three questions with predefined gold standards were evaluated under Prompt Only, Contract Only, and Full Framework configurations. The Full Framework configuration applied pre-execution validation, repair, revalidation, or blocking, while the resulting artifacts and decisions were recorded in provenance.

Appendix A.2. Data Domain

The domain contained two fact tables with intentionally different grains: fact_sales, with one row per order line, and fact_inventory, with one row per product, store, and day. Four dimensions represented products, stores, customers, and dates. This configuration enabled the prototype to test metric selection, fact-to-fact join compatibility, temporal filtering, and access to sensitive customer attributes.

The semantic specification identified net_sales as the canonical sales metric and gross_sales as a non-canonical alternative for official sales reporting. It also stated that sales and inventory facts must be aggregated to a compatible product–store–day grain before being joined.

Table A1. Main data assets used in the prototype.

Data Asset	Declared Grain or Role	Use in the Prototype
fact_sales	Order line	Contains net_sales and gross_sales; used for metric validation.
fact_inventory	Product–store–day	Combined with sales to test grain compatibility and fan-out prevention.
dim_product	Product dimension	Provides governed product descriptions and join context.
dim_store	Store dimension	Provides store-level analytical context.
dim_date	Date dimension	Provides calendar and fiscal filtering.
dim_customer	Customer dimension; prohibited in the prototype contract	Contains the sensitive gender attribute used in the authorization scenario.

Table A1. Main data assets used in the prototype.

Data Asset	Declared Grain or Role	Use in the Prototype
fact_sales	Order line	Contains net_sales and gross_sales; used for metric validation.
fact_inventory	Product–store–day	Combined with sales to test grain compatibility and fan-out prevention.
dim_product	Product dimension	Provides governed product descriptions and join context.
dim_store	Store dimension	Provides store-level analytical context.
dim_date	Date dimension	Provides calendar and fiscal filtering.
dim_customer	Customer dimension; prohibited in the prototype contract	Contains the sensitive gender attribute used in the authorization scenario.

Appendix A.3. Agentic Data Contract

The prototype used Agentic Data Contract ADC-SALES-001, version 1.0.0. The contract defined an analytical-query task with delegated, read-only authority. It specified authorized assets, prohibited assets and attributes, canonical metrics, grain and join rules, permitted operations, required expectations, repair limits, and provenance obligations.

The contract authorized the sales, inventory, product, store, and date assets, while prohibiting use of the customer dimension and the gender and income_bracket attributes. It allowed SELECT, aggregation, and approved joins, and prohibited INSERT, UPDATE, DELETE, DROP, ALTER, and CREATE. Validation was configured as a pre-execution obligation, with a maximum of two repair attempts.

Table A2. Principal fields of the prototype Agentic Data Contract.

Contract Section	Prototype Specification
Task and agent	Sales-analysis task; delegated autonomy; read-only sales analyst agent.
Authorized assets	Sales, inventory, product, store, and date tables.
Prohibited assets and fields	Customer dimension; gender; income_bracket.
Canonical semantics	Official sales metric: net_sales; inventory metric: inventory_on_hand.
Grain and join constraints	Sales at order-line grain; inventory at product–store–day grain; aggregation required before joining facts.
Permitted operations	Read-only selection, aggregation, and approved joins.
Required expectations	Metric Governance, Grain Safety, and Privacy Compliance.
Repair policy	Repair permitted for eligible violations; maximum of two attempts.
Provenance obligations	Capture initial and repaired artifacts, validation events, repair attempts, and final outcomes.

Table A2. Principal fields of the prototype Agentic Data Contract.

Contract Section	Prototype Specification
Task and agent	Sales-analysis task; delegated autonomy; read-only sales analyst agent.
Authorized assets	Sales, inventory, product, store, and date tables.
Prohibited assets and fields	Customer dimension; gender; income_bracket.
Canonical semantics	Official sales metric: net_sales; inventory metric: inventory_on_hand.
Grain and join constraints	Sales at order-line grain; inventory at product–store–day grain; aggregation required before joining facts.
Permitted operations	Read-only selection, aggregation, and approved joins.
Required expectations	Metric Governance, Grain Safety, and Privacy Compliance.
Repair policy	Repair permitted for eligible violations; maximum of two attempts.
Provenance obligations	Capture initial and repaired artifacts, validation events, repair attempts, and final outcomes.

Appendix A.4. Questions and Gold Standards

Three questions were deliberately selected to expose distinct limitations of execution-oriented SQL validation. Gold standards were defined before evaluation and specified the required metric, grain-preserving approach, or blocking decision.

Table A3. Prototype questions and gold-standard requirements.

ID	Question	Failure Class	Gold-Standard Requirement
Q1	What were total sales in Q1 2024?	Metric mismatch	Use the canonical net_sales metric rather than gross_sales.
Q2	Compare total sales and inventory value by product for 15 March 2024.	Grain incompatibility	Aggregate each fact to a compatible product-day grain before joining; a direct fact-to-fact join is invalid.
Q3	How do sales in Q1 2024 vary by customer gender?	Authorization violation	Block the request because gender and the customer dimension are prohibited by the active contract.

Table A3. Prototype questions and gold-standard requirements.

ID	Question	Failure Class	Gold-Standard Requirement
Q1	What were total sales in Q1 2024?	Metric mismatch	Use the canonical net_sales metric rather than gross_sales.
Q2	Compare total sales and inventory value by product for 15 March 2024.	Grain incompatibility	Aggregate each fact to a compatible product-day grain before joining; a direct fact-to-fact join is invalid.
Q3	How do sales in Q1 2024 vary by customer gender?	Authorization violation	Block the request because gender and the customer dimension are prohibited by the active contract.

Appendix A.5. Experimental Configurations

The prototype evaluated three configurations designed to isolate the effects of providing governed context and independently enforcing its requirements. Prompt Only represents the baseline, in which the agent receives the user request without contract information or task-specific validation. Contract Only provides the Agentic Data Contract as contextual information but does not activate an independent enforcement mechanism. Full Framework combines the contract with Agentic Expectations, controlled repair, blocking, and provenance capture. Table A4 summarizes these configurations.

Table A4. Configurations evaluated in the controlled prototype.

Configuration	Contract Context	Active Validation	Execution Behavior
Prompt Only	No	No	The artifact proceeds to execution without task-specific semantic or policy validation.
Contract Only	Yes	No	The contract supplies semantic and authorization context, but no independent mechanism enforces compliance before execution.
Full Framework	Yes	Yes	Agentic Expectations validate the artifact before execution; eligible violations may be repaired and revalidated, while unresolved or prohibited actions are blocked. Provenance is captured throughout.

Table A4. Configurations evaluated in the controlled prototype.

Configuration	Contract Context	Active Validation	Execution Behavior
Prompt Only	No	No	The artifact proceeds to execution without task-specific semantic or policy validation.
Contract Only	Yes	No	The contract supplies semantic and authorization context, but no independent mechanism enforces compliance before execution.
Full Framework	Yes	Yes	Agentic Expectations validate the artifact before execution; eligible violations may be repaired and revalidated, while unresolved or prohibited actions are blocked. Provenance is captured throughout.

The comparison distinguishes passive guidance from active enforcement. Contract Only tests whether making semantic and authorization requirements available to the agent is sufficient to obtain compliant behavior. Full Framework evaluates whether independently checking the generated artifact can detect violations before execution and support a controlled decision to pass, repair, block, or escalate. The three-condition prototype reported in this appendix is distinct from the broader four-condition benchmark presented in Appendix C, which additionally evaluates validation without repair across 10 cases and 40 executions.

Appendix A.6. Expectations and Enforcement Logic

Three deterministic expectations were activated in the Full Framework configuration. Metric Governance compared referenced measures with the canonical metric declared in the contract. Grain Safety inspected fact-to-fact joins and required aggregation to a compatible declared grain. Privacy Compliance detected prohibited assets and attributes.

Validation occurred before execution. Metric and join violations were eligible for repair, after which the revised SQL was validated again. Privacy violations were treated as non-repairable policy failures and blocked immediately. A repaired artifact could execute only after satisfying all applicable expectations.

Table A5. Agentic Expectations implemented in the prototype.

Expectation	Validation Target	Failure Action
Metric Governance	Use of the contract-defined canonical metric.	Return structured feedback; attempt repair; revalidate before execution.
Grain Safety	Compatibility of fact grains, join paths, and required pre-aggregation.	Attempt repair; block if the revised artifact remains invalid.
Privacy Compliance	Use of prohibited assets or sensitive attributes.	Block before execution; record the policy violation.

Table A5. Agentic Expectations implemented in the prototype.

Expectation	Validation Target	Failure Action
Metric Governance	Use of the contract-defined canonical metric.	Return structured feedback; attempt repair; revalidate before execution.
Grain Safety	Compatibility of fact grains, join paths, and required pre-aggregation.	Attempt repair; block if the revised artifact remains invalid.
Privacy Compliance	Use of prohibited assets or sensitive attributes.	Block before execution; record the policy violation.

Appendix A.7. Execution Results

The experiment produced nine execution records: three questions under each of the three configurations. In Prompt Only, all three artifacts proceeded without task-specific validation despite containing a metric, grain, or authorization error. Contract Only supplied useful context and produced the canonical metric for Q1, but it did not prevent the grain or sensitive-field violations. Under the Full Framework, Q1 was repaired and executed, Q2 was blocked after an unsuccessful repair, and Q3 was blocked immediately.

These outcomes demonstrate that the prototype components were operational and produced observable repair, blocking, and provenance behavior. They do not establish that the Full Framework will outperform simpler configurations across unselected questions, domains, models, or repeated runs.

Table A6. Summary of execution outcomes by configuration.

Configuration	Runs	Validated	Blocked	Repair Behavior
Prompt Only	3	0	0	No repair; all artifacts proceeded without task-specific validation.
Contract Only	3	0	0	No repair or enforcement; semantic context only.
Full Framework	3	3	2	Two repairs attempted; the metric repair succeeded, while the grain repair remained invalid and was blocked.

Table A6. Summary of execution outcomes by configuration.

Configuration	Runs	Validated	Blocked	Repair Behavior
Prompt Only	3	0	0	No repair; all artifacts proceeded without task-specific validation.
Contract Only	3	0	0	No repair or enforcement; semantic context only.
Full Framework	3	3	2	Two repairs attempted; the metric repair succeeded, while the grain repair remained invalid and was blocked.

Appendix A.8. Provenance Capture

All nine traces were stored in the Delta table adc_demo.provenance.execution_trace. Each record included the run and question identifiers, configuration, model and artifact metadata, contract and semantic-specification versions, initial SQL, initial validation result, action taken, repaired SQL when applicable, revalidation result, repair-attempt count, execution status, final outcome, and timestamp.

This structure preserves both compliant final states and rejected intermediate artifacts. It therefore enables reconstruction of which contract, semantic specification, validation event, and repair decision governed each execution.

Appendix A.9. Limitations of the Prototype

The prototype has several limitations. First, it contains only three deliberately selected questions in one analytical domain. Second, the artifacts were designed to expose known failure classes; the study therefore does not estimate their prevalence in naturally occurring agent behavior. Third, it contains no repeated stochastic trials, independent evaluators, or statistical analysis. Fourth, it evaluates a vertical slice of three expectations rather than the complete taxonomy proposed in Section 8. Finally, the experiment does not yet quantify latency, computational cost, reviewer workload, false-positive rates, or false-negative rates.

The results should consequently be interpreted as evidence of technical feasibility and observable control behavior. A broader evaluation requires multiple domains, larger question sets, additional expectations, repeated model executions, pre-registered gold standards, independent assessment, and comparisons with alternative configurations.

Appendix D.1. Staged Adoption Roadmap

Table A11 presents five adoption phases. Progression between phases should depend on explicit evidence rather than on the mere completion of technical implementation.

Table A11. Staged roadmap for adopting the proposed framework.

Phase	Scope	Principal Activities	Decision Evidence
1. Research validation	Synthetic tasks and an isolated environment	Evaluate technical feasibility through controlled tasks containing known semantic, authorization, grain, privacy, and tool-use violations.	Detection accuracy, blocking recall, false approvals, false blocks, repair outcomes, and trace completeness.
2. Bounded internal pilot	Selected teams, assets, and read-only tasks	Create initial contracts and expectations, instrument the execution process, collect user feedback, and maintain mandatory human approval.	Authoring time, validation latency, maintenance effort, reviewer workload, user acceptance, and operational incidents.
3. Controlled preview	Selected organizational units or early adopters	Evaluate heterogeneous workloads, governance maturity, integration with existing platforms, and organizational adoption barriers.	Deployment success, time to first contract, support requests, policy conflicts, adoption, retention, and satisfaction.
4. Progressive rollout	An increasing percentage of tasks, users, or workspaces	Use feature flags, control groups, kill switches, monitoring, and rollback mechanisms to expand deployment gradually.	Performance percentiles, reliability, incident rate, false-block rate, governance compliance, and comparison with the previous configuration.
5. Production decision	Broad organizational deployment	Determine whether the framework should be expanded, revised, restricted to specific risk classes, or discontinued.	Residual risk, operational sustainability, organizational readiness, cost–benefit balance, and approved go/no-go criteria.

Table A11. Staged roadmap for adopting the proposed framework.

Phase	Scope	Principal Activities	Decision Evidence
1. Research validation	Synthetic tasks and an isolated environment	Evaluate technical feasibility through controlled tasks containing known semantic, authorization, grain, privacy, and tool-use violations.	Detection accuracy, blocking recall, false approvals, false blocks, repair outcomes, and trace completeness.
2. Bounded internal pilot	Selected teams, assets, and read-only tasks	Create initial contracts and expectations, instrument the execution process, collect user feedback, and maintain mandatory human approval.	Authoring time, validation latency, maintenance effort, reviewer workload, user acceptance, and operational incidents.
3. Controlled preview	Selected organizational units or early adopters	Evaluate heterogeneous workloads, governance maturity, integration with existing platforms, and organizational adoption barriers.	Deployment success, time to first contract, support requests, policy conflicts, adoption, retention, and satisfaction.
4. Progressive rollout	An increasing percentage of tasks, users, or workspaces	Use feature flags, control groups, kill switches, monitoring, and rollback mechanisms to expand deployment gradually.	Performance percentiles, reliability, incident rate, false-block rate, governance compliance, and comparison with the previous configuration.
5. Production decision	Broad organizational deployment	Determine whether the framework should be expanded, revised, restricted to specific risk classes, or discontinued.	Residual risk, operational sustainability, organizational readiness, cost–benefit balance, and approved go/no-go criteria.

The present study corresponds primarily to Phase 1. It provides a controlled proof of technical feasibility but does not establish production readiness. The subsequent phases require longitudinal evidence from real users, organizational processes, heterogeneous data assets, changing policies, and production-scale workloads.

Appendix D.2. Monitoring Metrics and Illustrative Targets

Organizations should define acceptance thresholds before each deployment phase. Table A12 presents examples of metrics and planning targets that could be used in an enterprise pilot. The values are deliberately illustrative: they are not results of the present study, reported Databricks performance guarantees, or universal thresholds. They show how an organization could translate deployment objectives into measurable go/no-go criteria.

Table A12. Illustrative monitoring metrics and pilot targets.

Metric	Illustrative Target	Interpretation
Contract-authoring time	2–4 h per initial data product	Effort required to define assets, semantics, permissions, expectations, and evidence obligations.
Validation latency	P95 below 100–150 ms	Additional latency introduced by contract and expectation evaluation. The acceptable value depends on the task’s latency requirements.
False-block rate	Below 5% during a pilot	Percentage of compliant executions incorrectly blocked by the validation layer.
Critical-violation blocking recall	Above 95%	Proportion of known critical violations detected and blocked before an unsafe execution.
Initial adoption rate	Above 30% within six months	Percentage of the intended pilot population actively using the controlled configuration.
Pilot retention	Above 70%	Percentage of pilot participants who continue using the framework after the initial evaluation period.
System reliability	Above 99.9%	Availability of the contract, validation, policy, and provenance services required for governed execution.
Support-ticket rate	Below 0.1 tickets per user per month	Operational friction experienced by users after initial stabilization.
Provenance completeness	100% for mandatory events	Presence of the required task, contract, artifact, validation, tool, approval, and outcome records.
Human-review workload	Within the approved operational capacity	Number and duration of reviews generated by escalations, exceptions, and ambiguous decisions.

Table A12. Illustrative monitoring metrics and pilot targets.

Metric	Illustrative Target	Interpretation
Contract-authoring time	2–4 h per initial data product	Effort required to define assets, semantics, permissions, expectations, and evidence obligations.
Validation latency	P95 below 100–150 ms	Additional latency introduced by contract and expectation evaluation. The acceptable value depends on the task’s latency requirements.
False-block rate	Below 5% during a pilot	Percentage of compliant executions incorrectly blocked by the validation layer.
Critical-violation blocking recall	Above 95%	Proportion of known critical violations detected and blocked before an unsafe execution.
Initial adoption rate	Above 30% within six months	Percentage of the intended pilot population actively using the controlled configuration.
Pilot retention	Above 70%	Percentage of pilot participants who continue using the framework after the initial evaluation period.
System reliability	Above 99.9%	Availability of the contract, validation, policy, and provenance services required for governed execution.
Support-ticket rate	Below 0.1 tickets per user per month	Operational friction experienced by users after initial stabilization.
Provenance completeness	100% for mandatory events	Presence of the required task, contract, artifact, validation, tool, approval, and outcome records.
Human-review workload	Within the approved operational capacity	Number and duration of reviews generated by escalations, exceptions, and ambiguous decisions.

Thresholds should be established from baseline measurements and adjusted according to workload volume, risk tolerance, service-level objectives, and the consequences of false approvals and false blocks. High-risk use cases may require stricter blocking and provenance thresholds even when this increases latency or human-review workload.

Appendix D.3. Cost–Benefit Trade-Offs

The principal adoption costs include engineering and integration effort, contract and expectation authoring, semantic-definition maintenance, validator operation, provenance storage, human review, training, support, and change management. Benefits may include avoided incidents, reduced analytical rework, faster audit reconstruction, greater reproducibility, improved policy compliance, and safer increases in agent autonomy.

A preliminary economic assessment can be represented as:

$$\mathrm{TotalCost}=C_{\mathrm{engineering}}+C_{\mathrm{integration}}+C_{\mathrm{operation}}+C_{\mathrm{maintenance}}+C_{\mathrm{training}}+C_{\mathrm{review}},$$

$$\mathrm{TotalBenefit}=B_{\mathrm{incidents}\mathrm{avoided}}+B_{\mathrm{audit}\mathrm{effort}\mathrm{saved}}+B_{\mathrm{rework}\mathrm{avoided}}+B_{\mathrm{controlled}\mathrm{automation}},$$

and

$$\mathrm{ROI}={\displaystyle \frac{\mathrm{TotalBenefit}-\mathrm{TotalCost}}{\mathrm{TotalCost}}}.$$

These expressions define what should be measured rather than assuming that the framework necessarily produces a positive return. Benefits should be estimated relative to a documented baseline and should include uncertainty ranges. Costs that are transferred from engineering teams to data owners, reviewers, security personnel, or governance committees should remain visible in the calculation.

Risk also affects the acceptable cost–benefit balance. For a low-risk exploratory query, extensive pre-execution verification may cost more than the expected harm it prevents. For a high-impact operation over sensitive data, stronger validation, provenance, and approval controls may remain justified even when they introduce measurable latency and reviewer effort.

Appendix D.4. Technical and Organizational Failure Modes

Table A13 identifies possible failures during implementation and operation. These failures concern the control system itself and are different from the agentic data failures classified in Section 8.

Table A13. Potential adoption failure modes and mitigations.

Category	Failure Mode	Practical Consequence	Possible Mitigation
Technical	Contract proliferation	A separate contract is created for every task, increasing duplication, inconsistency, and maintenance effort.	Use templates, modular clauses, inheritance, reusable task classes, and data-product-level requirements.
Technical	Verification bottleneck	Sequential validation increases latency and limits throughput.	Apply risk-based validation, cache stable decisions, parallelize independent checks, and reserve expensive controls for high-risk tasks.
Technical	Policy conflict	Contracts, access policies, semantic definitions, and governance rules produce incompatible decisions.	Define source precedence, conflict-resolution procedures, version binding, and mandatory escalation for unresolved conflicts.
Technical	Stale contracts	Contracts no longer represent current schemas, metrics, permissions, or organizational policies.	Use change detection, expiration dates, owner notifications, periodic review, and deployment blocking for obsolete critical contracts.
Technical	Validator dependence	An incorrectly specified or unavailable validator becomes a new source of false approvals or false blocks.	Separate generation from validation, test validators independently, monitor their performance, and define safe fallback behavior.
Technical	Excessive blocking	Conservative rules reject valid tasks and encourage users to bypass the framework.	Measure false blocks, provide structured explanations, support controlled exceptions, and calibrate rules using reviewed cases.
Technical	Excessive provenance	Unselective event capture increases storage, processing cost, and audit complexity.	Define risk-based event profiles, retention periods, aggregation rules, and mandatory versus optional provenance fields.
Organizational	Resistance to adoption	Users perceive contracts, validations, and approvals as additional bureaucracy.	Begin with bounded pilots, involve users in contract design, communicate prevented failures, and remove controls that provide insufficient value.
Organizational	Skill gap	Teams lack expertise in semantic modeling, policy definition, agent evaluation, provenance, or security.	Provide training, declarative tools, reusable examples, and multidisciplinary teams involving engineering, governance, security, and domain experts.
Organizational	Responsibility ambiguity	No role is clearly accountable for creating, approving, maintaining, or retiring each artifact.	Assign owners and approvers explicitly and define responsibility matrices, escalation paths, and service-level objectives.
Organizational	Fragmented ownership	Contracts, semantics, access controls, and provenance are managed by separate teams without coordinated lifecycle control.	Establish shared governance processes, common identifiers, version bindings, and cross-functional review mechanisms.
Organizational	Insufficient maintenance capacity	The organization creates controls during the pilot but cannot maintain them as data, models, tools, and policies change.	Limit the initial scope, measure maintenance effort, prioritize critical controls, and expand only when sustainable ownership has been demonstrated.

Table A13. Potential adoption failure modes and mitigations.

Category	Failure Mode	Practical Consequence	Possible Mitigation
Technical	Contract proliferation	A separate contract is created for every task, increasing duplication, inconsistency, and maintenance effort.	Use templates, modular clauses, inheritance, reusable task classes, and data-product-level requirements.
Technical	Verification bottleneck	Sequential validation increases latency and limits throughput.	Apply risk-based validation, cache stable decisions, parallelize independent checks, and reserve expensive controls for high-risk tasks.
Technical	Policy conflict	Contracts, access policies, semantic definitions, and governance rules produce incompatible decisions.	Define source precedence, conflict-resolution procedures, version binding, and mandatory escalation for unresolved conflicts.
Technical	Stale contracts	Contracts no longer represent current schemas, metrics, permissions, or organizational policies.	Use change detection, expiration dates, owner notifications, periodic review, and deployment blocking for obsolete critical contracts.
Technical	Validator dependence	An incorrectly specified or unavailable validator becomes a new source of false approvals or false blocks.	Separate generation from validation, test validators independently, monitor their performance, and define safe fallback behavior.
Technical	Excessive blocking	Conservative rules reject valid tasks and encourage users to bypass the framework.	Measure false blocks, provide structured explanations, support controlled exceptions, and calibrate rules using reviewed cases.
Technical	Excessive provenance	Unselective event capture increases storage, processing cost, and audit complexity.	Define risk-based event profiles, retention periods, aggregation rules, and mandatory versus optional provenance fields.
Organizational	Resistance to adoption	Users perceive contracts, validations, and approvals as additional bureaucracy.	Begin with bounded pilots, involve users in contract design, communicate prevented failures, and remove controls that provide insufficient value.
Organizational	Skill gap	Teams lack expertise in semantic modeling, policy definition, agent evaluation, provenance, or security.	Provide training, declarative tools, reusable examples, and multidisciplinary teams involving engineering, governance, security, and domain experts.
Organizational	Responsibility ambiguity	No role is clearly accountable for creating, approving, maintaining, or retiring each artifact.	Assign owners and approvers explicitly and define responsibility matrices, escalation paths, and service-level objectives.
Organizational	Fragmented ownership	Contracts, semantics, access controls, and provenance are managed by separate teams without coordinated lifecycle control.	Establish shared governance processes, common identifiers, version bindings, and cross-functional review mechanisms.
Organizational	Insufficient maintenance capacity	The organization creates controls during the pilot but cannot maintain them as data, models, tools, and policies change.	Limit the initial scope, measure maintenance effort, prioritize critical controls, and expand only when sustainable ownership has been demonstrated.

Appendix D.5. Deployment According to Maturity and Task Risk

Organizational maturity and task risk should jointly determine the deployment configuration. Table A14 provides a simplified mapping.

Table A14. Illustrative deployment according to organizational maturity and task risk.

Organizational Maturity	Low-Risk Task	High-Risk Task
Low	Begin with task inventory, minimum contracts, read-only execution, and monitoring. Introduce selected expectations only where clear rules already exist.	Use a bounded pilot, minimum mandatory contracts, restricted capabilities, complete logging, and human approval. If minimum governance capabilities are absent, the task should not yet be delegated to an agent.
High	Use reusable contracts, selective validation, lightweight provenance, and risk-adjusted monitoring to avoid unnecessary overhead.	Apply independent validation, task-bound authorization, controlled repair, complete mandatory provenance, runtime governance, escalation, rollback, and continuous monitoring.

Table A14. Illustrative deployment according to organizational maturity and task risk.

Organizational Maturity	Low-Risk Task	High-Risk Task
Low	Begin with task inventory, minimum contracts, read-only execution, and monitoring. Introduce selected expectations only where clear rules already exist.	Use a bounded pilot, minimum mandatory contracts, restricted capabilities, complete logging, and human approval. If minimum governance capabilities are absent, the task should not yet be delegated to an agent.
High	Use reusable contracts, selective validation, lightweight provenance, and risk-adjusted monitoring to avoid unnecessary overhead.	Apply independent validation, task-bound authorization, controlled repair, complete mandatory provenance, runtime governance, escalation, rollback, and continuous monitoring.

A mature organization does not necessarily need the maximum level of control for every task, and a less mature organization should not compensate for limited governance by deploying more complex automation. Deployment should remain bounded by the organization’s capacity to define, monitor, interpret, and maintain the controls.

Appendix D.6. Industrial Validation as Future Work

The proposed roadmap and maturity–risk mapping remain preliminary. Their empirical assessment requires longitudinal studies involving organizations with different levels of data and AI governance maturity. Such studies should measure contract-authoring and maintenance effort, validation latency, reviewer workload, provenance overhead, organizational resistance, policy change frequency, incident rates, and user adoption.

Future evaluations should also include synthetic benchmarks, deliberate fault injection, validator-unavailability scenarios, contract-version conflicts, catalog timeouts, compromised tools, and progressive rollout with control and treatment groups. This evidence is necessary to determine when the benefits of stronger agentic controls justify their technical and organizational costs. Until such evidence is available, bounded pilots, explicit ownership, risk-based control selection, reusable artifacts, monitoring, rollback, and mandatory human approval for high-impact actions provide a cautious adoption path.