From Model Explainability to Operational Transparency in Agentic AI: A Transparency-by-Design Framework for Critical Energy Systems Under the EU AI Act

Abstract

Agentic AI can coordinate distributed energy resources, invoke tools, revise plans, and act through multiple interacting agents. In critical energy systems, however, model-level explainable artificial intelligence (XAI) cannot reveal how goals, constraints, tools, delegations, authority, human intervention, and system changes combine to produce an operational action. This creates a regulatory and engineering gap wherever EU AI Act requirements for transparency, record-keeping, and human oversight apply. The transparency-by-design framework addresses this gap by treating operational transparency as a compositional property created through evidence continuity across model, agent, interaction, system, and lifecycle levels. It translates five regulatory transparency functions into eight components, a seven-stage gated lifecycle, stakeholder responsibilities, evidence artefacts, and acceptance criteria. Together, these elements specify what must be transparent, to whom, when, and how adequacy should be assessed and maintained. Regulatory analysis and thematic synthesis of 101 studies provide the evidence base. Application to an agentic virtual power plant demonstrates how forecasts, goals, plan revisions, inter-agent decisions, operator interventions, execution records, and system versions can be joined within one reconstructable evidence chain. The framework extends XAI from local model explanation to lifecycle-wide operational transparency for compliance-oriented development, without claiming legal conformity or field effectiveness.

1. Introduction

Artificial intelligence (AI) is increasingly embedded in modern energy systems [1,2,3]. Cross-sector and power-system reviews document applications across clean energy and power-system operation [4,5], including machine-learning and deep-learning methods for load and renewable-generation forecasting, anomaly detection, and predictive maintenance [6,7,8,9], and reinforcement learning, optimisation, and multi-agent systems for demand response, flexibility management, trading, restoration, and operational decision support [10,11]. These applications respond to increasing operational complexity associated with distributed generation, electrification, storage, flexible demand, electric vehicles, sector coupling, and active consumers [12,13,14]. As AI moves closer to operational control, an insufficiently understood decision can affect system stability, continuity of supply, operational safety, market outcomes, or essential services [15,16,17]. The unresolved problem is therefore no longer only how to explain an individual model output, but how to make the distributed decision process that produces an operational action transparent and reconstructable.

The transition from bounded AI-enabled decision support towards agentic AI increases both capability and operational consequence [18,19]. Conventional systems commonly perform a defined analytical task [5,8] whereas agentic systems may maintain goals and state, plan actions, invoke tools, coordinate with other agents, adapt through feedback, and act with limited human involvement [18,19,20,21]. Energy applications already combine such capabilities for distributed resource coordination, grid control, and flexibility [22,23,24] while related multi-agent and agentic approaches support trading, charging, and restoration [25,26,27,28]. Recent reviews place these functions within wider agentic smart-grid architectures [29].

Agentic operation cannot therefore be understood solely from the input and output of an individual model. An operational outcome may depend on observations, forecasts, intermediate decisions, tool calls, memory retrievals, optimisation steps, communication, delegation, and human intervention. Transparency must reconstruct how the system’s goal, constraints, selected and rejected plans, tool use, inter-agent information, authority, and resulting action combined to produce the outcome.

Explainable artificial intelligence (XAI) has emerged as a principal response to the opacity of complex AI models [15,17]. Existing approaches distinguish between models whose operation is intrinsically interpretable and post hoc methods that seek to explain the behaviour of otherwise opaque models [30,31]. Feature-attribution, surrogate-model, and feature-importance approaches are widely used to interpret energy and sustainability models [32,33,34,35]. Spatial and visual explanations provide complementary forms of interpretation for siting and resource-assessment models [36,37]. Such methods can clarify which variables influenced a prediction, why two results differ, how a model responds to changing inputs, and where the reliability of an output may be limited [17,30].

Intrinsic interpretability may be preferable in high-stakes settings where a sufficiently capable interpretable model can be developed. A post hoc explanation does not necessarily provide a faithful account of the process that generated the output, particularly where the explanation approximates a complex model or is constructed after the decision has been made [31]. This limitation becomes more significant when a model forms only one component of a larger agentic decision process.

The energy-sector XAI literature has grown substantially [15,17]. Reviews have examined explanation techniques across energy and power-system applications and their relationship with governance and trustworthy AI [8,15,17,38]. Other syntheses connect XAI with distributed energy management, reinforcement learning, multi-agent systems, and IoT-edge-cloud architectures [4,5,9,28]. This literature provides an increasingly mature account of model-level explanation methods, application areas, implementation constraints, and evaluation challenges [8,28].

Agentic AI nevertheless creates transparency requirements that extend beyond conventional model-level explainability. The explanation object may include a learned policy, operational goal, planning constraint, action sequence, tool invocation, memory item, coordination process, authority transfer, execution trajectory, or human intervention. A method that explains the contribution of individual input features cannot establish why an agent selected a particular plan, how several agents resolved conflicting objectives, whether an external optimisation tool was used correctly, or why the system escalated a decision to an operator.

Research on explainable reinforcement learning illustrates this difference [39]. A reinforcement-learning agent makes sequential decisions intended to maximise cumulative outcomes rather than producing isolated predictions [39,40]. Policy summaries, reward decomposition, and state importance can explain different parts of the sequential decision process [39,40,41]. Interpretable policy structures provide an account of why an action was selected [42,43]. Emerging energy applications demonstrate the potential of interpretable policy structures, including differentiable decision trees, for providing understandable control rules while retaining adaptive decision-making capabilities [44].

Goal-driven agents introduce additional requirements because behaviour may depend on beliefs, intentions, goals, plans, and plan revisions. An explanation may need to clarify which goal governed the decision, which constraints limited the available actions, why the selected plan was preferred, and why an earlier plan was abandoned [45,46]. Multi-agent systems add a further source of opacity because collective behaviour may emerge from communication, negotiation, and delegation among specialised agents [22,24,47]. Coordination must also account for differences in information, capabilities, authority, and objectives [27,48]. A complete explanation must therefore account for the interaction that produced the system outcome rather than treating each agent as an independent decision-maker [49,50].

Transparency for agentic AI consequently spans several connected levels. Model-level transparency concerns data, assumptions, predictions, feature influence, and uncertainty. Agent-level transparency concerns goals, constraints, plans, policies, tools, memory, and action justification. Interaction-level transparency concerns communication, delegation, negotiation, conflict resolution, and responsibility among agents. System-level transparency concerns the complete execution trajectory, operational state, authority boundaries, human intervention, and relationship between decisions and consequences. Lifecycle transparency concerns intended purpose, design decisions, validation evidence, versioning, deployment configuration, monitoring, incidents, modification, and technical documentation.

Human oversight provides the operational context in which many transparency mechanisms acquire practical value [22,51]. Human involvement supports forecasting, control, fault detection, and decision support in power-system operation [52,53,54]. Effective oversight nevertheless requires more than the formal presence of an operator [22,51,55]. The operator must receive information that supports an appropriate understanding of system capabilities, limitations, uncertainty, and current behaviour [51,56]. The information must also be available at a level of detail and within a timeframe that permits meaningful intervention [22,51,56,57].

An explanation that is technically accurate but delayed, excessively complex, incomplete, or disconnected from operational responsibility may offer limited practical value. Transparency must therefore be evaluated according to its ability to support monitoring, interpretation, challenge, intervention, override, and safe interruption. The required content and timing depend on the stakeholder and the decision context. A developer investigating a model failure requires different information from an operator responding to an imminent network constraint.

The adoption of the EU Artificial Intelligence Act has strengthened the need to translate transparency into operational system requirements [58,59]. The regulation establishes a risk-based framework and identifies certain AI systems intended to be used as safety components in the management or operation of critical infrastructure as potentially high risk [58]. This classification does not apply automatically to all energy-sector AI [59]. The regulatory status depends on the intended purpose, operational function, deployment context, and relationship between the AI system and the management or operation of critical infrastructure [58,59].

For systems falling within the high-risk provisions, transparency forms part of a wider lifecycle structure involving risk management, technical documentation, record-keeping, information provision, human oversight, accuracy, robustness, cybersecurity, quality management, log retention, deployer responsibilities, and post-market monitoring [58]. The regulatory requirements do not define transparency solely as the provision of a local explanation for an individual prediction [58]. The wider objective is to ensure that relevant actors can understand, use, supervise, investigate, and document the system in accordance with their responsibilities [58,59].

The regulatory conception of transparency is therefore broader than conventional XAI [59,60,61]. Developers require information about models, data, system architecture, interfaces, testing, and failure modes [62,63]. Providers require evidence supporting risk management, technical documentation, system instructions, and monitoring [64,65]. Deployers require information about intended use, limitations, operating conditions, configuration, and supervision [61,66]. Energy operators require timely and actionable information concerning current goals, constraints, uncertainty, plans, and available interventions [60,61]. Auditors and regulators require traceable evidence connecting requirements, implementation, validation, operation, incidents, and corrective actions [62,63]. Affected stakeholders may require explanations adapted to the consequences of the system without receiving access to security-sensitive technical information [61,65].

Previous research has established several parts of this problem. Energy XAI reviews have mapped explanation methods and application areas [15,17,28]. Explainable reinforcement-learning research has examined policy and sequential action explanation [39,41]. Explainable-agent research has addressed goals, plans, and action rationales [21,46]. Multi-agent research has examined timely explanations and causal attribution across interacting agents [22,27,49]. Human-in-the-loop research has examined the role of human participation in energy-sector AI [52,55,56]. Agentic smart-grid research has begun to integrate autonomous control, reinforcement learning, multi-agent coordination, and digital twins [20,23,29]. Parallel work examines the safety, trustworthiness, and explainability of generative or agentic approaches in the power sector [18,19,21].

A structured bridge from regulatory transparency functions to the technical and organisational mechanisms required across the lifecycle of an agentic energy system remains underdeveloped [5,8,28]. Existing research provides limited guidance on what must be made transparent at each system level, which stakeholders require which information, how adequacy should be assessed, and which evidence artefacts should preserve the resulting records [5,28].

Without this translation, explainability risks remaining an isolated technical feature added after model development, while regulatory transparency remains an abstract objective without a systematic route to implementation. A development framework must therefore connect regulatory functions, evidence-derived transparency requirements, agentic system levels, stakeholder responsibilities, lifecycle stages, technical mechanisms, evaluation criteria, and documentation artefacts.

The research design combines a separate analysis of the EU AI Act with a focused, PRISMA-ScR-informed selection of scientific studies situated at the intersection of AI, transparency, critical energy systems, and regulation or public policy. The selected evidence is analysed thematically and translated through an explicit evidence-to-framework derivation process. This proportionate design supports framework construction without presenting the scientific search as an exhaustive mapping of every explainability or agentic-AI literature.

Against this gap, the study makes three contributions:

• It derives five regulatory transparency functions from the EU AI Act and relates them to operational transparency needs in critical energy systems.
• It develops a transparency-by-design framework in which operational transparency is treated as a compositional property across five connected system levels, implemented through eight components and a seven-stage gated lifecycle.
• It operationalises the framework through stakeholder responsibilities, linked evidence artefacts, acceptance criteria, and assessment methods, and demonstrates its use in an agentic virtual power plant while preserving the distinction between compliance-oriented engineering and legal conformity.

The remainder of the article is organised as follows. Section 2 defines the research design, regulatory corpus, scientific evidence-selection procedure, analysis, and framework-development method. Section 3 derives the regulatory transparency functions, and Section 4 synthesises the scientific evidence on relevant transparency mechanisms. Section 5 translates the combined evidence into design requirements, while Section 6 formalises the transparency-by-design framework. Section 7 demonstrates its application in a representative energy-system scenario. Section 8 discusses the contribution, implications, and limitations before Section 9 concludes the article.

2. Methodology

2.1. Research Design and Research Questions

The study adopts a review-informed design-science framework-development design. Design-science research addresses relevant problems through the construction and evaluation of purposeful artefacts [67,68]. The primary contribution is a transparency-by-design framework that translates regulatory functions and scientific evidence into requirements, mechanisms, responsibilities, evaluation criteria, and evidence artefacts for agentic AI in critical energy systems. The scientific review provides focused evidence for this construction rather than constituting a standalone scoping review.

Selected PRISMA-ScR reporting elements make database searching, deduplication, export-quality exclusion, title-and-abstract screening, inclusion, and flow reporting transparent [69]. They do not recharacterise the study as a full scoping review, because the review component serves framework development rather than exhaustive evidence mapping.

The regulatory corpus and the scientific corpus perform different epistemic functions. The regulatory corpus establishes formal obligations and regulatory functions, whereas the scientific corpus provides evidence about transparency problems, technical and organisational mechanisms, implementation conditions, evaluation practices, and limitations. The thematic findings are evidence-derived, but the requirements and framework components remain author-constructed design outputs rather than direct statements of the literature or the legal text.

The following research questions guide the regulatory analysis, evidence synthesis, design derivation, and framework construction:

• RQ1. Which transparency-related obligations and operational functions relevant to AI in critical energy systems arise from the EU AI Act?
• RQ2. Which transparency mechanisms for AI in energy-system operation are reported in studies that situate those mechanisms within a regulatory, legal, or public-policy context?
• RQ3. Under which implementation and evaluation conditions have these mechanisms been examined, and which limitations affect their use in critical energy operations?
• RQ4. Which design requirements for agentic AI in critical energy systems can be derived by integrating the regulatory and scientific evidence?
• RQ5. How can the derived requirements be operationalised and assessed through a transparency-by-design framework?

RQ1 establishes the regulatory reference, RQ2 and RQ3 reconstruct the available mechanisms and their evidential conditions, and RQ4 and RQ5 govern the transition from synthesis to framework construction and assessment. The questions define what must be discovered without predetermining the thematic structure or the framework components.

2.2. Regulatory Evidence Base

Regulation (EU) 2024/1689, as amended by Regulation (EU) 2026/1744 and retrieved from EUR-Lex in its final Official Journal form, constitutes the authoritative regulatory corpus [58,70]. The analysis focuses on Articles 9 to 20, 26, 72, and 73 together with Annexes III and IV, because these provisions establish the risk-management, data-governance, documentation, record-keeping, information, human-oversight, performance, organisational, provider, documentation-retention, corrective-action, deployer, monitoring, incident-reporting, and critical-infrastructure conditions relevant to transparency-oriented system design. The amended application timetable provides that Chapter III, Section 1, Section 2 and Section 3, apply from 2 December 2027 to AI systems classified as high risk pursuant to Article 6(2) and Annex III, and from 2 August 2028 to systems classified as high risk pursuant to Article 6(1) [70].

The legal text is analysed separately from the scientific corpus and is not subjected to bibliographic eligibility screening. Consistent with the doctrinal priority given to primary legal sources when establishing the content of law [71], scientific publications inform the interpretation of implementation challenges but do not establish formal obligations. The analysis is not presented as a comprehensive doctrinal legal study, and its purpose is to derive the regulatory functions relevant to engineering design.

Each relevant provision is coded by regulated actor, lifecycle stage, required information or capability, affected system object, oversight purpose, expected evidence artefact, and possible engineering implication. The legal requirement and the author-derived engineering interpretation are retained as distinct fields, which separates what the regulation states from what the framework proposes and avoids presenting the technical translation as a legal determination of compliance.

2.3. Scientific Evidence Identification and Selection

The principal search combines four required concept blocks covering AI-enabled agency, transparency, energy-system operation, and regulation or public policy. Specific agentic and multi-agent terms are combined with the broader expressions artificial intelligence and AI, because relevant energy studies may describe autonomous behaviour without using the newer term agentic AI. The transparency block captures explanation, interpretation, traceability, auditability, provenance, and trust-related terminology, while the energy and regulatory blocks delimit the operational and institutional context. The broader terms increase sensitivity but introduce predictable ambiguities, because polic* may denote a learned reinforcement-learning policy rather than public policy, trust* may refer to generic trustworthiness without a transparency function, and AI may occur as an unrelated abbreviation. These ambiguities are resolved through the operational eligibility rules applied at screening rather than through additional search blocks that could prematurely remove relevant records.

Table 1 records the four blocks and the terms used within each. Every block corresponds to one necessary condition of the review question, so a record is retrieved only when all four conditions are satisfied simultaneously.

Table 1. Concept blocks and terms used in the scientific evidence search.

Concept Block	Methodological Purpose	Search Terms
AI and agency	Identify agentic, autonomous, multi-agent, and broader AI-enabled systems without assuming consistent terminology.	“Agentic AI” OR “AI agent*” OR “multi-agent*” OR “multiagent*” OR “artificial intelligence” OR AI
Transparency and accountability	Retrieve mechanisms and properties supporting understanding, interpretation, verification, reconstruction, audit, and justified trust.	“transparen*” OR “explainab*” OR “trust*” OR “interpretab*” OR “traceab*” OR “auditab*” OR provenance
Energy-system context	Restrict the corpus to energy-system management and operation.	“energy system*” OR “power system*” OR “smart grid*” OR microgrid* OR “electricity grid*” OR “energy management”
Regulation and public policy	Requires engagement with regulation, law, compliance, governance, or public-policy conditions.	“regulation*” OR “law*” OR “polic*”

The search was updated until July 2026 in IEEE Xplore, Scopus, and Web of Science without a publication-year restriction. The four blocks were combined with the Boolean AND operator and applied to titles, abstracts, and author keywords through each database’s supported field syntax. Only field names and Boolean conventions were adapted across databases, while the concept blocks and terms remained unchanged.

Database-specific search expressions, database counts, the final selection audit, and operational exclusion codes are reported in Supplementary Sections S1–S3.

Before title-and-abstract screening, an automated export report identified records lacking an abstract, an author-keyword field, or a digital object identifier. These records were removed because the working procedure required all three fields for consistent processing. Since missing metadata does not establish substantive irrelevance, this operation is reported separately as a pragmatic export-quality filter rather than as an eligibility criterion, and its coverage implications are considered in Section 2.7.

The publication is the eligibility unit. Title-and-abstract screening retained records whose metadata supported the four substantive concept domains and provided extractable framework-relevant evidence. Reports of substantially overlapping work were assessed for distinct evidence rather than treated automatically as independent confirmation. Table 2 defines the operational criteria.

Table 2. Operational eligibility criteria for title-and-abstract screening.

Criterion and Exclusion Code	Include When	Exclude When
E1. Publication status, language, and retrieval	Full peer-reviewed journal article, conference paper, or scholarly review in English.	Non-scholarly material, editorial, abstract-only item, book review, or inaccessible-language record.
E2. AI relevance	The study substantively examines an AI method, AI-enabled system, intelligent agent, autonomous decision process, or multi-agent system.	Exclude records in which AI is an unrelated abbreviation or artificial intelligence is mentioned only generically without an analysed method or system.
E3. Energy-system relevance	The problem, application, data, evaluation, or governance analysis substantively concerns energy-system management or operation, including power systems, grids, microgrids, or energy management.	Exclude studies from non-energy domains and papers that mention energy only as an incidental example or general motivation.
E4. Transparency relevance	The study analyses or implements a mechanism, requirement, evaluation, stakeholder need, or evidence practice related to explainability, interpretability, transparency, traceability, auditability, provenance, or information-supported trust and oversight.	Exclude performance-only studies and work using transparency solely for open data, market information, communications visibility, or generic trustworthiness without an AI-transparency function.
E5. Regulatory or public-policy relevance	Regulation, law, compliance, governance requirements, accountability, formal oversight, or public policy materially shapes the research problem, design, evaluation, or implications.	Exclude papers in which polic* denotes only a reinforcement-learning policy, control policy, or operating policy, and energy-policy studies that do not substantively address AI transparency or governance.
E6. Agentic relevance or transferability	The evidence concerns agentic or multi-agent behaviour directly, or provides a transparency mechanism that can be assigned plausibly to a model, decision, interaction, oversight, trace, or lifecycle function within an agentic energy architecture.	Exclude general AI evidence that cannot be connected to a material component, decision process, human-oversight function, interaction, execution record, or lifecycle requirement of agentic operation.
E7. Extractable evidence	The title, abstract, and metadata contain sufficient detail to extract at least one relevant transparency mechanism, governance implication, operational requirement, or evidence-quality observation.	The available record does not provide reconstructable evidence for the framework.
E8. Distinct contribution	A report contributes evidence not already represented more completely in another report of the same study.	Link companion reports; use the most complete report for shared evidence and retain another report only when it contributes a distinct method, evaluation, or finding.

Borderline decisions preserved the conjunction represented in the search design. An energy-AI study required substantive regulatory, governance, or public-policy relevance together with an extractable transparency requirement, mechanism, oversight function, or evidence implication. Trust-related studies were retained only where trust was connected to explanation, information, uncertainty communication, verification, audit, traceability, provenance, or human oversight, and not where trust was used as an undefined synonym for acceptable performance. The transferability rule in criterion E6 did not displace the energy-system requirement for the principal corpus.

Records were deduplicated by digital object identifier where available and otherwise through normalised title, author, year, and venue comparison. Each screened record received one primary exclusion code corresponding to the first substantive criterion not satisfied. After deduplication and application of the separately reported export-quality filter, the remaining database records underwent title-and-abstract screening against the operational criteria in Table 2.

Additional sources were identified while reading the full texts of the included studies, where a cited work provided primary evidence underlying an eligible review or documented a mechanism that could not be reconstructed sufficiently from the database record. This route was opportunistic rather than a systematic backward and forward citation search. The additional sources were therefore assessed as a separate supplementary evidence stream. Inclusion required compliance with criteria E1, E2, and E4 to E8 together with transferable evidence concerning explainable agency, multi-agent interaction, goal or plan explanation, provenance, or transparency standardisation addressing an agent-level or interaction-level function insufficiently represented in the energy corpus. The scientific corpus therefore comprises 101 studies.

Figure 1 reports the identification, deduplication, export-quality assessment, screening, eligibility, and inclusion outcomes for both identification routes, following the PRISMA 2020 flow structure [72].

Figure 1. PRISMA-ScR-informed flow diagram for the scientific corpus.

2.4. Evidence Extraction, Coding, and Thematic Synthesis

Full texts were retrieved for every record retained through title-and-abstract screening and were used for eligibility confirmation and evidence extraction. Criterion E7 therefore governs the screening decision rather than the depth of the subsequent coding. The publication is the selection unit, while the transparency-relevant evidence instance is the unit of analysis. An instance is an extractable statement or result concerning a regulatory requirement, transparency problem, technical or organisational mechanism, stakeholder information need, implementation condition, evaluation approach, limitation, evidence artefact, or design implication. Each instance was coded for its study and AI-system characteristics, energy and decision context, regulatory concern, transparency object and function, implementation mechanism, stakeholder and lifecycle stage, evidence artefact, evaluation design, criteria and outcome, limitation, and author-stated implication. Evidence status distinguished direct agentic-energy evidence, transferable energy-AI component evidence, transferable adjacent-domain evidence, and contextual regulatory or methodological synthesis. Conceptual and normative claims were recorded separately from empirical evaluations.

Coding followed a hybrid deductive and inductive procedure [73] and recorded the function performed rather than the terminology chosen by the source. The research questions, regulatory coding fields, and distinctions among problems, mechanisms, implementation conditions, evaluations, and limitations provided the initial structure, while inductive codes captured recurring concepts or relations not represented adequately. Regulatory correspondence was assigned only after the scientific evidence instance had been reconstructed, which reduces the risk of forcing a technical mechanism into a predetermined legal category. Ten studies, selected to span the evidence-status categories, were coded independently by both authors, and the resulting differences were used to refine the code definitions. The first author then coded the remaining corpus and the second author reviewed the applied codes, with ambiguous instances resolved through documented discussion. No quantitative agreement statistic is reported, because the double-coded sample is too small to support a stable coefficient, and no reliability claim rests on one.

Thematic analysis compared the coded instances and organised recurring transparency problems, mechanisms, implementation conditions, stakeholder needs, evaluation practices, limitations, and regulatory relations. Codes were divided where they obscured materially different functions and merged where their distinction was terminological. Thematic synthesis then integrated these patterns into higher-order findings by comparing regulatory functions with the scientific evidence, identifying dependencies and gaps, and preserving the distinctions among direct evidence, transferable evidence, contextual synthesis, and author interpretation [74]. The analysis organises patterns, whereas the synthesis produces the findings that inform, but do not predetermine, the subsequent design construction.

The complete extraction and coding fields are reported in Supplementary Section S4.

2.5. Design-Science Framework Construction

Framework construction translates knowledge about the problem domain into a purposeful artefact rather than treating the framework as a direct literature finding [67]. Each regulatory function is compared with the relevant thematic findings, expressed as a design consequence, and reformulated as one or more inspectable requirements. Each requirement is then allocated to a system level, stakeholder, lifecycle stage, framework component or relation, candidate implementation mechanism, evaluation criterion, and evidence artefact.

A framework element is retained only where it addresses a derived requirement and performs a necessary explanatory, governance, design, or assessment function. Candidate components and relations are examined iteratively for regulatory coverage, evidence traceability, terminological consistency, and coherence with the system and lifecycle boundaries. A requirement unsupported by a sufficiently mature mechanism is retained as an unresolved development need rather than represented as a validated solution. The translation chain, evaluation criteria, and evidence-artefact allocations remain author-constructed decisions, which preserves the distinction among literature findings, synthesis findings, derived requirements, and proposed framework elements.

2.6. Scenario-Based Demonstration and Proportionate Evaluation

The framework is demonstrated through a representative agentic virtual-power-plant scenario coordinating photovoltaic generation, battery storage, electric vehicles, flexible building loads, market participation, and distribution-network constraints. The scenario is decomposed into transparency-relevant decision and interaction instances, and for each instance the framework identifies the applicable requirement, stakeholder information need, implementation mechanism, evaluation criterion, intervention possibility, and evidence artefact. This constitutes a design-science demonstration, because it shows how the artefact can structure a representative critical-energy context [68], and the virtual power plant is not treated as a substantive empirical case study.

A separate criterion-based examination provides a limited formative evaluation. Regulatory coverage examines whether the framework addresses the functions derived from the legal corpus. Evidence traceability examines whether each principal component can be followed back through a requirement and thematic finding to its supporting evidence. Structural coherence examines whether components, relations, and allocations are internally consistent. Scenario-based applicability examines whether the framework can structure transparency specification across the representative decision sequence. Such artificial evaluation is appropriate during early artefact development, where its limits are stated explicitly [75]. The evaluation can establish evidence grounding, internal coherence, and scenario-based applicability, but not legal compliance, field effectiveness, transferability across energy systems, generalisability, or empirical validation.

2.7. Methodological Limitations

The focused search does not claim exhaustive coverage of the XAI, autonomous-agent, multi-agent, human-oversight, or the software-observability literatures. Requiring all four concept blocks increases relevance to the regulatory energy-system problem but may omit technical mechanism studies whose searchable metadata do not mention regulation or public policy. Conversely, the broad terms AI, trust*, and polic* increase false matches, which makes the semantic distinctions in Table 2 essential. The export-quality filter adds a further limitation, because records lacking an abstract, an author-keyword field, or a digital object identifier were removed even though missing metadata do not demonstrate irrelevance. Separate reporting makes this loss visible but cannot recover potentially eligible evidence.

The supplementary sources were identified opportunistically during full-text reading rather than through a systematic citation search, so the supplementary stream is not reproducible in the same way the database route is. Database indexing, the English-language restriction, inconsistent use of agentic terminology, evidence-status classification, and regulatory correspondence introduce further judgement. The corpus was coded by one author following a guide calibrated against an independently double-coded sample, with review by the second author, so the classification of evidence status and regulatory correspondence rests on a single coder’s judgement moderated by calibration and review rather than on full independent double coding. Calibration, verification, exclusion coding, and evidence traceability reduce but do not eliminate these limitations.

The framework remains a designed and proportionately evaluated artefact. Its internal coherence and scenario-based applicability do not establish legal conformity or operational effectiveness. The regulatory mapping is an engineering interpretation of the current legal text and may require refinement as harmonised standards, official guidance, implementation practice, and case law develop. Empirical evaluation across implemented systems, stakeholders, and operational contexts is required before stronger claims of effectiveness, transferability, or validation can be made.

3. Regulatory Transparency Requirements for Agentic AI in Critical Energy Systems

3.1. Applicability to Critical Energy Operations

The regulatory analysis begins with the intended purpose of the AI system. The EU AI Act identifies AI systems “intended to be used as safety components in the management and operation” of critical infrastructure as potentially high risk [58]. “Critical energy systems” is used in this article as a domain term rather than as a separate legal classification under the AI Act. Consequently, not every AI application deployed in an energy system is automatically classified as high risk. The classification depends on the system’s intended purpose, operational function, deployment context, and relationship to a safety-related function in the management or operation of critical infrastructure [58,70].

The distinction is important because similar technical components can support materially different purposes. A forecasting model used for research, planning, or non-binding analysis does not perform the same function as an agent that uses the forecast to schedule grid assets, activate flexibility, modify an operating constraint, or issue a control command. An advisory agent also differs from an autonomous controller where an operator retains effective decision authority.

The degree of autonomy does not independently determine regulatory classification, but greater autonomy changes the transparency and oversight mechanisms required to manage the resulting risk. A system that directly executes a physical action requires stronger evidence concerning authority, constraints, execution, and intervention than a system that provides a recommendation for independent human assessment.

Regulation (EU) 2026/1744 also introduced a nomenclature of notified-body competence codes in which agentic AI appears as a distinct horizontal technology code, alongside a sibling code covering systems that learn from their environment other than agentic AI [70]. These codes are administrative designations rather than a legal definition or a separate risk classification. They neither define an agent nor attach obligations to autonomy. Their relevance here is that Union law now recognises agentic AI as a distinct technology class, while the transparency obligations applying to such a system continue to derive from its classification under Article 6(2) and Annex III.

Applicability transparency is therefore the first derived function. Applicability transparency requires a clear description of the intended purpose, operational boundary, critical function, degree of autonomy, authorised actions, human responsibilities, and relationship to other technical safeguards. This information provides the basis for determining which additional transparency requirements apply.

3.2. Information and Interpretive Sufficiency

For high-risk AI systems within the critical-infrastructure scope, the EU AI Act establishes transparency and information-provision requirements rather than a general standalone explainability requirement. Article 13 requires sufficient transparency to enable deployers to interpret system outputs and use them appropriately [58]. Explainability can support this objective, but it operates alongside technical documentation, record-keeping and logging, traceability, human oversight, and lifecycle evidence within the wider high-risk system framework. The required information concerns intended purpose, capabilities, limitations, performance, foreseeable risks, operating conditions, interpretation of outputs, oversight arrangements, and access to relevant records [58]. The separate right to explanation established by Article 86 does not provide the legal basis for the present framework because it expressly excludes the critical-infrastructure systems listed in Annex III, point 2 [58].

For agentic AI, interpretation cannot be limited to the final recommendation or action. A deployer or operator may need to understand the goal pursued by the system, the constraints applied, the information and tools used, the uncertainty affecting the selected plan, and the alternatives considered. A fluent natural-language rationale may be understandable without being faithful to the actual decision process. Interpretive information must therefore remain connected to verifiable system states, records, and model outputs.

The second regulatory function is interpretive sufficiency. Interpretive sufficiency exists when the responsible stakeholder receives accurate, role-appropriate, contextual, and timely information that supports an appropriate understanding of the output or action. Interpretive sufficiency may require explanation content appropriate to the stakeholder and decision context, but it is not reducible to explanation. The function also depends on limitation disclosure, uncertainty communication, operating instructions, system status, and access to relevant evidence.

3.3. Record-Keeping and Reconstructable Operation

The record-keeping provisions require technical capabilities for automatically recording relevant events throughout system operation [58]. The regulatory purpose includes traceability, monitoring, risk identification, post-market monitoring, and investigation of system behaviour [58].

Agentic AI creates a more complex record-keeping problem than a bounded predictive model. Relevant events may include observations, predictions, uncertainty estimates, goal changes, plan generation, plan revision, tool calls, retrieved memory, inter-agent messages, delegated tasks, rejected actions, authority checks, operator interventions, and executed commands. Recording only the final recommendation or command may be insufficient to explain how the system reached the operational outcome.

The third regulatory function is reconstructable operation. Reconstructable operation requires authorised stakeholders to recover the material sequence of information, decisions, interactions, approvals, and actions that produced an outcome. Reconstruction does not require indiscriminate retention of every intermediate computation. The function requires a defined record of material events, their provenance, temporal order, responsible components, authority status, and relationship to the resulting action.

3.4. Human Oversight and Intervention

The human oversight provisions require a system design that enables effective supervision during use [58]. Responsible personnel must be able to understand system capacities and limitations, monitor operation, recognise anomalies, interpret outputs, avoid inappropriate reliance on automation, disregard or override an output, and interrupt the system where necessary [58].

Effective oversight requires more than access to information. The information must support a judgement within the available operational time. A comprehensive explanation delivered after a control deadline cannot support intervention. An interface presenting large volumes of undifferentiated detail may also weaken oversight by obscuring the material issue.

The fourth regulatory function is intervention-enabling transparency. The function requires information and controls that support recognition of a situation requiring intervention, assessment of the likely consequences, selection of an appropriate response, and exercise of the assigned authority. Relevant capabilities include the ability to inspect or challenge a recommendation, pause or constrain autonomous execution, select an alternative plan, override a decision, and initiate safe fallback operation.

3.5. Lifecycle Evidence Continuity

Regulatory transparency is embedded in a wider lifecycle structure involving risk management, technical documentation, quality management, accuracy, robustness, cybersecurity, deployer responsibilities, log retention, and post-market monitoring. Runtime explanations cannot recover undocumented design assumptions, validation decisions, or system changes. Technical documentation cannot reconstruct an operational event that was not recorded.

The obligations governing the retention of technical documentation and automatically generated logs, the corrective action required where a system no longer conforms, and the reporting of serious incidents extend this structure beyond the point of deployment [58]. Evidence produced during development therefore acquires a continuing regulatory function, because it must remain retrievable and interpretable at the later moment when an incident, a corrective action, or a market-surveillance request requires it.

The fifth regulatory function is lifecycle evidence continuity. Lifecycle evidence continuity requires traceable relationships among intended purpose, requirements, architecture, system configuration, validation results, known limitations, operating instructions, runtime records, incidents, corrective actions, and subsequent modifications.

Lifecycle evidence is especially important for adaptive and configurable agentic systems. A change to a model, tool, prompt, memory source, agent role, authority rule, or operating constraint may alter the validity of an earlier explanation mechanism or oversight arrangement. The evidence structure must therefore show which version was active, which assumptions applied, and whether a change required renewed assessment.

3.6. Consolidated Regulatory Function Model

The regulatory analysis produces five connected functions that define the transparency objectives of the framework. Table 3 consolidates the five regulatory transparency functions. Each function is expressed as an engineering interpretation together with the evidence artefacts through which fulfilment can be demonstrated, rather than as a restatement of the legal text.

Table 3. Regulatory transparency functions and engineering implications.

Regulatory Function	Engineering Interpretation	Representative Evidence Artefacts
Applicability transparency	Define intended purpose, critical function, autonomy, system boundary, authorised actions, and human responsibilities	Intended-purpose statement, system-boundary description, authority model
Interpretive sufficiency	Provide accurate, contextual, role-specific, and timely information supporting appropriate interpretation and use	Instructions for use, explanation specification, limitation record, uncertainty record
Reconstructable operation	Preserve the material sequence from information acquisition to operational action	Execution trace, event log, tool-use record, interaction record
Intervention-enabling transparency	Support monitoring, challenge, override, interruption, and safe fallback	Operator interface, intervention controls, escalation specification, intervention record
Lifecycle evidence continuity	Connect intended purpose, requirements, design, validation, deployment, operation, incidents, and modifications	Technical documentation, validation report, version record, incident report, monitoring record

A regulatory function cannot normally be satisfied by one technical method. Interpretive sufficiency may require model explanations, plan explanations, uncertainty information, documentation, and interface controls. Reconstructable operation requires both technical instrumentation and organisational retention arrangements. Effective oversight requires technical functions, assigned responsibility, operator competence, and operational authority. The regulatory function model therefore defines what must be supported without assuming one universal implementation.

4. Explainability and Transparency Mechanisms for Agentic AI

4.1. Structure of the Evidence Base

Evidence is analysed as transparency-mechanism instances rather than publication-level categories. A source may contribute several instances addressing different transparency objects, stakeholders, or lifecycle functions. This preserves operational detail while permitting cross-domain comparison. Because contributions can span several levels, level counts are non-exclusive and are not used as evidence-maturity scores.

The evidence spans complementary traditions. Energy-sector XAI concentrates on forecasts, classifications, anomaly detection, and optimisation [33,38,76,77], while application studies extend interpretable methods to regulation-aware decision support, regulatory-change analysis, and industrial energy prediction [35,78,79]. Explainable reinforcement learning addresses policies and sequential actions [39,41], explainable-agent research addresses goals, plans, tools, memory, and rationales [19,46], multi-agent explainability addresses communication, delegation, and responsibility [22,27,49], and autonomous-systems transparency extends to stakeholder information, traceability, intervention, and lifecycle assurance [15,17,80].

Direct energy evidence is strongest at the model level and growing for reinforcement-learning and multi-agent applications, while memory provenance, tool-use traceability, cross-agent responsibility, and lifecycle transparency remain less mature. The synthesis therefore distinguishes direct energy evidence, transferable empirical or methodological evidence, conceptual proposals, and mechanisms lacking operational evaluation.

4.2. Model-Level Explainability

Model-level explainability connects data, features, assumptions, and internal representations to a prediction or estimate. Intrinsically interpretable approaches include decision trees, rule sets, and symbolic constraints [31,81,82], with energy applications using rule extraction and interpretable reinforcement-learning structures [40,41,42]. Post hoc methods include feature attribution, feature importance, and local surrogates [32,33,34,35], complemented by spatial and visual explanations [36,37]. Reviews identify corresponding strengths and limitations [17,30,83].

These mechanisms remain directly relevant because agentic energy systems depend on forecasts of demand, generation, prices, congestion, equipment state, and flexibility [76,79,84]. Operators may therefore need to understand why a forecast or alert changed and how uncertainty affects the resulting plan [51,85].

Model-level explanations remain bounded: attribution can explain a forecast change but not why an agent selected, delegated, or revised an operational action. Model evidence must therefore remain linked to the subsequent decision process.

Uncertainty is part of that link. Critical energy decisions depend on uncertain weather, markets, asset availability, and user response; explanations should therefore expose material uncertainty and its effect on the selected action rather than present influential variables as if the inputs were certain.

4.3. Agent-Level Transparency

Agent-level transparency concerns goal-directed behaviour, including current goals, constraints, state, plans, expected outcomes, rejected alternatives, reward structure, tools, memory, and reasons for revision.

Explainable reinforcement-learning methods clarify policies, rewards, and state importance [39,40,41] while interpretable policy structures show how actions are selected [42,43]. Differentiable decision trees demonstrate that adaptive energy-management policies can retain inspectable decision structures [44].

Goal and plan explanations must state what the agent sought to achieve and why the action formed part of the selected plan [45,46]. In energy operation, cost, congestion, reserves, comfort, asset protection, and resilience can conflict, making the active priority material to interpretation.

Tool-use transparency is required when an agent invokes an optimisation engine, digital twin, or forecasting service [20,23,86], and likewise for knowledge bases, market interfaces, or control APIs [19,21,22,87]. Relevant evidence records the selected tool, inputs, returned output, warnings or errors, and its influence on the plan; a rationale without tool provenance remains unverifiable.

Memory transparency concerns retained or retrieved information that affects action. Its provenance and currency matter because a plan may depend on a current measurement, earlier instruction, operating rule, or retrieved document that has become outdated.

4.4. Interaction-Level Transparency

Multi-agent systems distribute perception, decision-making, and action across specialised components [22,24,47], including related distributed-control structures in microgrids and secure coordination [20,27,48,88]. Energy architectures may combine asset, market, forecasting, network, cybersecurity, digital-twin, and coordinating agents [28,29], so decisions can emerge from interaction rather than from one identifiable component.

Interaction-level transparency records which agents participated, what information was exchanged, what task or authority was delegated, how conflicts were resolved, and which contributions affected the outcome. Independent agent explanations are insufficient because correct local outputs can still be combined incorrectly; the evidence must preserve the relations that produced the collective result.

Causal explanations support attribution in sequential multi-agent environments [27], while counterfactual simulation can identify which event or agent materially changed the outcome [50]. Such mechanisms are transferable to energy systems where one agent changes the feasible actions of others.

Interaction transparency also requires explicit authority [22,57]. Communication alone does not establish decision ownership or permitted delegation; the evidence must connect communication, authority, responsibility, and execution.

4.5. System-Level Transparency and Execution Traceability

System-level transparency combines component evidence into the operational trajectory, including system state, plan transitions, agent decisions, operator interventions, commands, and resulting physical or market consequences.

Execution traceability is central at this level [86,89]. Provenance can be preserved through tamper-evident operational records [47,90], and cross-system energy architectures create corresponding traceability needs [14]. A useful trace links material inputs, models, tools, agents, messages, decisions, authorisations, actions, and outcomes in temporal order without retaining irrelevant internal activity.

Observability exposes state, performance, and events; transparency adds the relations among goals, decisions, authority, and consequences. A battery-discharge event therefore becomes transparent only when it can be connected to the governing plan, triggering conditions, authorised operating envelope, and approval or execution rule.

System-level evidence must also distinguish model output, agent proposal, operator approval, and executed command because these stages allocate responsibility differently during operation and incident analysis.

4.6. Lifecycle Transparency

Runtime explanations cannot recover undocumented design assumptions or changes [62,86]. Lifecycle transparency therefore links operation to intended purpose, requirements, architecture, validation, deployment, monitoring, modification, and retirement [63,89], together with the artefacts needed to identify the active configuration and its validated assumptions.

Lifecycle transparency also requires change traceability [62,63,86]. Model, prompt, tool, agent-role, access-right, goal, or constraint changes can invalidate earlier explanations or training, so affected requirements and validation evidence must be identifiable.

Permitted adaptation within a validated operating envelope must remain distinguishable from material change to tools, authority, agent roles, or other assumptions that alter behaviour or risk.

4.7. Human–Machine Interfaces

Transparency must be communicated through an interface suited to the stakeholder and task [22,51,52] consistent with power-system evidence linking information provision to operational responsibility [53,54,55]. Technical completeness is insufficient if detail, terminology, or presentation prevents timely interpretation.

Operator interfaces should prioritise current goals, constraints, uncertainty, anomalies, selected plans, expected consequences, and available interventions, while allowing progressive access to greater detail.

Explanation depth must also match decision time: day-ahead planning permits deliberation that an intervention required within seconds does not.

Human-centred evaluation must test whether stakeholders understand and can use the information for the required task [22,51]. Satisfaction or trust alone is insufficient [30,60]; relevant measures include comprehension, decision quality, cognitive burden, response time, and recognition of system limitations [56,85].

4.8. Evaluation of Transparency Mechanisms

Transparency mechanisms must be evaluated against the function that each mechanism is intended to support [51,60,85]. No single metric can establish transparency alone.

Technical evaluation concerns fidelity, completeness, stability, robustness, reproducibility, trace coverage, uncertainty calibration, and latency. These criteria test whether explanations reflect the decision process, include material factors, behave consistently in comparable cases, and preserve the required event record.

Human-centred evaluation concerns comprehensibility, cognitive burden, relevance, and mental-model formation, while operational evaluation concerns actionability, anomaly recognition, intervention support, and response time.

Governance evaluation concerns evidence continuity, responsibility allocation, access control, and auditability, including clear ownership of implementation, verification, operation, and maintenance.

Table 4 summarises the identified mechanisms together with the principal criterion through which each mechanism can support an acceptance decision.

Table 4. Transparency mechanisms and principal evaluation criteria.

System Level	Transparency Objects	Candidate Mechanisms	Principal Evaluation Criteria
Model	Data, prediction, feature influence, uncertainty	Interpretable models, attribution, counterfactuals, causal explanation, uncertainty intervals	Fidelity, stability, calibration, comprehensibility
Agent	Goal, constraint, policy, plan, tool, memory, action	Goal and plan explanation, policy summary, reward decomposition, interpretable policy, provenance	Faithfulness, completeness, relevance, latency
Interaction	Communication, delegation, conflict, responsibility	Interaction trace, causal attribution, authority record, contribution analysis	Causal relevance, responsibility clarity, scalability
System	State, trajectory, approval, execution, outcome	Event log, execution trace, provenance graph, intervention record, operational interface	Trace coverage, reproducibility, timeliness, auditability
Lifecycle	Intended purpose, validation, deployment, change, incident	Technical documentation, validation record, version history, change-impact analysis	Completeness, consistency, currency, evidence continuity

4.9. Implementation Limitations and Evidence Gaps

Evidence remains concentrated on model outputs: energy XAI supports forecasts, classifications, and anomaly detection [15,17,38] but rarely captures multi-step planning, delegation, tool or memory use, and agent interaction [5,8,28].

Explanation fidelity remains a second limitation because generated rationales can be understandable without reflecting the process that produced the action [18,91]. Power-sector LLM research identifies related safety and governance risks [19,21,87,92], which makes execution records, structured agent states, causal analysis, or interpretable decision structures necessary anchors [86].

Operational validation also remains limited. Many mechanisms are tested in simulations or general user studies [77,93], with less evidence on control-room cognitive load, explanation latency, intervention quality, and integration with operating procedures [22,51].

Evidence discontinuity persists because model explanations, agent logs, communication records, and interfaces are often developed separately [12,14,23,94] while lifecycle records are not automatically connected to runtime traces [62,63,86,95]. This fragmentation prevents end-to-end reconstruction from information to collective decision and action.

Transparency itself can expose personal or commercially sensitive information [43,76,92,96] and security-relevant procedures or attack surfaces [90,97,98,99,100]. Disclosure must therefore remain role-specific, access-controlled, and proportionate to stakeholder responsibility [60,61,63,65].

Finally, transparency is often specified without measurable acceptance criteria [60,85]. Framework requirements must therefore be testable, reviewable, monitorable, and maintainable rather than remain aspirational.

5. Evidence-Derived Transparency Requirements and Framework Derivation

5.1. Stakeholder-Specific Requirements

Transparency is relational because adequacy depends on stakeholder responsibility and decision context. One common information view cannot satisfy every role.

Developers require detailed information concerning models, agents, data, tools, interfaces, execution behaviour, and failure modes [62,63,86]. Providers require evidence connecting intended purpose, risk management, system design, validation, instructions, and monitoring [16,60,64]. Deployers require information supporting configuration, supervision, operating procedures, training, and incident response [61,65]. Operators require concise and timely information concerning current goals, constraints, uncertainty, anomalies, plans, and intervention options [22,51,52,54]. Assurance personnel require traceable relationships among requirements, implementation, tests, operational records, incidents, and corrective actions [62,63,86].

Affected stakeholders may require explanations concerning outcomes but should not necessarily receive access to technical, personal, or security-sensitive information [101,102,103,104]. Stakeholder-specific transparency therefore requires controlled views over a common evidence base rather than separate and potentially inconsistent explanations.

5.2. Lifecycle Requirements

Lifecycle transparency begins with intended-purpose definition and continues through requirements, architecture, validation, deployment, operation, incident analysis, and modification. The organisation must define the system purpose, material decisions and actions, operating boundaries, human responsibilities, stakeholders, available decision time, intervention requirements, and records needed for reconstruction; architecture and development must then allocate the required transparency mechanisms to models, agents, interactions, interfaces, and documentation processes.

Verification and validation must test fidelity, trace coverage, latency, comprehensibility, uncertainty communication, and intervention support [60,63,85]. Deployment must configure access rights, logging, escalation, fallback, and monitoring [58,62]. Operation must generate the specified evidence and present role-appropriate information [22]. Incident analysis must reconstruct the decision trajectory and identify contributing factors [86,89]. Modification must assess the effect of the change on explanation validity, oversight arrangements, and retained evidence [62,63].

5.3. Cross-Level Transparency Requirements

Cross-level transparency requires evidence continuity from material information through agent reasoning, inter-agent interaction, execution, and lifecycle context. The requirement is not that every level provide the same explanation, but that the evidence produced at one level can be related to the decisions and records at the next.

A transparency break occurs where evidence cannot be followed between adjacent levels. A model may provide an explanation, but the system may not record whether the agent relied on the explained output. An agent may record a selected plan, but the system may not show which delegated component executed the plan. An execution trace may identify an action, but the technical documentation may not define whether the action was authorised.

Cross-level traceability is therefore a core requirement. Each material operational decision should connect its inputs, models, goals, and constraints within a reconstructable evidence chain [62,86,89]. The chain should also preserve interactions, authority, actions, and outcomes [48,86].

5.4. Transparency-by-Design Principles

The thematic synthesis produces nine design principles: (1) Transparency should be proportionate to system autonomy, operational consequence, and time sensitivity. (2) Explanations should be grounded in the actual decision and execution process. (3) Evidence should remain traceable across model, agent, interaction, system, and lifecycle levels. (4) Information should reflect stakeholder responsibility and decision context. (5) Uncertainty and limitations should remain visible throughout the decision process. (6) Transparency should support intervention rather than only retrospective understanding. (7) Agent authority, delegation, and responsibility should be explicit. (8) Transparency mechanisms should protect security, privacy, and commercially sensitive information. (9) Evidence should remain continuous across development, deployment, operation, monitoring, incident analysis, and modification.

The principles define required properties without prescribing one universal algorithm or architecture. The implementation depends on the system purpose, operational context, risk profile, and stakeholder responsibilities.

5.5. Evidence-to-Framework Derivation

The framework derivation connects each regulatory function with coded transparency mechanism instances and the constructive themes obtained through thematic synthesis. The derivation follows a fixed translation chain from regulatory function to transparency requirement, system level, stakeholder, lifecycle stage, implementation mechanism, evaluation criterion, and evidence artefact.

The derivation is reported below for each regulatory function. Each paragraph states the synthesis finding, the design consequence, the requirement expressed in inspectable language, and the framework element that carries it. Table 5 provides a compact traceability summary of the five derivations, while implementation mechanisms, acceptance criteria, evidence artefacts, and accountability are consolidated in the component register in Table 7.

Table 5. Representative evidence-to-framework derivation.

Regulatory Function	Evidence-Derived Requirement	System Level	Supporting Evidence
Applicability transparency	Define system purpose, boundary, autonomy, authority, and critical function	System and lifecycle	European Parliament and Council of the European Union [58]; Winfield et al. [80]; Arslan & Sibilla [19]
Interpretive sufficiency	Explain the basis, limitations, and uncertainty of material recommendations	Model and agent	Miller [30]; Rudin [31]; Machlev et al. [17]; Milani et al. [39]
Reconstructable operation	Preserve the sequence from information acquisition to execution	Agent, interaction, and system	Alzetta et al. [49]; Sado et al. [46]; Gunasekara et al. [86]; Tightiz et al. [90]
Intervention-enabling transparency	Provide timely information and controls supporting challenge and override	System	Anderson et al. [51]; Cai et al. [22]; Guo et al. [52]; Nguyen & Ahn [56]
Lifecycle evidence continuity	Connect purpose, requirements, design, validation, operation, and change	Lifecycle	El-Haber et al. [63]; Jørgensen & Ma [62]; Benitez [66]

Applicability transparency derives from the finding that transparency obligations attach to a system whose purpose, boundary, autonomy, and authority have been declared, whereas agentic systems frequently lack such a declaration because capability is distributed across models, tools, and coordinating agents [18,19]. This creates a requirement to define the system before any further transparency claim can be assessed, because the stakeholder-dependence of transparency cannot be resolved without a declared boundary and an identified authority [80]. The framework addresses the requirement through an intended-purpose model, an authority map, and an autonomy classification at the system and lifecycle levels, assessed through boundary clarity and responsibility clarity, and evidenced by an intended-purpose statement.

Interpretive sufficiency derives from the finding that energy explainability concentrates on model-level attribution and that attribution can be persuasive without being faithful to the process that produced an action [17,30]. This creates a requirement that explanation be assessed for fidelity rather than for perceived plausibility, because an explanation that is understandable but unfaithful transfers risk to the operator rather than reducing it [31]. The framework addresses the requirement through model explanation, uncertainty communication, and goal and plan explanation at the model and agent levels, assessed through fidelity, stability, and calibration, and evidenced by an explanation record.

Reconstructable operation derives from the finding that agent-level and interaction-level evidence remains less mature than model-level evidence, and that reconstructing an event requires records of information acquisition, tool invocation, and inter-agent exchange rather than an explanation of a single prediction [46,49]. This creates a requirement to preserve the execution sequence, because an explanation generated after an event cannot substitute for a record of what occurred. The framework addresses the requirement through execution traces, tool provenance, and interaction logs at the agent, interaction, and system levels, assessed through trace coverage, temporal consistency, and reproducibility, and evidenced by an operational trace. Ledger-based provenance studies demonstrate that such records can be preserved in tamper-evident form [86,90].

Intervention-enabling transparency derives from the finding that oversight requires more than the formal presence of an operator, and that information must arrive at a level of detail and within a timeframe that permits intervention [22,51]. This creates a requirement that transparency be evaluated against the operator task rather than against the explanation alone, because comprehension without actionable control does not constitute oversight [56]. The framework addresses the requirement through a prioritised operator view, alternatives, alerts, and intervention controls at the system level, assessed through latency, actionability, and intervention success, and evidenced by interface validation and intervention records.

Lifecycle evidence continuity derives from the finding that lifecycle and organisational records form an evidence structure that is not automatically connected to runtime traces [62,63]. This creates a requirement to connect purpose, requirements, design, validation, operation, and change, because evidence produced at one lifecycle stage loses its value if it cannot be related to the system state that produced an incident. The framework addresses the requirement through a traceability matrix, version control, and change-impact assessment at the lifecycle level, assessed through completeness, consistency, and currency, and evidenced by technical documentation and a change record. Regulatory and lifecycle studies support integrating this connection before deployment rather than retrospectively [66].

Two elements of the derivation rest on design judgement rather than on retrieved evidence. The assignment of evaluation criteria to requirements and the allocation of evidence artefacts to lifecycle stages are author-derived constructs introduced to make the framework inspectable. They are consistent with the evidence but are not established by it, and they should be assessed as design decisions rather than as findings.

5.6. Evidence Strength and Transferability

Figure 2 uses three ordinal evidence-maturity categories. Mature energy-sector evidence denotes mechanisms supported by multiple directly relevant energy studies together with empirical or operational evaluation. Transferable or growing evidence denotes mechanisms with limited direct energy evaluation but stronger support from adjacent autonomous-system domains or a developing energy evidence base. Emerging evidence denotes mechanisms supported mainly by conceptual, methodological, or early-stage studies with limited operational evaluation. The classification concerns the maturity of evidence for a mechanism rather than the necessity of the transparency requirement and is not a formal study-quality score. Supplementary Sections S5 and S6 report the classification rule and a non-exclusive descriptive profile of sources cited in the five level-specific synthesis sections.

Figure 2. Multi-level transparency architecture mapped to the regulatory transparency functions. Each populated cell identifies the mechanism through which the function is realised at that level. Shading records the ordinal maturity of supporting evidence defined in Section 5.6 rather than the number of publications. B1–B4 mark representative cross-level transparency breaks: B1, model output not linked to the agent plan; B2, agent decision or delegation not linked to interaction authority; B3, collective decision not linked to the executed command; and B4, runtime event not linked to the active lifecycle version.

Framework components differ in evidence maturity. Model-level XAI mechanisms have comparatively mature energy-sector evidence [17,33,38]. Explainable reinforcement-learning applications provide growing evidence for interpretable energy-management policies [40,41,42,43]. Review evidence similarly documents the expanding use of deep reinforcement learning in energy management [105]. Direct validation for critical grid operation nevertheless remains less mature [44,106].

Goal and plan explanations have established conceptual evidence in adjacent autonomous-system domains [46,49]. Causal multi-agent explanations provide related evidence for attributing outcomes across interacting agents [27,50]. Human–machine power-system research demonstrates the operational relevance of interaction and oversight [22,52,54,55]. Direct validation of these mechanisms in energy control remains limited. Tool-use, memory, and agent-workflow provenance remain emerging areas [21,86]. Lifecycle evidence mechanisms draw on regulatory interpretation, autonomous-systems transparency, software engineering, and established assurance practices [62,63,80].

The framework therefore distinguishes the necessity of a requirement from the maturity of the mechanism. A regulatory or operational requirement may be well established even where the available implementation remains immature. Such cases define research priorities rather than established solutions.

6. Transparency-by-Design Framework

6.1. Purpose, Scope, and Boundaries

The transparency-by-design framework supports the specification, implementation, evaluation, documentation, and lifecycle maintenance of transparency requirements for agentic AI in critical energy systems. It translates regulatory functions and evidence-derived requirements into assessable system properties, development activities, mechanisms, responsibilities, and evidence artefacts for providers, developers, deployers, energy operators, assurance personnel, and other responsible stakeholders.

Application of the framework assumes three conditions. The agentic system must have a definable operational boundary within which its material decisions and actions can be identified. At least one human role must hold intervention authority over those actions. The operating organisation must be able to generate and retain the specified records for the period required by its regulatory and operational obligations. Where any of these conditions does not hold, the framework identifies the resulting deficiency but cannot compensate for it.

Four boundaries delimit what the framework provides. It does not determine legal conformity, because conformity assessment also depends on system classification, risk management, and quality management arrangements outside the transparency function. It does not prescribe a particular algorithm, explanation method, or system architecture, because the appropriate mechanism depends on the decision architecture, model class, time horizon, and operating context. It does not prescribe universal numerical thresholds, because defensible values require application-specific operational evidence; Section 6.6 instead specifies how such thresholds should be set and justified. It does not replace data governance, cybersecurity assessment, or privacy management, each of which constrains which evidence may be generated and disclosed. The framework therefore specifies what must be made transparent, to whom, on what evidence, and through what assessment, while implementation technology and legal conformity remain responsibilities of the relevant organisations and competent authorities.

6.2. Multi-Level Transparency Architecture

The framework comprises five connected levels. The model level covers data provenance, model assumptions, predictions, feature influence, uncertainty, and known limitations. The agent level covers goals, constraints, state, plans, policies, tool and memory use, action selection, and plan revision. The interaction level covers communication, delegation, negotiation, conflict resolution, contribution, and responsibility among agents. The system level covers the operational trajectory, authority boundaries, human interaction, execution, physical consequences, and safe fallback. The lifecycle level connects intended purpose, requirements, architecture, validation, deployment, monitoring, incidents, changes, and retirement. Together, the levels locate the evidence needed to interpret model information, explain agent behaviour, reconstruct collective decisions, relate them to executed outcomes, and maintain validity over time.

Common identifiers connect the five levels into one evidence chain. For each material operational decision, the chain links the originating data or observation to the relevant model output, the agent goal and active constraints, the selected plan and material tool or memory use, inter-agent contribution and delegation, approval or authority state, execution, observed outcome, and active system version or configuration. Component C8 governs identifier uniqueness, the completeness of relations between adjacent levels, version linkage, and detection of missing relations. A transparency break is therefore observable when, for example, a model output cannot be linked to the plan that used it, a delegation lacks an authority relation, or an executed action cannot be linked to the active system version.

Figure 2 maps the five levels against the five regulatory transparency functions derived in Section 3. Each regulatory function is realised through mechanisms at one or more specific levels rather than through explanation at a single point. The shading represents the ordinal maturity of supporting evidence defined in Section 5.6 rather than publication counts, while the common-identifier relation beneath the architecture represents the continuity whose absence constitutes a transparency break.

6.3. Transparency-by-Design Lifecycle

The multi-level architecture defines where transparency must exist. The lifecycle defines when each transparency obligation is created, discharged, and revisited. The two dimensions remain distinct, because a mechanism allocated to a system level acquires operational meaning only once it has been specified, implemented, accepted, instrumented, monitored, and reassessed. Transparency is developed and maintained through seven lifecycle stages, denoted L1 to L7, which form a specification phase, a construction and qualification phase, and an operation and maintenance phase.

The specification phase establishes what must be made transparent. Stage L1 defines the intended purpose, operational boundary, criticality, decision architecture, autonomy, and authority of the system, and produces the intended-purpose statement and authority model carried by component C1. The decision architecture records whether material agents or decision components are rule-based, optimisation-based, reinforcement-learning, LLM-based, or hybrid where applicable. Architecture determines which transparency mechanisms can meaningfully expose the decision process, whereas autonomy and authority determine the required depth, latency, oversight, and evidence. Stage L2 identifies the material transparency objects, the stakeholders whose decisions depend on them, the oversight decisions those stakeholders must be able to take, and the records required to reconstruct the resulting operation. Its transparency requirements specification fixes the required depth, stakeholder view, evidence-retention condition, and acceptance criterion for the remaining components, so an omission at L2 propagates as an absent requirement rather than a failed one.

The construction and qualification phase establishes how the specified transparency is realised and whether it is adequate. Stage L3 allocates transparency mechanisms to models, agents, interactions, interfaces, and documentation processes, and produces the explanation specification together with the mechanism allocation recorded in the component register. Stage L4 defines the acceptance criteria and the validation procedure through which each allocated mechanism is assessed, and produces the validation plan and the validation report. The separation of the two stages is deliberate. Allocating a mechanism establishes an intention, whereas defining and applying an acceptance criterion establishes whether the intention was met, and a framework that combines the two cannot distinguish an implemented mechanism from an adequate one.

The operation and maintenance phase establishes whether the accepted transparency continues to hold. Stage L5 configures runtime instrumentation, access control, escalation, and fallback, so that the evidence assumed by the acceptance criteria is generated and retained in the deployed configuration. Stage L6 monitors explanation quality, trace completeness, operator use, incidents, and system changes, and produces the monitoring and incident records through which degradation becomes visible. Stage L7 reassesses the transparency requirements whenever the system, its models, its authority allocation, or its operating context changes. Because agentic systems are frequently modified through model replacement, tool addition, and policy revision, stage L7 governs whether a previously accepted explanation still describes the system that is actually operating.

Each stage concludes with a readiness gate. Gate G4 accepts, conditionally accepts, or rejects an allocated mechanism; G5 separately checks whether the deployed configuration can generate, protect, and retain the evidence on which that acceptance depends; and G7 determines whether change has invalidated continued acceptance. A negative decision returns to the earliest stage able to resolve the deficiency. Conditional progression is permitted only where the deficiency is bounded, a compensating measure and re-evaluation date are recorded, and no mandatory legal requirement, unacceptable safety risk, or non-compensable human-oversight capability is waived.

The lifecycle is iterative because validation, deployment, monitoring, or system change can expose a deficiency that invalidates earlier assumptions. Each stage therefore produces both evidence and a readiness decision, so transparency is maintained as an operating-system capability rather than treated as a one-time validation result.

Figure 3 makes the rework logic explicit. Deficiencies in purpose, boundary, autonomy, or authority return to L1; requirement gaps to L2; mechanism inadequacy or operational degradation to L3; deployment-instrumentation deficiencies to L5; and material system changes to L1. Rework therefore starts at the earliest stage able to correct the identified deficiency, rather than restarting the full lifecycle.

Figure 3. Transparency-by-design lifecycle. Each stage produces a defined evidence artefact and concludes with a readiness gate. The labelled rework triggers distinguish mechanism inadequacy at G4 (R1, return to L3), deployment-instrumentation deficiency at G5 (R2, repeat L5), operational degradation at G6 (R3, return to L3), and material change at G7 (R4, return to L1). Specification deficiencies at G1 and G2 repeat L1 or L2, respectively. Component identifiers refer to the framework component register.

6.4. Stakeholder Responsibility Model

The provider defines the intended purpose, system boundary, transparency requirements, and evidence architecture. Developers implement and test the required mechanisms. Deployers configure the system for the operational context, assign oversight responsibilities, and ensure that users receive appropriate information and training. Operators monitor system behaviour and exercise the assigned intervention authority. Assurance personnel assess evidence traceability, coverage, and consistency. Incident-response personnel reconstruct events and coordinate corrective action.

Consistent with the stakeholder-specific requirements in Section 5.1, these roles use controlled views over the same evidence base rather than separate explanation records. Assurance and regulatory personnel require end-to-end traceability and lifecycle evidence; deployers and operators require configuration, current system state, uncertainty, authority, and intervention information; affected end users require outcome-relevant explanations without unnecessary access to technical, personal, commercially sensitive, or security-sensitive records. Access control therefore limits disclosure while preserving one underlying evidence chain.

Responsibility may be distributed across organisations where models, software platforms, optimisation services, data services, and control systems have different suppliers. Each component therefore receives four responsibility types: one accountable role, one or more producing roles, one verifying role, and one or more using roles. A missing accountable or verifying role constitutes a transparency gap even where the technical mechanism exists.

Providers, developers, and deployers primarily specify, produce, and configure transparency artefacts, whereas operators, assurance personnel, and incident-response personnel primarily verify or use them in operational, assurance, or investigative decisions. This distinction keeps component ownership separate from decisions made on the resulting evidence.

Table 6 converts the responsibility allocation into a completeness test. Every applied component must have one accountable role, a verifying role, and at least one using role; a missing assignment identifies a deployment deficiency rather than an unowned transparency obligation.

Table 6. Responsibility allocation for the framework components. ‘A’ denotes the role accountable for the component being specified, adequate, and maintained. ‘P’ denotes a role that produces or specifies its content. ‘V’ denotes the role that verifies it against its acceptance criterion. ‘U’ denotes a role that uses the resulting evidence in an operational, assurance, or investigative decision.

Component	Provider	Developer	Deployer	Operator	Assurance Personnel	Incident-Response Personnel
C1 System definition and authority map	A	P	P	U	V	U
C2 Data and variable influence disclosure	P	A	–	U	V	U
C3 Objective and constraint disclosure	P	A	–	U	V	U
C4 Plan and action justification	P	A	–	U	V	U
C5 Contribution and delegation provenance	A	P	P	U	V	U
C6 Execution and authority record	P	P	A	P, U	V	U
C7 Operator interface and intervention controls	P	P	A	P, U	V	U
C8 Lifecycle evidence continuity	A	P	P	U	V	P, U

Where responsibility is distributed across organisations, the allocation must be reproduced in the contractual arrangements governing the supply of foundation models, software platforms, optimisation services, data services, and control systems. A supplier that produces a component without contractually providing the information, access, and evidence required for independent verification transfers an unverifiable artefact to the deployer, who then cannot discharge the accountability that the deployment context assigns. Responsibility allocation is therefore a procurement requirement as well as an engineering one.

6.5. Mechanism and Evidence Architecture

The mechanism and evidence architecture assigns each transparency question to the component that carries it and to the stakeholder decision it supports. Component C2 carries data and variable influence; C3, objectives and constraints; C4, plan and action justification; C5, agent, tool, memory, contribution, and delegation provenance; C6, execution and authority; C7, operator interpretation and intervention; and C8, lifecycle continuity. Component C1 defines the system boundary, decision architecture, autonomy, and authority within which these questions acquire meaning.

The corresponding artefacts form one connected evidence architecture. The intended-purpose statement defines the material decisions and authority model; the transparency requirements specification identifies required mechanisms, stakeholder views, and acceptance criteria; the validation report records whether those criteria were met; runtime records preserve the material operational chain; and incident, monitoring, version, and change records show whether the accepted transparency remains valid as the system evolves.

Table 7 registers the framework components. Every component carries an identifier, a level, the regulatory function it serves, the criterion through which it is accepted, the artefact through which fulfilment is evidenced, and the role accountable for it. Assurance personnel assess every component for traceability, coverage, and consistency.

Table 7. Framework component register.

Component	Framework Element	System Level	Governing Regulatory Function	Acceptance Criterion	Evidence Artefact	Accountable Role
C1	System definition and authority map	System and lifecycle	Applicability transparency	Boundary, decision-architecture, autonomy, and responsibility clarity	Intended-purpose statement; authority model	Provider
C2	Data and variable influence disclosure	Model	Interpretive sufficiency	Minimum fidelity, stability, and uncertainty calibration	Explanation specification; uncertainty record	Developer
C3	Objective and constraint disclosure	Agent	Interpretive sufficiency	Coverage of material goals and constraints	Explanation record	Developer
C4	Plan and action justification	Agent	Interpretive sufficiency	Coverage of alternatives and action reasons	Explanation record	Developer
C5	Contribution and delegation provenance	Interaction	Reconstructable operation	Records of material delegations, messages, and authority transfers	Interaction record; tool-use record	Provider
C6	Execution and authority record	System	Reconstructable operation	Reconstruction of a representative operational event within a specified time	Execution trace; event log	Deployer
C7	Operator interface and intervention controls	System	Intervention-enabling transparency	Comprehension, response time, cognitive burden or situational awareness, intervention success, and handover continuity where applicable	Operator interface; intervention and handover record	Deployer
C8	Lifecycle evidence continuity	Lifecycle	Lifecycle evidence continuity	Complete traceability from regulatory function to change history, with unique identifiers, complete relations between adjacent levels, version linkage, and detection of missing relations	Technical documentation; validation, version, and incident records	Provider

6.6. Evaluation and Acceptance Criteria

Each transparency requirement should be expressed through testable or reviewable criteria. A criterion is testable where conformity can be established by measurement against a defined procedure, and reviewable where conformity can be established by structured judgement against defined coverage conditions. The distinction matters because model-level explanation properties admit measurement, whereas the sufficiency of an agent-level explanation for a particular oversight decision does not.

Model explanations may require minimum fidelity, stability, and uncertainty-calibration performance. Agent explanations may require coverage of material goals, constraints, alternatives, and action reasons. Interaction traces may require records of material delegations, messages, and authority transfers. System traces may require reconstruction of a representative operational event within a specified time. Operator interfaces should support accurate comprehension, task-appropriate situational awareness, acceptable cognitive burden, timely intervention, and continuity across handover where oversight is distributed across operators or shifts. Lifecycle evidence may require complete traceability from regulatory function to requirement, implementation, validation, operational record, and change history.

A criterion becomes operational only when the method through which it is assessed is specified. Fidelity, stability, and calibration are assessed through quantitative testing against held-out operating conditions. Goal, constraint, plan, and action coverage are assessed through structured review of sampled decisions. Trace criteria are assessed through completeness audits of sampled operational episodes, while reconstruction criteria are assessed through timed reconstruction exercises on representative events. Interface criteria are assessed through scenario-based operator trials using personnel with representative competence or training under representative time pressure; the trials should examine interpretation accuracy, response time, cognitive burden or situational awareness, intervention performance, and handover continuity where applicable. Lifecycle criteria are assessed through traceability audit from regulatory function to change history. These methods differ in kind, so reporting them under a single undifferentiated term such as validation would obscure which components rest on measurement and which on structured judgement.

Each evaluation is located at a lifecycle stage and assigned to a role. Pre-deployment evaluation occurs at stage L4 and establishes initial acceptance. In-operation evaluation occurs at stage L6 and establishes continued acceptance, because explanation fidelity, trace completeness, and operator interpretation can degrade through model updating, tool addition, data drift, and changes in operating regime. Assurance personnel verify each component against its criterion at both stages, while the accountable role recorded in Table 6 remains responsible for the component meeting it. Separating verification from accountability prevents the assessment of a mechanism from resting solely on the organisation that produced it.

The acceptance decision has three outcomes. A component is accepted when it meets its criterion. It is conditionally accepted when a bounded deficiency can be compensated and the compensating measure and re-evaluation date are recorded. It is rejected when the deficiency is neither bounded nor compensable, in which case the work returns to L3. Typical compensating measures include reduced autonomy, an additional approval step, a restricted operating envelope, or a shorter reassessment interval. Conditional acceptance requires explicit recording because it permits progression despite an unmet criterion.

Acceptance criteria and any numerical thresholds must reflect the application context. Thresholds are set at L4 for a defined stakeholder task and operating envelope by considering decision architecture, autonomy, operational consequence, available decision time, baseline mechanism performance, and acceptable residual risk. The validation plan must record both the selected value or coverage condition and its justification. A threshold suitable for a day-ahead advisory decision is therefore not presumed suitable for second-scale autonomous control, and a threshold established for one operating regime must be reconsidered when the system or context changes.

Figure 4 consolidates quantitative measurement and structured judgement within one acceptance path. Gate G4 records the pre-deployment decision, and G6 reassesses the same decision during operation. Conditional acceptance carries a compensating measure and re-evaluation date, while rejection returns the component to L3. Acceptance is therefore a maintained state rather than a result obtained once.

Figure 4. Evaluation workflow and acceptance gates. Each component of the register is assessed through the method appropriate to its criterion, and the acceptance decision is taken at gate G4 before deployment and at gate G6 during operation. Rejection returns the component to mechanism allocation at stage L3, and conditional acceptance carries a recorded compensating measure and a re-evaluation date.

The framework therefore specifies the criteria, assessment methods, decision structure, and threshold-setting logic without prescribing universal numerical values for fidelity, stability, calibration, latency, or retention. The reviewed evidence does not provide values validated across critical energy operations, so application-specific thresholds and their rationale become part of the validation evidence and are reassessed during operation at L6 and after material change at L7.

6.7. Conditions of Application and Proportionality

The framework is applied from the earliest stage at which the intended purpose can be defined. Where the agentic system is already in operation, application begins with a retrospective execution of stages L1 and L2 against the deployed system, followed by a gap assessment in which each component is classified as present and accepted, present but unaccepted, or absent. The classification determines whether the subsequent work is acceptance evaluation, mechanism implementation, or requirement specification, and it produces the same evidence artefacts as a forward application.

Proportionality adjusts the depth, latency, retention, and oversight applied to each component rather than removing components from the framework. Recommendation-only agents require sufficient interpretive information for independent human judgement and comparatively limited execution provenance. Approval-dependent agents require stronger plan, uncertainty, authority, and intervention evidence at the approval point. Bounded autonomous agents executing without prior approval require the strongest runtime traceability, authority records, monitoring, and interruption capability, with requirements increasing further as operational consequence or time sensitivity rises. The applicable transparency mechanism also depends on decision architecture: rule-based, optimisation, reinforcement-learning, LLM-based, and hybrid agents expose different internal objects, but all remain governed through the same component structure.

All eight components are retained because each performs a distinct transparency or evidence-continuity function that no other component discharges. Figure 2 shows that applicability transparency, interpretive sufficiency, and reconstructable operation each depend on mechanisms at more than one system level, whereas intervention-enabling transparency and lifecycle evidence continuity have their primary carriers at the system and lifecycle levels respectively. Where a component cannot be implemented at the required depth, the deficiency is recorded as a residual transparency limitation together with its compensating measure rather than tolerated as an implicit gap. The record of residual limitations is itself lifecycle evidence, and it identifies the points at which the transparency of the system depends on organisational measures rather than on technical mechanisms.

7. Scenario-Based Operationalisation and Applicability Assessment

7.1. Scenario and System Boundary

The framework is operationalised through an agentic virtual power plant coordinating photovoltaic generation, battery storage, electric vehicles, flexible building loads, and electricity-market participation. The virtual power plant interacts with market signals and distribution-network constraints. The operational purpose is to transform distributed resources into operational flexibility while respecting asset requirements, network limits, contractual obligations, and operator authority.

For demonstration purposes, the scenario is treated as a deployment in which the agentic virtual power plant performs a safety-relevant operational function within the critical-infrastructure context discussed in Section 3.1. This assumption provides a regulatory setting in which the framework can be exercised; it does not classify virtual power plants generally, or every AI function within a virtual power plant, as high risk under the EU AI Act.

The system contains a forecasting agent, market agent, network-constraint agent, asset agents, flexibility-coordination agent, and operator-interface agent. Forecasting models estimate demand, generation, prices, and asset availability. The market agent identifies economic opportunities. The network-constraint agent evaluates grid limitations. Asset agents report flexibility, state, and local constraints. The coordination agent develops and revises the portfolio plan. The operator retains authority over actions exceeding defined impact, uncertainty, novelty, or risk thresholds.

7.2. Agent Architecture and Decision Process

The decision process begins with forecasts, market information, network limits, and asset states. The coordination agent formulates a planning problem based on the operational goal and active constraints. An optimisation tool identifies a preferred flexibility schedule. Asset agents verify local feasibility. The network-constraint agent evaluates the expected grid effect. The coordination agent resolves conflicts, revises the plan where necessary, and submits the resulting plan for approval or execution according to the authority policy.

The process contains several explanation objects. Forecast explanations concern demand, generation, prices, and uncertainty. Agent explanations concern goals, constraints, plans, tool use, and revisions. Interaction transparency concerns negotiation among the market, network, and asset agents. System transparency concerns approval, execution, and physical consequences. Lifecycle transparency concerns the version, configuration, validated envelope, and known limitations of each component.

7.3. Representative Operational Event

A high-price period creates an opportunity to export stored electricity. The market agent recommends discharging the portfolio battery and postponing flexible consumption. The network-constraint agent reports that export remains feasible within the current transformer limit. The coordination agent generates an initial plan based on the market opportunity.

Before execution, the battery agent reports a newly detected availability constraint. The coordination agent invokes the optimisation tool again and develops an alternative using electric-vehicle flexibility and temporary load reduction. One electric vehicle has a departure requirement that conflicts with the proposed discharge. The relevant asset agent rejects that part of the plan. The coordination agent develops another alternative involving lower export and greater load reduction.

The revised plan remains technically feasible but carries greater forecast uncertainty and a larger effect on building comfort. The system therefore exceeds the threshold for autonomous execution and requests operator approval. The operator receives a prioritised explanation describing the market opportunity, battery constraint, electric-vehicle requirement, relevant alternatives, uncertainty, expected comfort effect, network margin, and consequences of approving or rejecting the plan. The operator modifies the load-reduction limit and approves the adjusted schedule.

7.4. Framework Application

At the model level, the system records the price, load, generation, and network forecasts used by the agents together with the associated uncertainty. The operator can inspect why the price opportunity and network margin changed.

At the agent level, the trace identifies the initial economic objective, active network and asset constraints, optimisation results, rejected alternatives, and the reason for each revision. The battery constraint and electric-vehicle departure requirement remain visible as material causal factors.

At the interaction level, the trace shows which agent proposed, rejected, or modified each part of the plan. The evidence records the authority of the asset agents to reject infeasible schedules and the authority of the coordination agent to propose, but not independently execute, a high-impact plan.

At the system level, the trace connects forecasts, agent interactions, operator modification, approved schedule, issued commands, and observed response. The evidence distinguishes the plan generated by the coordination agent, the plan approved by the operator, and the commands executed by the control platform.

At the lifecycle level, each component is linked to the active version, validated operating envelope, known limitations, and applicable transparency specification. The event can therefore be assessed against the system configuration that was active at the time.

The scenario exercises every component of the framework. Components C2, C3, and C4 carry the interpretation of the forecasts, the objective and active constraints, and the reasons for each plan revision. Components C1 and C5 carry the authority allocation and the record of which agent proposed, rejected, or modified each part of the plan. Component C7 carries the prioritised operator view through which the schedule was modified and approved. Components C6 and C8 carry the execution record and the version linkage through which the event can be reassessed against the configuration active at the time.

Table 8 records the scenario-based operationalisation. Each transparency need is paired with the framework component, mechanism, stakeholder, acceptance criterion, and evidence artefact that address it, so applicability is assessed component by component rather than for the framework as a whole.

Table 8. Scenario-based operationalisation and applicability criteria.

Transparency Need	Framework Component	Mechanism	Stakeholder	Acceptance Criterion	Evidence Artefact
Interpret the initial export recommendation	C2, C3, C4	Forecast explanation, uncertainty information, goal and plan explanation	Operator	Material drivers and uncertainty are correctly understood	Explanation record
Reconstruct plan revisions	C4, C5	Execution and interaction trace	Developer, operator, assurance personnel	Material constraints, rejections, and revisions can be recovered	Agent and interaction trace
Identify responsibility	C1, C5	Authority and delegation records	Provider, deployer, assurance personnel	Decision ownership and execution authority are unambiguous	Authority model and event record
Support intervention	C7	Prioritised alternatives and consequence view	Operator	The plan can be modified or rejected within the required time	Interface test and intervention record
Support incident or performance review	C6, C8	End-to-end provenance and version linkage	Developer and assurance personnel	The event can be reconstructed from retained evidence	Operational trace and lifecycle record

7.5. Applicability and Coverage Assessment

The scenario demonstrates that no individual XAI mechanism can provide the required transparency. Feature attribution can explain a forecast but cannot explain the collective plan. A plan explanation can describe the selected alternative but cannot verify the optimisation output or authority transfer. An event log can record system activity but may not communicate the operational significance of the event. Technical documentation can describe the architecture but cannot reconstruct a specific operational decision.

The framework addresses this limitation by connecting mechanisms across the five transparency levels. The scenario covers applicability transparency, interpretive sufficiency, reconstructable operation, intervention-enabling transparency, and lifecycle evidence continuity.

The scenario also exposes unresolved implementation requirements. These requirements include scalable multi-agent provenance, reliable explanation of adaptive reasoning, automated identification of material events, secure retention of traces, and evaluation of operator performance under realistic time pressure.

The scenario exercises the specification and construction stages of the lifecycle but not the acceptance decision itself. Each component can be identified, allocated to a system level, and paired with an acceptance criterion, which corresponds to stages L1 to L3 and to the decisions taken at gates G1 to G3. The scenario does not establish that any component would pass gate G4, because that determination requires the quantitative testing, structured review, completeness audit, and operator trial specified in Section 6.6 rather than a described operational sequence.

The scenario provides a structured applicability assessment. Operational validation requires implementation of the architecture, controlled operator studies, cybersecurity assessment, and evaluation across normal, abnormal, and emergency operating conditions.

7.6. Formative Framework Evaluation

Section 2.6 defined four criteria for the proportionate formative evaluation of the framework. The outcome of each is reported here together with the boundary of what that outcome can establish, so that the assessment is not read as validation.

Table 9 records the procedure, formative outcome, and boundary for each criterion. The four criteria differ in kind. Regulatory coverage and structural coherence are properties of the framework that can be checked against its own specification, whereas evidence traceability depends on an author-constructed derivation and scenario applicability depends on a constructed operational sequence.

Table 9. Formative evaluation of the framework against the criteria defined in Section 2.6.

Criterion	Procedure	Formative Outcome	Boundary of the Outcome
Regulatory coverage	Crosswalk from the five regulatory functions of Table 3 to components C1 to C8	Each of the five functions has at least one accountable carrier and one acceptance route; no function is left unassigned	Does not establish legal conformity, which also depends on classification, risk management, and conformity assessment
Evidence traceability	Audit of each component through its derived requirement, thematic finding, and supporting evidence in Table 5 and Table 7	Every component resolves to a derived requirement and a cited evidence base; the derivation reported in Table 5 is representative rather than exhaustive	The chain is author-constructed and is not fully reproducible from the article alone
Structural coherence	Consistency examination across functions, levels, components, roles, lifecycle stages, gates, and evidence artefacts	The accountable role recorded in Table 6 matches the accountable role recorded in Table 7 for all eight components, and every component carries one accountable, one verifying, and at least one using role	Internal consistency only, established by the authors rather than by external review
Scenario applicability	Application of components C1 to C8 to the operational event described in Section 7.3	All eight components can be instantiated and paired with a mechanism, stakeholder, acceptance criterion, and evidence artefact	Covers stages L1 to L3 and gates G1 to G3 only; no acceptance test at gate G4 and no field evaluation

The evaluation is formative and internal. It establishes that the framework is complete with respect to the regulatory functions it derives, internally consistent across its own components and roles, traceable to the evidence from which it was constructed, and applicable to a representative critical-energy sequence. It does not establish legal conformity, operational effectiveness, or generalisability, and no component has been subjected to the acceptance evaluation specified in Section 6.6.

8. Discussion

8.1. From Model Explainability to Operational Transparency in Agentic AI

The principal theoretical contribution is to move transparency from the output of an individual model to the behaviour of the agentic system and its lifecycle. Model-level XAI remains necessary, but operational actions can also depend on goals, constraints, plans, tools, memory, interactions, authority, and human intervention.

Operational transparency is therefore compositional: it depends on evidence continuity across system levels rather than on one explanation algorithm. Interpretable models do not prevent opacity in agent coordination, and understandable plan explanations do not correct unreliable model inputs. Cross-level traceability is needed to expose these inconsistencies.

This also separates transparency from persuasive rationale generation. Natural-language explanations may improve accessibility, but they must communicate verified information from models, agent states, tools, interactions, and execution traces; narrative coherence cannot substitute for fidelity.

8.2. Relationship with Existing Research

Energy-sector XAI has established interpretable and post hoc methods across forecasting, anomaly detection, and energy management [15,17,28], together with recurring implementation and evaluation concerns [8,38,77] and application evidence across several energy tasks [33,35,83,107]. The framework retains this foundation but treats model explanation as one level of a wider operational transparency architecture.

Explainable reinforcement learning extends XAI to policies and sequential decisions [39,40,41], with energy studies demonstrating rule extraction, interpretable policies, and human feedback [42,43]. The framework connects this evidence to agent authority, interaction, intervention, and lifecycle documentation.

Explainable-agent research addresses goals, plans, and action rationales [46,49], human–machine research examines timely information and operational intervention [22,52,54,55], and multi-agent research examines communication, delegation, and causal interaction [27,47,50]. The framework combines these strands with system traceability and stakeholder responsibility in critical energy operation.

Agentic smart-grid research integrates autonomous control, reinforcement learning, multi-agent systems, and digital twins [12,20,23,29] with related architectures for virtual power plants, charging, and AI-enabled energy ecosystems [14,25,26,108]. Power-sector reviews add safety, trustworthiness, and explainability concerns [18,19,21,28]. The present contribution complements these architectures by translating transparency functions into assessable system requirements and evidence artefacts.

EU energy-sector AI research has mapped regulatory barriers and obligations [16,59,64,65], while smart-grid governance adds policy and stakeholder perspectives [61,66,109]. The framework extends this work through an engineering an operationalisation of transparency requirements.

8.3. Implications for Algorithms and System Design

For algorithm development, interpretability should influence model and policy selection rather than be added only after training [31,81,82]. Empirical evidence shows that interpretable structures can be embedded in control and policy learning [40,41,42,44], while post hoc methods should be evaluated for fidelity and stability against the intended oversight function [33,35]. Model choice is therefore both a predictive and a transparency decision.

Planning and optimisation should expose goals, active constraints, material alternatives, plan changes, tool inputs, objective functions, warnings, and results; reinforcement-learning systems should expose reward priorities, policy behaviour, uncertainty, and relevant counterfactual actions.

Multi-agent algorithms should preserve communication and delegation semantics [24,47], including the contributing agent, task, authority, and relation to the resulting decision [22,48], so collective outcomes can be attributed rather than reduced to independent agent logs [27].

Tool and memory interfaces should preserve which external function, data source, or memory item influenced a material decision [21,62,86,89], particularly where foundation-model orchestration can hide stale information, unsupported assumptions, or incorrect tool outputs.

Runtime mechanisms should capture material events in structured traces without retaining unnecessary or security-sensitive internal activity, creating research needs in trace selection, causal event extraction, secure provenance, and role-specific presentation.

Evaluation must extend beyond local explanation quality [51,60,85] because overseeability depends on evidence distributed across the five levels. Cross-level trace coverage, authority consistency, latency, uncertainty communication, operator actionability, intervention success, and event reproducibility are therefore system-level measures.

8.4. Implications for Energy-System Development and Operation

Energy-system developers should establish transparency during intended-purpose definition [16,60,63] and integrate it before deployment [66]. Procurement and architecture decisions should require models, agents, tools, and interfaces to expose the evidence needed for explanation or traceability; otherwise, high predictive performance alone may be insufficient for critical operation.

Operator interfaces should align with established responsibilities [22,51,52] rather than transfer interpretive burden without corresponding authority. Human–machine power-system evidence supports this alignment [53,54,55]. Interfaces should prioritise goals, constraints, uncertainty, anomalies, authority, and intervention options, and their adequacy should be verified under operating conditions.

Deployers must determine where human approval remains necessary [22,52,57], consistent with evidence on operational authority [54,55]. Approval of every action can create workload and delay, while excessive autonomy weakens oversight; escalation should therefore depend on impact, uncertainty, novelty, and operating-envelope conditions.

Incident management benefits from end-to-end evidence continuity [62,63,86], including provenance that reconstructs contributing models, decisions, interactions, approvals, and commands [89]. This supports correction and accountability but requires integration across AI, software, control, and organisational records.

8.5. Regulatory and Standardisation Implications

The EU AI Act establishes functional transparency and information requirements without prescribing a complete technical architecture [58]. Its technology-neutral form allows context-specific implementation but creates a need for standards and sector guidance that translate regulatory functions into measurable system properties.

The five-level model provides one such structure: established XAI supports model-level requirements, while agent and interaction levels require stronger methods for goals, plans, provenance, delegation, and responsibility; system and lifecycle levels require execution traceability, intervention, documentation, versioning, and change assessment.

IEEE P7001 is the closest standardisation reference because it treats transparency as a stakeholder-dependent property of autonomous systems [80]. The present framework retains that orientation but binds transparency to five EU AI Act-derived functions, five system levels, component ownership, gated acceptance, and cross-level evidence continuity. It is therefore complementary to P7001. Energy-sector guidance can build on both while reflecting critical-infrastructure responsibilities [16,60,61], consistent with the wider need for sector-specific operationalisation [65,66].

Transparency-by-design remains one part of wider assurance. System classification, intended purpose, risk and quality management, and conformity assessment establish its regulatory context [16,60,61,65] while cybersecurity and data governance constrain safe disclosure [92,97,99] and privacy, ethical lifecycle analysis, and technical governance provide complementary controls [63,96,110].

8.6. Limitations and Future Research

Combining regulatory and heterogeneous scientific evidence broadens the framework but introduces variation in terminology, application context, and evaluation depth. Evidence-status classification improves comparability without turning repeated reports of the same underlying evidence into independent confirmation.

Evidence maturity remains uneven. Model-level XAI has the strongest energy-sector base [8,28,33,38] including recent power-system applications [35,41,77] while agent and interaction mechanisms are less mature [21,22,27] and tool, memory, and workflow provenance remain emerging [19,21,86]. Some components therefore rely on transferable adjacent-domain evidence [49,80].

The regulatory-to-engineering mapping remains an engineering interpretation subject to future guidance, standards, implementation practice, and case law.

The scenario is an applicability demonstration, not operational validation. Stronger evaluation requires implemented systems and stakeholder studies, including interpretation, cognitive load, automation bias [51,52], response time, anomaly recognition, and intervention quality [22,54].

Further research should address faithful agent-reasoning and tool or memory explanations [18,21,87], scalable multi-agent attribution and operator-centred evaluation [27,85], and secure, privacy-preserving lifecycle provenance [86,92,96,97]. Standardised benchmarks remain necessary [60], with critical-energy testbeds and digital twins providing suitable environments for evaluation under normal, uncertain, conflicting, cyber, and resilience conditions [111].

9. Conclusions

Agentic AI changes the transparency problem in critical energy systems from explaining isolated model outputs to reconstructing operational behaviour across models, goals, constraints, plans, tools, memory, inter-agent coordination, authority, execution, and human intervention. The study addresses this problem by integrating EU AI Act requirements with the scientific evidence and deriving five connected regulatory transparency functions: applicability transparency, interpretive sufficiency, reconstructable operation, intervention-enabling transparency, and lifecycle evidence continuity. These functions establish a regulatory-to-engineering basis for operational transparency without equating technical transparency with legal conformity.

The principal contribution is an author-constructed transparency-by-design framework comprising five connected transparency levels, eight operational components, and a seven-stage gated lifecycle. The five levels locate where transparency must be maintained, the components specify what information and evidence must be produced, and the lifecycle governs when requirements are defined, implemented, accepted, instrumented, monitored, and reassessed. Stakeholder responsibilities, controlled information views, acceptance criteria, assessment methods, and linked evidence artefacts make the framework usable by providers, developers, deployers, operators, and assurance personnel. Application to an agentic virtual power plant demonstrates how these elements can be instantiated within one reconstructable evidence chain from information acquisition to authorised operational action.

The contribution remains bounded by the maturity of the evidence and the formative nature of the evaluation. The virtual power plant is a representative design-science demonstration rather than an empirical case study or a legal classification of virtual power plants generally. The internal evaluation establishes regulatory coverage, evidence traceability, structural coherence, and scenario-based applicability, but it does not establish legal conformity, field effectiveness, transferability, or generalisability. Evidence for model-level explanation is more mature than evidence for agent, interaction, tool, memory, and lifecycle transparency, and universal numerical acceptance thresholds cannot yet be justified for critical energy operation.

The next research stage should evaluate the framework through external expert or assurance walkthroughs and implemented critical-energy settings, calibrate acceptance thresholds and operator-centred measures under realistic time pressure, and test scalable cross-level provenance for agent reasoning, tool and memory use, delegation, execution, and system change. Until such evidence is available, the framework should be understood as a structured and traceable engineering basis for specifying, assessing, and maintaining operational transparency rather than as a validated conformity solution. Its central implication is that transparency for agentic AI is a maintained system capability to interpret, reconstruct, oversee, and reassess how an authorised operational action came to occur.