Key Value Indicators for Sustainable AI-Based Communication Networks: Are They Adequate?

Abstract

As climate and societal challenges intensify, achieving sustainable development requires a holistic approach that balances economic growth with environmental preservation and social equity. The United Nations’ 2030 Agenda outlines 17 Sustainable Development Goals with corresponding global indicators. However, adapting these high-level goals to the specific context of telecommunications networks, which are more and more governed by AI-based components, presents major challenges. This paper proposes an extensive framework for assessing the adequacy of Key Value Indicators, used to evaluate the Key Values brought by network services and applications. The framework encompasses the phases of indicator elicitation, analysis, technical realization, and assessment. Applying this framework ensures that Key Value Indicators remain relevant, actionable, and pertinent to the objectives throughout the system’s evolution. Case studies from leading projects in the telecommunications sector illustrate how the proposed methodology effectively identifies deficiencies and helps refine the framework.

1. Introduction

In an era of global challenges like climate change, social inequality, and economic instability, sustainability is crucial for long-term well-being. Sustainable development requires frameworks for organizations to identify Key Values (KVs), which are guiding principles for what technology should accomplish in the long term, beyond mere technical performance. Examples of these KVs are social inclusivity, workplace safety and diversity, open collaboration, fairness, security, and trust. In this context, the Sustainable Development Goals (SDGs) constitute a milestone, as they provide international guidelines established under the United Nations 2030 Agenda for Sustainable Development, comprising 17 objectives. Efforts to realize these global goals focus on three fundamental sustainability principles: environmental, social, and economic, which correspond to the planet, people, and profit. To effectively address the three pillars, all relevant stakeholders must actively participate in developing future networks. Focusing on a single domain without a holistic perspective cannot yield sustainable networks designed for longevity. Telecommunications are key enablers of digital and ecological transitions, offering sustainable alternatives to physical transport by enabling electronic information exchange. They help bridge the digital divide, especially in rural areas, while supporting education, healthcare, and economic development. The evaluation of KVs leads to the development of Key Value Indicators (KVIs), metrics used to assess their progress [1]. KVIs, alongside other Key Indicators (KIs), such as Key Performance Indicators (KPIs) and Key Quality Indicators (KQIs), provide a comprehensive view of corporate wellness, supporting decision-making and planning. Emerging technologies like AI and virtualization now play a central role in managing and scaling these systems [2]. The Smart Networks and Services Joint Undertaking (SNS JU) began evaluating KVIs in the context of the 6G-IA SNVC (Societal Needs and Value Creation Sub-Group) by analyzing the KVIs of the main project on this topic [3]. KVIs have been classified in the different categories, such as Environmental Sustainability, Societal Sustainability, and Economic Sustainability & Innovation among others. However, no KVI definition or structure has yet been concretely proposed and standardized.

In this paper, we propose a set of criteria for evaluating the adequacy of KVIs in modern networks, leveraging advanced technologies such as Intent-Based Networking (IBN) and the Digital Twin (DT) [4]. These criteria are intended to be used within a framework for defining the KVIs, which includes the elicitation, analysis, realization, and assessment phases. Moreover, we review the major KVIs that have been defined until now, which, in most cases, do not fully satisfy the identified criteria. We then claim that further efforts are needed to define effective indicators for this setting, which requires an interdisciplinary approach and standardization. AI modules can help implement the proposed framework by leveraging their inherent capabilities to evaluate the contribution of each network slice and component to the overall KVs.

The rest of the paper delineates state of the art (Section 2), defines additional criteria to evaluate the adequacy of indicators (Section 3), conceptualizes a method to evaluate the adequacy (Section 4), and shows examples of how to use the criteria presented in this paper to evaluate the adequacy of a single KVI or a set of KVIs (Section 5). The paper concludes with a discussion of the approach’s limitations and future perspectives (Section 6).

2. State of the Art

The evolution toward AI-native communication networks has fundamentally changed how network management and performance evaluation are approached. In 5G, the introduction of Software-Defined Networking (SDN), Network Function Virtualization (NFV), and network slicing enabled programmable and flexible infrastructures, but network management remained largely manual and reactive, relying on rule-based configurations and operator expertise [5]. Beyond 5G marked a shift toward intelligent autonomy: AI/ML-driven analytics, zero-touch service management, and intent-based interfaces began translating high-level objectives into automated network actions, enabling self-monitoring, self-diagnosis, and self-healing capabilities. In 6G, this trajectory reaches its logical conclusion in AI-native, goal-oriented architectures where the network does not merely transport data but interprets intents, predicts demand, and autonomously optimizes behavior across slices and domains [6]. This evolution is also bringing a shift from performance-driven to value-driven network management, where success is measured not only by throughput and latency but also by contribution to carbon reduction, digital equity, and social inclusion [7], which is precisely what motivates the development of KVI frameworks capable of capturing these broader dimensions. Traditional KPIs, while still necessary, are insufficient to assess whether a network is truly sustainable and aligned with societal goals; KVIs fill this gap by linking network behavior to Key Values that reflect the long-term objectives of all stakeholders [1]. While KVIs are increasingly referenced in the literature, their definitions are often incomplete and heterogeneous. Therefore, this work focuses on evaluating and refining partially defined indicators rather than assuming fully mature ones. The most in-depth work about this topic is “Key value indicators: A framework for values-driven next-generation ICT solutions” [8]. It presents a five-step framework for developing KVIs. The first step, Definition (I), defines scenarios, Use Cases (UCs), objectives to be achieved, stakeholders involved, and requirements. The second step, Elicitation (II), identifies the outcomes and the Key Values associated with the UCs. The third step, Analysis (III), identifies the relevant KVIs, determines their ranges, and optimizes them. The fourth step, the Technical Realization (IV) or Realization, is an iterative design process that relates the enablers to the KVIs. Finally, the Assessment (V) evaluates KVIs through various methods such as expert assessments, simulations, and twinned systems. KVIs may be reformulated after the assessment based on improved understanding, and further mitigation actions may further enhance the proposed solutions.

Figure 1 illustrates how these phases operate within the broader ecosystem in which KVI frameworks are deployed. The four phases form an iterative cycle, with stakeholders actively participating in all phases. The AI-native network architecture, comprising the IBN layer and the Network Digital Twin (NDT), is deployed during the Realization phase and continuously monitored during Assessment, feeding insights back into the Elicitation phase when KVI definitions require revision.

Figure 1. Overview of the proposed framework ecosystem. The four phases (Elicitation, Analysis, Realization, and Assessment) form an iterative cycle around the stakeholders, who participate in all phases. The AI-native network architecture is deployed during the Realization phase and continuously monitored during the Assessment phase, feeding insights back into the Elicitation phase.

The transition to AI-native network architectures introduces additional sustainability concerns that KVI frameworks must address. AI models deployed in network management and orchestration consume significant computational resources during both training and inference, contributing to the overall energy footprint of the network [6,9]. Achieving network-wide sustainability requires that AI-driven orchestrators optimize jointly across slices and domains, rather than locally, to avoid shifting inefficiencies rather than eliminating them [2]. Recent works have demonstrated that AI-native closed-loop control in O-RAN architectures can achieve measurable energy savings while preserving quality of service, confirming the feasibility of embedding sustainability objectives directly into network management functions [10].

The sustainability of ICT systems is also increasingly evaluated through Environmental, Social, and Governance (ESG) frameworks, which provide structured criteria for assessing non-financial performance. While ESG reporting was originally developed for corporate governance, its criteria overlap significantly with the KVI dimensions discussed in this paper. Notably, the qualitative assessment methods used in ESG reporting, such as expert panel evaluations, third-party audits, and standardized disclosure frameworks, offer useful methodological precedents for KVIs that cannot be measured through purely objective metrics [11]. Trustworthy AI and explainability are further dimensions requiring that AI-driven decisions must be interpretable and auditable, so that the system’s behavior can be understood and challenged [6,12]. Moreover, a trustworthy AI component must be robust to input perturbations, resilient to failures, available continuously, and protected against adversarial interference through effective access control and accounting mechanisms [13].

As far as the authors know, no method exists for evaluating network KVIs and many studies highlight the need to define and assess KVIs [3,14,15] in a standardized manner. Therefore, this paper proposes adapting the criteria typically used for other indicators in the literature: the SMART criteria [16] (Specific, Measurable, Achievable, Relevant, and Time-bound), the Spangenberg’s criteria [17] (General, Indicative, Sensitive, and Robust), and the L.1480 guidelines [18] (Accurate, Complete, Conservative, Consistent, Relevant, and Transparent) defined by the International Telecommunication Union (ITU). Although the latter are designed for indicators that measure gas emissions, we believe they are also suitable for other KIs. Some criteria across frameworks overlap, so we summarize them more compactly in Section 3. However, once an initial set of KVIs is defined, the major stakeholders must agree on their definitions, converge them, and standardize their procedures so KVIs can be used in a multi-tenant system. Various international projects such as Hexa-X [19] and ADROIT6G [20] are already addressing the initial four phases of the design and development of KVIs (definition, elicitation, analysis, and realization). In the following section, we propose a framework to evaluate the adequacy of KVIs throughout their development, thereby completing the final phase (assessment).

3. Additional Criteria for the Adequacy Evaluation of Key Value Indicators

Evaluating the adequacy of KVIs in measuring KVs or SDGs requires ensuring they effectively capture the desired outcomes and provide actionable insights. We believe that a set of criteria is needed to effectively assess the adequacy of indicators for network sustainability, integrating those from the literature. The adoption of these criteria at this stage of sustainable network development is essential to ensure that the overall process can effectively achieve its intended objectives. As discussed earlier, we argue that the criteria already adopted in other contexts can also be applied to our scenario. These include the SMART criteria, Spangenberg’s criteria, and the L.1480 guidelines. We believe that, with appropriate adaptations, these frameworks can be effectively employed for our purposes, while additional criteria may be introduced to address specific requirements. Specifically, we suggest adding the following additional principles to complement the existing criteria: Integration, Explainability, and SDG Comprehensiveness. The rest of this section provides more detailed information about the additional principles we propose, and then we present the revision on the existing criteria that brings to the final complete list.

3.1. Integrate

Integration is composed of three enfolded elements [21]: Integrability, Interoperability, and Composability. Integrability refers to components that are compatible and can communicate effectively with each other. It can be hindered by compatibility issues, resource limitations, or implementation costs. Interoperability requires adopting common standards for communication and data interpretation. It ensures that all stakeholders consistently apply and understand the definitions of indicators, measurement units, and acceptable value ranges across the network. Composability enables components to work seamlessly together, allowing them to be combined in different ways to meet specific requirements. A composable indicator can be decomposed into smaller, independent sub-indicators, thereby supporting system scalability and enabling reusability across contexts.

3.2. Explainable

Explainability enables AI systems to clarify their operations and decisions through human-understandable explanations. Customers prefer services that match their values, prompting companies to adopt sustainable policies. Therefore, a key challenge is effectively communicating the company’s values and how they are achieved in a transparent and reliable manner. Explanations should be personalized to account for individual preferences and backgrounds. Moreover, dialogue facilitates knowledge sharing and progress monitoring [12]. With the rise of IBN technologies, some authors [13] propose training a Large Language Model (LLM) to explain metrics linked to KVIs. Expert teams should focus on creating initial definitions and periodic verifications (e.g., annually or whenever there are abnormal peaks of requests for clarification). At the same time, users can assess their understanding through surveys.

3.3. SDG Comprehensive

Unlike other criteria, SDG Comprehensiveness refers to a set of indicators viewed as a whole. While no single KVI can represent all the SDGs, advancing sustainable development requires considering the full spectrum of these goals. The set of KVIs should be minimal yet sufficient, covering all key sustainability dimensions without overwhelming stakeholders or obscuring the overall picture. In practice, the set of SDGs that a network operator must address is shaped by two complementary forces: regulatory obligations and user preferences. On one side, national and regional legislation increasingly translates SDG targets into binding requirements, such as carbon emission caps, energy efficiency mandates, or digital inclusion obligations. On the other side, users and enterprise customers may actively seek providers whose service offerings align with their sustainability values, creating market-driven pressure to address additional SDG dimensions beyond legal minima. This observation motivates an approach based on service packages: a network operator defines a baseline package of KVIs whose thresholds are determined by applicable legal constraints, and one or more optional packages that address additional SDG-aligned preferences expressed by users or other stakeholders. This structure also provides a natural mechanism to manage conflicts between SDGs. When two goals are in tension, for instance, economic growth (SDG 8) and climate action (SDG 13), the prioritization follows an explicit ordering: legally mandated thresholds take precedence and cannot be traded off, while conflicts among preference-driven KVIs are resolved through stakeholder negotiation at the time of package definition. Indicators that address SDGs for which neither legal obligations nor documented user demand currently exist can still be listed as not yet actionable in the KVI-to-SDG Matrix, rather than omitted. This labeling acknowledges the gap without claiming coverage that does not exist and signals which dimensions require future standardization efforts. Under this framework, the coverage of the KVI set is considered sufficient when: (i) all SDGs for which binding regulatory requirements exist in the deployment context are addressed by at least one KVI with a defined threshold; and (ii) the set of optional KVIs covers the sustainability preferences elicited through a structured stakeholder engagement process, such as surveys, focus groups, or deliberative workshops, and documented prior to package definition.

As a preliminary step, each KVI can be mapped to one or more relevant SDGs to assess alignment. For example, indicators related to energy efficiency might be linked to SDG 7 (Affordable and Clean Energy) and SDG 13 (Climate Action). A useful approach is to construct a KVI-to-SDG Matrix, which visually links indicators to specific SDG targets. This approach not only reveals synergies and overlaps but also helps identify potential conflicts between goals. For example, even though an indicator for reducing greenhouse gas (GHG) emissions may be strongly related to energy efficiency, it might inadvertently discourage the refurbishment of outdated equipment, delaying the adoption of newer, more energy-efficient technologies. Such conflicts are resolved by explicitly ordering priorities: legally mandated thresholds take precedence, while tensions among preference-driven KVIs are addressed through stakeholder negotiation at the time of package definition. To guide this process, the United Nations Conference on Trade and Development (UNCTAD) published guidelines on core indicators for SDG-aligned corporate reporting [11].

3.4. Further Considerations About the Criteria

To reduce redundancy in our framework of criteria (listed in Table 1), we merged the definitions of Relevant (SMART and Spangenberg’s frameworks) and Indicative (ITU framework), and extended the definition of measurable indicators because KVIs are not always based on objective measurements. Thus, when subjective metrics are used, they must be derived from statistically representative samples through aggregation or averaging. Concretely, survey-based KVIs should adopt a standardized measurement scale (e.g., a Likert scale with explicit anchoring) and be administered to a stratified sample that includes representatives of the main stakeholder categories: network providers, end users, and regulators. Inter-rater reliability should be assessed using established coefficients such as Krippendorff’s alpha, particularly for KVIs evaluated through expert judgment rather than user surveys. In the absence of specific guidelines, the evaluation should be performed by an inclusive panel of experts to ensure minimal cross-cultural validity. Examples of panel composition can include at least one expert from a high-income country and at least one from a lower-middle-income or low-income country, following the World Bank income classification [22], or equivalently, at least one expert from a developed country and one from a developing or least developed country (LDC), according to the UN Statistics Division classification [23]. Auditing across evaluation cycles is also recommended: comparing results obtained at different points in time allows detecting systematic drift in subjective assessments, which may reflect changes in social norms or stakeholder expectations rather than actual changes in the measured Key Value. However, the criteria cannot be completely independent. For instance, increasing explainability may require simplifying models, potentially reducing accuracy. These trade-offs should be explicitly managed by prioritizing criteria based on the application context and stakeholder needs.

Table 1. Complete Criterion List of the Proposed Framework.

Criterion	Description
1. Specific	The indicator should be well-defined and focus on a clear and precise aspect of the goal it measures.
2. Measurable	The indicator should provide quantifiable data to enable progress tracking.
3. Attainable	The indicator should be realistic and achievable, given the available resources and constraints.
4. Relevant/Indicative	The indicator should capture meaningful aspects and be truly representative of the phenomenon it is intended to characterize.
5. Time-bound	The indicator should include clear time information, enabling timely evaluations of progress.
6. General	The indicator should apply broadly across the network, independently of specific situations or societies.
7. Sensitive	The indicator should react timely and sensibly to changes in the monitored phenomenon.
8. Robust	The indicator should remain reliable and stable, showing no significant changes in case of adjustments.
9. Accurate	The indicator should minimize biases and uncertainties as much as possible.
10. Complete	The indicator should include all impacts that affect and contribute to the overall result.
11. Conservative	The indicator should avoid overestimating positive impacts in uncertain qualitative assessments.
12. Consistent	The indicator should ensure comparability over time by using compatible evaluation methods.
13. Transparent	The indicator should provide sufficient information to support the interpretation of the results.
14. Integrate	The indicator should be integrable, interoperable, and composable.
15. Explainable	The indicator must be clear to the public, fostering trust by effectively communicating Key Values.
16. SDG Comprehensive	The framework of indicators should sufficiently and minimally cover all SDGs.

4. How KVIs Achieve Adequacy

The adequacy of KVIs cannot be verified in a single step but emerges from an iterative process that spans the entire lifecycle of indicator development. Rather than treating adequacy as a final validation stage, we consider it as a continuous property that must be progressively refined from conception to deployment and operation. The proposed approach aligns with the phases introduced in [8], namely elicitation, analysis, technical realization, and assessment. Each phase introduces a subset of criteria that are evaluated at that stage, although fully satisfying them may require work across multiple phases. Figure 2 illustrates how different adequacy criteria gain prominence at each development phase. As KVIs evolve, previously satisfied criteria must be periodically re-evaluated to ensure their validity under changing system conditions. Criteria should not be rigidly assigned to specific stages. The actual progression depends on the nature of the KVI and the choices made by those implementing it within the network. The following subsections describe the role of each phase in the adequacy evaluation process, including the most relevant conflicts that may arise.

Figure 2. Phases of the assessment of the adequacy and the most important characteristics that a KVI should have in each phase (note that each phase includes the criteria of the previous ones).

4.1. Elicitation (or Conception)

The elicitation phase defines the initial structure of KVIs by aligning them with stakeholder objectives, use cases, and targeted Key Values. At this stage, indicators are still abstract and should satisfy the SMART criteria. Specificity and relevance ensure that the indicator captures a well-defined aspect of the intended value. Measurability and time-boundness constrain the indicator to observable and temporally defined quantities, enabling preliminary validation. These properties support the assessment of attainability, which reflects whether the indicator can be realistically implemented given available resources. The outcome of this phase is a clearly defined indicator, including its purpose, scope, measurement approach, and expected temporal resolution.

4.2. Analysis

In the analysis phase, individual KVIs are structured into a coherent and non-redundant set. The focus shifts from individual indicators to their interaction and collective coverage of sustainability objectives. Indicators should be general enough to remain valid across different deployment contexts, while also supporting integration across heterogeneous network domains. This includes interoperability of definitions, units, and measurement procedures. Decomposition into sub-indicators is evaluated to improve composability and reduce overlap. At the same time, sensitivity, which can only be assessed after the measurability criterion is satisfied, should be assessed to ensure that each KVI reacts meaningfully to changes in the underlying Key Values. This phase also introduces SDG comprehensiveness, verifying that the set of KVIs provides sufficient coverage of sustainability goals without unnecessary duplication. Involving an interdisciplinary team at this stage supports validating the system before it is operational, thereby incorporating diverse perspectives into the evaluation.

4.3. Technical Realization (or Realization)

During technical realization, KVIs are implemented and validated against real or simulated data. This phase enables a more rigorous evaluation of criteria that depend on operational behavior. Accuracy is assessed by comparing measured values with ground truth or reference data, when available. Consistency requires stable measurement methodologies over time and across system components. Completeness ensures that all relevant factors influencing the indicator are accounted for, while conservativeness prevents systematic overestimation of positive outcomes under uncertainty. Integration is further validated by verifying that KVIs can be effectively computed using available data sources and embedded into the network monitoring and control infrastructure. Indicators that do not meet these criteria must be refined, possibly requiring adjustments to their definition.

4.4. Assessment

The assessment phase evaluates the adequacy of KVIs under real operating conditions and over extended periods. Unlike previous phases, it focuses on long-term validity and stakeholder perception. Robustness is verified by analyzing the stability of the indicator under variations in system conditions. Transparency requires that both the measurement process and the resulting values can be inspected and understood. Explainability ensures that the meaning and implications of KVI values can be communicated to different stakeholders, including non-technical users. During this phase, indicators must be monitored to prevent obsolescence, especially in response to external changes such as new standards or policies. When necessary, indicators should be updated or replaced, ensuring backward compatibility. Feedback mechanisms, such as audits or stakeholder surveys, can be used to complement qualitative evaluation, particularly for indicators that include subjective components. Furthermore, when AI-based components are used to support KVI measurement or assessment, additional requirements apply: the outputs of such components must be traceable, auditable, and reliable. This implies that AI-assisted evaluations should be accompanied by provenance records, confidence estimates, and mechanisms for human oversight, so that stakeholders can verify and challenge the results.

4.5. Iterative Refinement

The four phases should not be interpreted as a linear pipeline. Instead, they form a feedback loop in which insights from later stages inform earlier ones. For example, issues identified during technical realization may reveal that an indicator is not measurable as initially defined, requiring a revision of its formulation. Similarly, assessment results may highlight a lack of robustness or relevance, prompting updates in the analysis phase. This iterative process ensures that KVIs remain aligned with both system evolution and sustainability objectives, avoiding the risk of becoming static or obsolete.

Following the Implementation phase, when the IBN layer is operational and KVI targets are established through the Elicitation and Analysis phases, the Assessment phase can be enhanced by a continuous real-time monitoring loop featuring IBN and DT components, as shown in Figure 3. The Intent-Fulfillment layer receives the stakeholder requests or predefined service packages (Section 3) as intents and translates them into KVI thresholds. This coupling is critical: if a stakeholder requirement cannot be mapped to any defined KVI during the preceding phases, it remains unmonitored, and the corresponding sustainability objective cannot be enforced at runtime. An NDT, which replicates both the logical behavior and the virtualized infrastructure of the system [24], operates continuously in two complementary roles. On the ond hand, it acts as a real-time simulation environment: before any configuration change is applied to the live network, the NDT evaluates whether the resulting KVI values would remain within the target ranges defined by the intents. On the other hand, it supports continuous monitoring by collecting telemetry data and measuring service level. Intent-Assurance functions [25] continuously compare the observed network behavior against the desired status to assess intent compliance and calculate KVI drift (i.e., when one or more KVIs deviate from their target ranges) over time. When drift is confirmed, Intent Compliance Actions trigger a corrective intervention automatically or alert operators when human judgment is required. Abstraction, Aggregation, and Reporting functions provide human-readable summaries of KVI compliance status. The corrective intervention may follow two paths depending on the nature of the drift. If the deviation is due to a change in network conditions, the network handles it autonomously: the configuration is updated to realign the KVIs with the existing targets. If the deviation reveals that a KVI definition or threshold is no longer adequate under the evolved system, the system requires support from a human operator, who might intervene by generating a new request or changing the KVI mapping, and the Intent-Fulfillment layer re-propagates the updated intent, closing the loop. Reporting functions are fundamental in the long term for identifying gaps in metrics to achieve specific goals, so that the mapping between intents and KVIs can be revised and/or new KVIs added.

Figure 3. Assessment phase workflow for KVI management. The Intent-Fulfillment layer defines the KVI mapping; the NDT performs real-time simulations before applying the requested changes; Intent Assurance functions detect KVI drifts and trigger corrective actions that either update the network configuration or revise the KVI definitions.

4.6. Operationalizing the Criteria: Priorities and Conflicts

The progression of phases can guide the adequacy verification: early phases focus on conceptual clarity and measurability, while later phases address operational robustness and stakeholder communication. However, the mapping should not be interpreted as a rigid checklist. The actual prioritization depends on the nature of the KVI and the choices made by those implementing it within the network, as illustrated in Section 5.3. Criteria introduced in a given phase begin to be evaluated at that stage, but may not be fully satisfied by its end: partial satisfaction is acceptable as long as the identified gaps are documented and addressed in subsequent phases. This reflects the reality that some properties, such as robustness or explainability, can only be properly evaluated once the indicator is implemented and operating under real conditions. When criteria conflict, two types of tension may arise: conflicts within the same phase, where satisfying one criterion makes another harder to achieve, and conflicts between criteria introduced in different phases, where a new criterion challenges a decision made earlier. Table 2 lists the most relevant conflicts for each criterion.

Table 2. Mapping of adequacy criteria to the four development phases and potential conflicts. A filled circle (•) indicates that the criterion is first introduced and begins to be evaluated in that phase; an open circle (○) indicates it is re-evaluated or carried forward. Conflicts identify criteria numbers that may be in tension when both are being evaluated.

Criterion	Elicit.	Analysis	Tech. Real.	Assess.	Conflicts
1. Specific	•	○	○	○	3
2. Measurable	•	○	○	○	3
3. Attainable	•	○	○	○	1, 2
4. Relevant	•	○	○	○	–
5. Time-bound	•	○	○	○	–
6. General		•	○	○	1
7. Sensitive		•	○	○	8
8. Robust				•	7
9. Accurate			•	○	15
10. Complete			•	○	8, 11
11. Conservative			•	○	10
12. Consistent			•	○	13
13. Transparent				•	12
14. Integrate		•	○	○	2
15. Explainable				•	9
16. SDG Comprehensive		•	○	○	–

In the Elicitation phase, the central tension involves Attainable (3) against Specific (1) and Measurable (2). A more precisely defined indicator with a well-specified measurement scale makes it easier to assess whether the objective is realistically achievable. At the same time, excessive specificity or a demanding measurement scale may make the indicator unattainable given available resources. The recommended approach is to seek a level of specificity and measurability sufficient to evaluate attainability and to defer further refinement to the Analysis phase if needed.

General (6) may conflict with Specific (1): broadening the indicator to apply across deployment contexts may dilute the precision of its original definition. This tension is best resolved by decomposing the indicator into sub-indicators, preserving specificity at the sub-indicator level while achieving generality at the aggregate level. Integrate (14) may also complicate Measurable (2): decomposing an indicator into interoperable sub-indicators can make the overall measurement procedure more complex. Complete (10) may conflict with both Conservative (11) and Robust (8): accounting for all relevant factors increases the risk of overestimating positive outcomes under uncertainty and may make the indicator more sensitive to changes or new findings that require updates to its definition.

Transparent (13) may conflict with Consistent (12): making the measurement process fully auditable may require methodological changes that reduce comparability over time. When this occurs, backward compatibility should be ensured by maintaining a transition period during which both formulations are reported in parallel. Accurate (9) may conflict with Explainable (15) because more sophisticated measurement methods improve accuracy but reduce interpretability for non-technical stakeholders. The recommended approach is to adopt the simpler metric for operational monitoring and reserve the more complex one for periodic expert audits. Robust (8) may conflict with Sensitive (7): an indicator verified as stable across conditions may respond more slowly to meaningful changes in the underlying Key Value. In all cases, conflicts that cannot be resolved within a phase should be explicitly documented rather than silently deferred, and revisited in the subsequent phase or during iterative refinement.

5. Evaluating the Adequacy of Some KVIs from Other Projects

Unfortunately, at this early stage of defining criteria, it is difficult to evaluate the adequacy and measurement of KVIs. There is significant variation in the definitions of the Key Values and the analytics used, both for networks and for 6G applications and end solutions. Moreover, the majority of projects have not yet fully defined the validation methods and their target values, which poses a significant challenge for next-generation networks [3]. In this section, we list and discuss some Key Value Indicators proposed by Adroit6G [26], and provide an overall evaluation of the KVI framework of Hexa-X [27].

To support reproducibility, the evaluations in this section follow a uniform protocol based on the 16 criteria listed in Table 1. For each KVI or KVI framework under analysis, each criterion is assessed against the published definition using a three-level scale: satisfied, when the definition provides sufficient information to meet the criterion; partially satisfied, when the definition addresses the criterion only in part or ambiguously; and not satisfied, when the criterion is not addressed. An example is shown in Section 5.3. The assessment is grounded exclusively in publicly available documents; criteria that cannot be evaluated due to missing information are explicitly noted as such rather than assumed to be met. This protocol allows independent researchers to replicate the analysis by applying the same criteria and scale to the same source documents.

5.1. An Overall Analysis of the KVI Framework in Adroit6G

The KVI framework developed in the Adroit6G [20] project distinguishes 14 KVIs into three categories: Innovation (I1: Safety, I2: Security, I3: Regulation, I4: Responsibility, and I5: Energy efficiency), Democracy (D1: Privacy, D2: Fairness, D3: Digital Inclusion, and D4: Trustworthiness), and Ecosystem (E1: Sustainability, E2: Business Value, E3: Economic Growth, E4: Open collaboration, and E5: New Value Chain). Each KVI is described through (i) relevance, (ii) aim, and (iii) how to measure it. This structure focuses on only three criteria (relevance, specificity, and measurability), omitting others (e.g., robustness, accuracy, completeness, conservativeness, consistency, and integration) that are often overlooked. Most KVIs lack a timeline for measurement, which is essential for tracking progress, or suggest inadequate measurement methods. Finally, indicators based on qualitative methods (e.g., surveys and interviews) may introduce subjectivity and lack clear guidelines. The framework lays a solid foundation for assessing the alignment of proposed 6G solutions with societal and economic goals. However, only two indicators, “Sustainability” and “Energy Efficiency,” address environmental aspects. Broad definitions or overlapping elements could complicate the identification of the correct KVI during the assessment. The following paragraphs analyze some important KVIs, defined in [26].

5.1.1. I2 Security

The definition of the KVI I2 Security in the ADROIT6G deliverable [26] reads as follows:

• Relevance: Protection of data and 6G systems in a way that prevents negative impact.
• Aim: To consider security in the design of 6G systems given the billions of interconnected devices, the millions of subnetworks that typically cannot be trusted, the open interfaces, architectural disaggregation, heterogeneous cloud environment as well as a mix of open-source and multi-vendor software, and prove their robustness against cyber threats leading to the offering of secure solutions.
• How to measure it: Identify 6G system vulnerabilities and security concerns through questionnaires and focused group surveys from webinars, workshops and events. Furthermore, through the security issues log identified in Proof of Concepts (PoCs).

Although the literature on security is rich [28,29], the description of this KVI is overly broad, and the measurement methods appear naive given the current state of network security tools. According to the literature, security is composed of three pillars (CIA triad), namely confidentiality, integrity, and availability. Additional components not considered in the provided definition are authenticity, nonrepudiation, and privacy. These features can be incorporated into the primary security indicator. Some of the sub-indicators listed above are metrics that can be measured using KPIs (e.g., uptime, encryption strength, or penetration-test success rates). Adding time-bound elements enables tracking progress through defined deadlines or periodic reviews. For example, a time-bound formulation could require that critical vulnerabilities are patched within 7 days, with progress reviewed quarterly.

5.1.2. D2 Fairness

The definition of the KVI D2 Fairness in the ADROIT6G deliverable [26] reads as follows:

• Relevance: Ability of the AI/ML algorithms to perform decisions that are free from discrimination and bias.
• Aim: To create a next generation 6G architecture that results in both highquality techniques and social justice through de-biasing of metadata and cocreation.
• How to measure it: ADROIT6G will design and build data and metadata sets for AI/ML training and fairness optimisation to avoid bias of data as measured through its testing and verification methodology during the PoCs. The methodology will use white box testing that reviews the data used to train the AI/ML models to avoid bias.

Adroit6G’s definition of this indicator is vague and lacks specific criteria for defining fairness in AI/ML decision-making. Terms like social justice and de-biasing are not objective, leaving room for interpretation and making it unclear how bias reduction or fairness will be quantitatively evaluated. Consequently, it is difficult to assess whether a target is achievable. While the mentioned white-box testing aligns with transparency and explainability, the definition does not specify how fair decisions will be communicated to end-users or stakeholders. Fairness is a complex indicator aligned with both SDG 5 (Achieve gender equality and empower all women and girls) and SDG 10 (Reduce inequality within and among countries). Furthermore, it can also refer to the providers’ policies regarding their employees and the network’s policies concerning end-users. To operationalize this KVI, fairness assessments should rely on a standardized survey instrument administered to a stratified panel of stakeholders. Inter-rater reliability should be verified through established coefficients such as Krippendorff’s alpha, and the panel composition should satisfy the cross-cultural diversity requirement described in Section 3.4.

5.1.3. E4 Open Collaboration

The definition of the KVI E4 Open Collaboration in the ADROIT6G deliverable [26] reads as follows:

• Relevance: Use methods based on collaboration and knowledge sharing.
• Aim: To foster open collaboration and knowledge sharing.
• How to measure it: Through clustering and joint activities with other ongoing relevant 6G projects in the 5G-PPP/SNS JU, technology clusters, associations, DIH, and the 6G-IA, reporting such practices and measuring their level of engagement.

Although the provided definition of this indicator highlights collaboration and knowledge sharing, it lacks specificity about the type of collaboration (e.g., technical, organizational, or strategic) and its intended outcomes. The definition should be refined by specifying the intended type and scope of collaboration, such as co-developing technologies, sharing research findings, or influencing policy and standards. Examples of quantitative metrics include the number of collaborations, co-published papers, co-developed standards, and active participation in knowledge-sharing platforms. Where quantitative metrics are unavailable or insufficient, qualitative assessments should follow the validation protocol outlined in Section 3.4. Periodic auditing across evaluation cycles is particularly relevant here, as collaboration patterns may shift significantly over time in response to policy changes or the emergence of new standards bodies.

5.2. An Overall Analysis of the KVI Framework in Hexa-X

The KVI framework developed in the Hexa-X [19] project distinguishes three Key Values: Sustainability, Inclusiveness, and Trustworthiness. The Sustainability KV is primarily related to energy consumption optimization and the reduction of unnecessary signaling. The Inclusiveness KV encompasses support for workers to enhance the quality of work (e.g., facilitating remote work, improving the usability of digital work tools) and to reduce spatial barriers (e.g., expanding the population covered by essential services, supporting elderly healthcare). Finally, the Trustworthiness KV involves privacy preservation as well as the availability and reliability of services. The KVIs listed in the public Hexa-X documents are typically linked with their enablers; however, there are no details about their implementation, and their measurement may be interpreted in multiple ways.

5.3. Applying the Framework to a KVI: An Illustrative Example

The following example illustrates how the proposed framework can guide the progressive refinement of a KVI through the four development phases. It is worth noting that, as discussed in Section 4, the mapping between criteria and phases is not rigid: while some criteria are primarily evaluated in each phase, the actual progression depends on the nature of the KVI and the choices made by those implementing it within the network. The example in Table 3 reflects one possible evaluation of the D2 Fairness (Section 5.1.2) indicator from ADROIT6G as a starting point, as its published definition provides a concrete but incomplete basis for development.

Table 3. Adequacy evaluation of the ADROIT6G Fairness KVI against the proposed criteria. S = Satisfied; PS = Partially Satisfied; NS = Not Satisfied.

Criterion	Rating	Notes
1. Specific	NS	Terms like “social justice” and “de-biasing” are not precisely defined; the boundaries of the indicator are unclear.
2. Measurable	PS	White-box testing and PoC-based verification are mentioned, but no measurement scale, threshold, or aggregation method is defined.
3. Attainable	NS	No discussion of feasibility with respect to available resources or constraints.
4. Relevant	S	The definition clearly identifies the objective: bias-free AI/ML decisions in 6G systems.
5. Time-bound	NS	No temporal reference or review cycle is specified.
6. General	S	Within the scope of the project, the definition applies across 6G deployment scenarios and stakeholder categories.
7. Sensitive	NS	No indication of how the indicator reacts to changes in training data or model updates.
8. Robust	NS	Not addressed.
9. Accurate	NS	No procedure to minimize bias in the measurement process itself.
10. Complete	NS	Sources of bias beyond training data (e.g., model architecture, post-processing) are not considered.
11. Conservative	NS	Not addressed.
12. Consistent	NS	No indication of comparability across measurement cycles.
13. Transparent	NS	Criteria for interpreting results are not specified.
14. Integrate	NS	No reference to interoperability or composability with other indicators.
15. Explainable	PS	White-box testing implies some interpretability, but how results are communicated to stakeholders is not specified.
16. SDG Comprehensive	–	Not applicable to a single KVI.

Elicitation. At this stage, the SMART criteria are applied. The definition identifies a relevant objective (bias-free AI/ML decisions) and is specific enough to anchor the indicator to a measurable phenomenon. However, it is not yet time-bound: no review cycle or deadline is specified, but whoever defines the indicator should at least question whether it can have a time reference. Measurability is only partially addressed, as white-box testing and PoC-based verification are mentioned, but no measurement scale, threshold, or aggregation method is defined. Attainability cannot be assessed without knowing the resources available for data collection and expert evaluation. The outcome of this phase is a refined formulation: the percentage of AI/ML decisions free from demographic bias, measured quarterly through standardized fairness metrics (e.g., equalized odds or demographic parity) applied to PoC datasets.

Analysis. The indicator is general within the 6G context, applying across deployment scenarios and stakeholder categories. Sensitivity should be assessed after measurability: the indicator should react measurably to changes in training data composition or model updates. Integration requires that the fairness metrics adopted be interoperable with other KVIs, for instance, sharing demographic data with other indicators to avoid redundant data collection. SDG comprehensiveness is evaluated for the whole set of KVIs. Thus, the SDG matrix can be created to identify the target goals of each indicator. Specifically, Fairness maps to SDG 5 (Gender Equality) and SDG 10 (Reduced Inequalities), filling a gap in the set. During the analysis, KVI interactions should be considered to better evaluate integration and SDG comprehensiveness. The outcome of this phase is a structured indicator with defined scope, sensitivity requirements, interoperability constraints, and SDG alignment documented in the KVI-to-SDG Matrix.

Realization. The indicator is implemented and validated against real or simulated data. Accuracy requires minimizing bias in the measurement process itself: the stratified panel described in Section 3.4 is applied. Completeness is assessed: the definition must account for all relevant sources of bias, including not only training data but also model architecture and post-processing steps. A trade-off emerges here between Explainable and Accurate: more sophisticated fairness metrics (e.g., counterfactual fairness) are more accurate but harder to communicate to non-technical stakeholders. A solution is to adopt equalized odds as the primary metric for operational monitoring and to reserve more complex metrics for periodic expert audits. Conservative assessment requires avoiding overestimating fairness improvements under uncertainty. The outcome of this phase is a validated indicator with a defined measurement pipeline, a primary operational metric, and explicit documentation of the trade-off between accuracy and explainability.

Assessment. Under real operating conditions, robustness is verified by testing whether the indicator remains stable across different demographic distributions and network configurations. Transparency requires that the measurement methodology, dataset composition, and panel criteria be publicly documented. Explainability is addressed by providing stakeholders with human-readable summaries of fairness compliance, as described in Section 3.2: an LLM-assisted reporting function can generate plain-language explanations of KVI values and trends. During this phase, the loop of Figure 3 may be triggered if fairness definitions evolve in response to new regulations, requiring the indicator to be revised and re-propagated through the Intent-Fulfillment layer. The outcome of this phase is a fully operational indicator that is publicly documented, explainable to non-technical stakeholders, and embedded in the network’s continuous monitoring loop. As discussed in Section 4.6, the progression illustrated above is not rigid. In this example, the conflict between Accurate and Explainable, first identified during Realization, is resolved by adopting a simpler primary metric and reserving more complex ones for expert audits. The conflict between Sensitive and Robust, anticipated during Analysis, is revisited during Assessment by verifying stability across demographic distributions. Criteria that remain only partially satisfied at the end of a phase, such as Sensitive in Technical Realization, are explicitly carried forward and addressed in the subsequent phase, as reflected in the heatmap of Table 4.

Table 4. Evolution of adequacy ratings for the Fairness KVI across the four development phases. S = Satisfied; PS = Partially Satisfied; NS = Not Satisfied.

Criterion	Elicitation	Analysis	Realization	Assessment
1. Specific	PS	S	S	S
2. Measurable	PS	PS	S	S
3. Attainable	PS	S	S	S
4. Relevant	S	S	S	S
5. Time-bound	PS	S	S	S
6. General	S	S	S	S
7. Sensitive	NS	PS	PS	S
8. Robust	NS	NS	NS	S
9. Accurate	NS	NS	PS	S
10. Complete	NS	NS	PS	S
11. Conservative	NS	NS	PS	S
12. Consistent	NS	NS	NS	S
13. Transparent	NS	NS	NS	S
14. Integrate	NS	PS	S	S
15. Explainable	NS	NS	PS	S
16. SDG Comprehensive	NS	PS	PS	S

6. Discussion

AI-native networks, including IBNs, enable autonomous network configuration, allowing real-time changes based on parameters inferred from requests, including natural-language ones. Digital twins and network digital twins support autonomous monitoring and intervention based on predictions of network demand and behavior. KVIs can be measured directly from the network (sensor-based KVIs) or derived from certifications (certified KVIs) [30]. In the case of sensor-based indicators, these can be measured in real-time directly by the provider to whom the metric relates: for example, in the case of measuring the percentage of energy generated from renewable sources, the metric will be an aggregate of the measurements of the devices or groups of devices that produce energy (e.g., wind turbines, solar panels, gas power plant, etc.). In the case of certified KVIs, however, the measures require time-consuming evaluations and subjective assessments involving expert judgment. For example, a certifier can verify the existence and application of some company-wide standards. Indeed, one of the major challenges in sustainable networking is the lack of standardization.

The KVIs proposed so far meet only a few of the criteria we have proposed and thus require further analysis and definition. Given their maturity and their coupling with the network’s functionalities, some KVs have the premises for a more detailed definition. However, it is important to establish a venue or forum where private and public organizations can contribute to defining KVIs through an interdisciplinary approach and initiate standardization processes. These indicators need to be standardized and made usable by the network so that it can define configurations suited to the needs of its stakeholders. For this reason, we envision that indicators must be certified by external entities and made integrable into the network via Application Programming Interfaces (APIs). The certifying bodies themselves will have to provide, publish, and maintain these interfaces. This approach offers transparency and credibility, as external evaluations reduce bias from internal assessments.

Limitations

Despite its generality, the proposed framework presents several limitations. First, a degree of subjectivity is unavoidable. Criteria such as relevance, explainability, and SDG comprehensiveness depend on stakeholder interpretation and domain expertise. Even when structured evaluation procedures are adopted, different evaluators may reach different conclusions, which can affect reproducibility. Second, the framework may introduce scalability challenges. Evaluating a large set of KVIs across complex, distributed networks requires significant effort, especially when iterative refinement and multi-stakeholder validation are involved. This issue is exacerbated in dynamic environments where indicators must be continuously monitored and updated. Third, the assessment of several criteria depends on data availability and quality. In many practical scenarios, complete or reliable data may not be accessible, limiting the evaluation of properties such as accuracy, completeness, and sensitivity. This is particularly relevant for indicators relying on external certifications or qualitative inputs. Fourth, the criteria are not independent and may introduce trade-offs. For example, improving explainability may require simplifying models or indicators, potentially reducing accuracy or completeness. Similarly, increasing generality may conflict with specificity, making indicators less informative in specific contexts. The framework does not prescribe a universal strategy to resolve these conflicts, leaving prioritization to stakeholders. Fifth, the framework assumes the existence of at least partially defined KVIs. However, as highlighted in the literature, KVI definitions are often incomplete or heterogeneous. As a result, the proposed approach may require additional preliminary effort to formalize indicators before adequacy can be meaningfully evaluated. Finally, the integration of AI-based support introduces additional risks. While AI models can assist in tasks such as indicator definition, mapping to SDGs, or explanation generation, they may also introduce bias, lack transparency, or produce unreliable outputs if not properly validated. Ensuring trustworthy AI integration requires governance mechanisms that are outside the scope of this work.

7. Conclusions

The paper highlights the importance of developing a comprehensive framework for assessing the adequacy of KVIs in network systems. It proposes an iterative method that encompasses the elicitation, analysis, realization, and assessment phases to ensure that KVIs remain aligned with stakeholders’ goals, including the SDGs. Additional criteria, such as the integration, explainability, and comprehensiveness of the SDGs, are emphasized to improve the adequacy of indicators. In addition, the paper analyzes the KVI frameworks developed in the Adroit6G and Hexa-X projects, highlighting strengths and areas for improvement. Finally, it acknowledges that some SDGs are difficult to integrate directly into networks, suggesting the use of certified indicators to measure providers’ commitment to supporting these goals. Future work should investigate methods to ensure the reliability, traceability, and auditability of AI-assisted KVI assessment, especially in scenarios involving subjective or composite indicators. Additionally, the definition of KVIs specifically targeting AI sustainability dimensions, such as training and inference energy consumption, bias auditing, and lifecycle sustainability of AI models, represents a natural direction for future work to which the proposed framework can be directly applied.