Applying Integrated Delphi–AHP to Maintenance Competency Prioritization in Industry 4.0: A Formally Specified Group Decision Framework with Consistency and Sensitivity Diagnostics

Abstract

As Industry 4.0 transforms manufacturing operations, maintenance organizations face a group decision-making problem: how to consolidate diverse expert judgments into a defensible, transparent ranking of the competencies that maintenance personnel most need. This paper applies an integrated Delphi–AHP framework—with explicit notation, operators, and diagnostics—to prioritize maintenance competencies in advanced-manufacturing settings. The Delphi stage consolidates expert-generated items under median–interquartile-range consensus and round-to-round stability rules, while the Analytic Hierarchy Process (AHP) transforms validated pairwise comparisons into ratio-scale priority weights through geometric-mean Aggregation of Individual Judgments (AIJ) and eigenvector derivation. Consistency screening (CI/CR), inter-rater agreement (Kendall’s W), and perturbation-based sensitivity analysis accompany the resulting weight vector. A bounded AI-assisted consistency-check step supports terminology harmonization during Delphi statement consolidation, subject to explicit human-validation constraints. A panel of fifteen industry experts participated in the study; five competency dimensions and twenty-nine indicators were retained through three Delphi rounds. AHP weighting identified Basic Knowledge and Skills as the highest-priority dimension, followed by Safety and Regulation Awareness and Problem-Solving Ability. Aggregated pairwise comparison matrices, local and global weights, and sensitivity results are reported to support reproducibility. The study contributes a rigorously specified application of combined Delphi–AHP to a domain—Industry 4.0 maintenance asset management—where multi-criteria decision analysis has seen limited formal application, and closes common specification gaps in published Delphi–AHP implementations.

1. Introduction

1.1. Background and Motivation

Group decision-making problems arise when a set of alternatives or criteria must be prioritized based on qualitative judgments collected from multiple experts. Such problems are common in Multi-Criteria Decision Analysis (MCDA), where the challenge is to transform subjective pairwise comparisons and textual statements into a reproducible priority vector, while controlling for inconsistency, aggregation bias, and stability across iterative rounds [1]. Two widely used tools in this context are the Delphi method for structured consensus formation and the Analytic Hierarchy Process (AHP) for deriving ratio-scale weights from pairwise comparisons [2,3,4,5].

Despite their broad adoption, practical implementations often under-specify the mapping from raw expert inputs to the final weight vector. Delphi synthesis can be sensitive to terminology and statement consolidation, and AHP results can be affected by inconsistency and the choice of aggregation operator [3,6]. These issues motivate a formulation that makes the operators, diagnostics, and acceptance rules explicit so that the full pipeline can be audited and reproduced—particularly in applied domains where MCDA has not yet been deployed with full procedural transparency.

In settings that emphasize complex socio-technical criteria, consensus modeling and weighting pipelines are frequently combined with complementary decision tools. Representative applications include sustainable selection frameworks [7], workforce and competency indicator development [8,9,10], and broader assessment problems in energy, sustainability, and urban systems [11,12,13,14,15]. These examples illustrate that the value of MCDA lies not only in producing a ranking, but also in making the path from expert input to final priorities transparent and auditable.

1.2. Application Context: Maintenance Competency in Industry 4.0

The application domain considered in this paper is maintenance-competency prioritization in advanced-manufacturing settings—a context that aligns with the Industry 4.0 and asset-management themes of contemporary MCDA research. Maintenance work in modern manufacturing facilities combines technical knowledge, safety-critical procedures, fault diagnosis under uncertainty, cross-functional coordination, and continuous learning, all under real operational constraints [16,17].

As manufacturing transitions toward cyber-physical systems, IoT connectivity, predictive maintenance, and digital twins, the competency requirements for maintenance personnel are evolving rapidly [18,19,20,21,22]. Several studies have examined how Industry 4.0 reshapes maintenance roles: Dhalmahapatra et al. [23] identified skills gaps in safety-critical maintenance under smart manufacturing conditions, while Mourtzis et al. [24] proposed a framework for operator training in human–robot collaborative maintenance environments. Recent work on AI-enhanced robotics for autonomous inspection and predictive maintenance in smart manufacturing [25] reinforces the position that competency requirements at the maintenance–automation boundary are evolving in tandem with the underlying digital infrastructure. Pintelon and Parodi-Herz [17] argue that maintenance should be treated as a core business process rather than an overhead function, reinforcing the need for systematic competency assessment.

Despite this growing recognition, no published study has applied a fully specified Delphi–AHP framework—with explicit aggregation operators, consistency diagnostics, and sensitivity analysis—to prioritize maintenance competencies for Industry 4.0 operations. The present study addresses this gap by demonstrating how combined MCDA techniques can support evidence-based workforce development in an asset-management context, a topic that is directly relevant to safety at work, Industry 4.0 readiness, and maintenance assessment systems.

1.3. Related Work and Specification Gap

Delphi–AHP combinations are widely used to elicit, screen, and weight criteria in applied decision problems [2,3,4,5]. To substantiate the claim that procedural specification is often incomplete, we examined four recent Delphi–AHP studies across four evaluation dimensions: (a) explicit consensus/retention thresholds, (b) identification of the group aggregation operator, (c) consistency handling procedure, and (d) sensitivity or robustness analysis.

Table 1. Procedural transparency in four representative Delphi–AHP studies. A check mark (✓) indicates the procedural detail was explicitly reported; a dash (–) indicates it was absent or described only narratively without formal specification.

Study	Consensus Thresholds	Aggregation Operator	Consistency Handling	Sensitivity Analysis
Liu et al. (2023) [3]	✓	–	✓	–
Abdul et al. (2024) [4]	–	–	✓	–
Yildiz & Ozkan (2024) [5]	✓	–	✓	✓
Dang & Nguyen (2021) [2]	–	–	✓	–
Present study	✓	✓	✓	✓

summarizes the findings.

As Table 1 shows, while consistency checking (CR $\le 0.10$) is near-universal, the aggregation operator used to synthesize group judgments—a critical choice that affects the mathematical properties of the resulting matrix [26,27]—is rarely stated. Consensus retention thresholds and sensitivity analyses are reported in some but not all studies. None of the four studies provides the complete procedural chain—from consensus rules through aggregation operator specification to robustness diagnostics—in a single, auditable workflow.

Recent reviews emphasize both the breadth of AHP usage and the importance of transparent model construction and validation [1,28]. Building on this foundation, the present work formalizes the end-to-end mapping from expert text and pairwise comparisons to a normalized priority vector, together with agreement and robustness diagnostics. The intent is not to introduce a new MCDA method, but rather to demonstrate a rigorously specified application of combined Delphi–AHP—with all operators and thresholds made explicit—in a domain where such transparency has been lacking.

1.4. Problem Statement and Objectives

Let k experts provide (i) free-text statements that must be consolidated into a finite item set and (ii) pairwise comparisons to weight the resulting criteria. The problem is defining a transparent mapping (text inputs, pairwise comparisons) $\mapsto w\in {\Delta }^{n-1}$, where w is a normalized priority vector on the simplex, subject to consensus and stability criteria for item retention and consistency diagnostics for the comparison matrices. Any preprocessing used for statement harmonization must be explicitly bounded so that it cannot introduce items or alter weights without human acceptance. Within this scope, the study seeks to (a) specify Delphi item-consolidation operators under median–IQR consensus and stability rules; (b) formulate AHP prioritization with geometric-mean group aggregation (AIJ) and standard consistency diagnostics (CI/CR), with theoretical justification for the aggregation choice; (c) specify agreement and stability checks (Kendall’s W and perturbation-based sensitivity); and (d) demonstrate the full computation pipeline through a worked application to maintenance-competency prioritization in advanced manufacturing, reporting all intermediate and final results for reproducibility.

1.5. Significance and Contributions

This study makes two complementary contributions:

• Application contribution: It presents the first formally specified application of integrated Delphi–AHP with full diagnostics to Industry 4.0 maintenance-competency prioritization—a domain where MCDA has been underutilized despite the inherent multi-criteria nature of workforce assessment in asset-intensive operations.
• Procedural contribution: It closes common specification gaps in published Delphi–AHP implementations by making the complete procedural chain explicit: consolidation and retention rules, group aggregation operator (with theoretical justification), normalization, consistency screening, and agreement/robustness reporting. A bounded AI-assisted consistency-check step for terminology harmonization is formalized with explicit constraints and expert-panel validation.

The combination of a new application domain with rigorous procedural specification distinguishes this work from both method-development papers (which typically use illustrative examples) and application papers (which often compress procedural details into narrative descriptions).

1.6. Scope, Limitations, and Paper Organization

The mathematical workflow is general and can be applied to other group prioritization settings. The application uses an expert panel from an advanced-manufacturing maintenance context in Taiwan, which may limit external generalization of the example-specific weights, but does not limit the generality of the framework specification.

The remainder of this paper is organized as follows. Section 2 introduces notation and the mathematical framework, including Delphi consensus/stability metrics, AHP weighting with group aggregation, theoretical justification of the aggregation operators, diagnostic measures, and algorithmic summaries. Section 3 presents the application to maintenance-competency prioritization, reporting Delphi results, AHP weights, agreement, and sensitivity outcomes with full numerical detail. Section 4 discusses interpretation, implications, and limitations. Section 5 concludes with a summary of contributions and directions for future work.

1.7. Methodological Rationale

The Delphi method provides a structured mechanism for iterative consensus formation on an item set, while AHP provides a ratio-scale prioritization method with established consistency diagnostics [28]. The choice of geometric-mean aggregation (AIJ) is grounded in the Aczél–Saaty theorem [26], which establishes that the geometric mean is the unique aggregation function preserving the reciprocal property of pairwise comparison matrices. The bounded AI-assisted consistency-check step is treated as preprocessing to reduce terminology-induced variance during consolidation; it is explicitly constrained and does not replace expert acceptance. The combined pipeline falls within the scope of multi-criteria decision analysis and is well-suited to the group decision-making and combined-technique themes of contemporary MCDA research.

2. Mathematical Framework and Methods

2.1. Notation and Problem Formulation

Let K denote the number of experts. Let $\mathcal{I}=\{1,\dots ,n\}$ index the retained items (criteria or indicators) after consensus formation. For each expert $k\in \{1,\dots ,K\}$, let $A^{\left(k\right)}={\left[a_{ij}^{\left(k\right)}\right]}_{i,j\in \mathcal{I}}$ denote a positive reciprocal pairwise-comparison matrix, where $a_{ij}^{\left(k\right)}>0$, $a_{ij}^{\left(k\right)}=1/a_{ji}^{\left(k\right)}$, and $a_{ii}^{\left(k\right)}=1$.

The goal is to compute a priority vector $w\in {\mathbb{R}}^n$ with $w_i>0$ and ${\sum }_{i=1}^n{w}_i=1$, together with diagnostic quantities that characterize: (i) matrix consistency for each $A^{\left(k\right)}$ and the aggregated matrix, (ii) agreement across experts, and (iii) robustness of the resulting priorities under small perturbations of the top-level weights.

2.2. Delphi Consensus and Stability Metrics

In the Delphi stage, items are retained based on a consensus rule defined on a bounded rating scale. For each item $i\in \mathcal{I}$ in round r, let $x_{i,r,1},\dots ,x_{i,r,K}$ denote the expert ratings. Define the median $m_{i,r}$ and interquartile range ${\mathrm{IQR}}_{i,r}$. The retained-item condition used in this study is

$$m_{i,r}\ge 4,{\mathrm{IQR}}_{i,r}\le 1,$$

on a five-point scale. Stability across consecutive rounds is checked via a max-median-change rule

$$\underset{i\in \mathcal{I}}{max}\left|m_{i,r}-m_{i,r-1}\right|\le 0.5,$$

which terminates iteration once met.

2.3. AHP Prioritization and Group Aggregation

• Pairwise Comparison Matrix

For each level of the hierarchy, a pairwise comparison matrix A is constructed by each expert $k\in \{1,\dots ,K\}$. The expert assigns a numerical value $a_{ij}$ representing the relative importance of element i over element j using Saaty’s 1–9 scale [28,29]. The matrix is defined as   

$$A^{\left(k\right)}=\left[a_{ij}^{\left(k\right)}\right]=\left[\begin{array}{cccc}1& a_{12}^{\left(k\right)}& \dots & a_{1n}^{\left(k\right)}\\ 1/a_{12}^{\left(k\right)}& 1& \dots & a_{2n}^{\left(k\right)}\\ ⋮& ⋮& \ddots & ⋮\\ 1/a_{1n}^{\left(k\right)}& 1/a_{2n}^{\left(k\right)}& \dots & 1\end{array}\right]$$

(1)

where $a_{ij}>0$, $a_{ij}=1/a_{ji}$, and $a_{ii}=1$.

• Aggregation of Group Judgments (AIJ)

To synthesize individual expert matrices into a single consensus matrix, the Aggregation of Individual Judgments (AIJ) method is used. The geometric mean aggregates individual entries while preserving the reciprocal property:

$$a_{ij}^G={\left({\displaystyle \prod _{k=1}^K}{a}_{ij}^{\left(k\right)}\right)}^{1/K}$$

(2)

where $a_{ij}^G$ is the aggregated group judgment for the pair $(i,j)$ and K is the total number of experts.

• Priority Vector and Consistency Check

The priority weights vector w is derived by solving the eigenvalue problem on the aggregated group matrix $A^G$:

$$A^{G}w={\lambda }_{max}w$$

(3)

where ${\lambda }_{max}$ is the maximum eigenvalue and w is the normalized eigenvector corresponding to ${\lambda }_{max}$. The reported AHP priority vector is

$$w_i={\displaystyle \frac{v_i}{{\sum }_{j=1}^n{v}_j}},i=1,\dots ,n,$$

(4)

so that $w\in {\Delta }^{n-1}$, where v denotes the principal eigenvector of $A^G$.

To ensure the reliability of the judgments, the Consistency Ratio ($CR$) is calculated:

$$CI={\displaystyle \frac{{\lambda }_{max}-n}{n-1}},CR={\displaystyle \frac{CI}{RI}}$$

(5)

where $CI$ is the Consistency Index, n is the matrix dimension, and $RI$ is the Random Index for matrices of size n [28]. A threshold of $CR\le 0.10$ is enforced.

2.4. Theoretical Justification of the Aggregation Operators

The choice of aggregation operator is not arbitrary; it has precise mathematical consequences for the properties of the resulting group matrix.

• The Aczél–Saaty Theorem

Aczél and Saaty [26] proved that the geometric mean is the unique aggregation function satisfying the following conditions for positive reciprocal matrices: (i) separability (the aggregated entry for pair $(i,j)$ depends only on the individual entries for that pair), (ii) unanimity (if all experts agree, the group judgment equals the common value), and (iii) reciprocal preservation ($a_{ij}^G·a_{ji}^G=1$). The arithmetic mean violates condition (iii), because in general ${\overline{a}}_{ij}·{\overline{a}}_{ji}\ne 1$ when individual matrices differ. This theoretical result is the basis for choosing AIJ with geometric-mean aggregation throughout this study.

• Eigenvector Method Validity

Saaty [29] demonstrated that for a perfectly consistent matrix ($a_{ij}=w_i/w_j$ for all $i,j$), the principal eigenvector recovers the true weight vector exactly. For near-consistent matrices (small $CR$), the eigenvector remains a rank-preserving estimator of the underlying priorities. The $CI/CR$ diagnostic quantifies the departure from perfect consistency, and the $CR\le 0.10$ threshold is a widely accepted convention [28].

• AIJ Versus AIP

Forman and Peniwati [27] distinguish two group aggregation strategies: Aggregation of Individual Judgments (AIJ), which aggregates pairwise entries before computing a single group eigenvector, and Aggregation of Individual Priorities (AIP), which first computes individual eigenvectors and then aggregates the resulting weight vectors. AIJ is appropriate when the group acts as a synergistic unit seeking a single consensus matrix, which matches the present study design in which experts contribute to a shared competency assessment. AIP is more appropriate when individuals retain separate identities and only their final priorities are combined.

• Compensatory Nature and Distinction from Weighted-Sum Methods

AHP is a compensatory method: strong performance on one criterion can offset weak performance on another within the aggregated priority vector. This property differs from outranking methods (e.g., ELECTRE, PROMETHEE) that avoid full compensation. The eigenvector-based weight derivation used in AHP also differs fundamentally from the simple weighted-sum method (WSM), which assigns weights directly to criteria scores. In AHP, weights emerge from ratio-scale pairwise comparisons rather than being assigned a priori, and the consistency check guards against arbitrary or contradictory judgments. The sensitivity analysis in Section 3 partially addresses the compensatory concern by testing whether small perturbations in top-level weights induce rank reversals.

• Independence Assumption and Scope

AHP does not require strict statistical independence of the criteria; it requires that the criteria be hierarchically decomposable, that is, that priorities can be derived within each cluster of the hierarchy and aggregated across clusters under the assumption that clusters interact only through their shared parent [28]. This is a weaker condition than full independence and it is the condition the present specification satisfies. In this study the five competency dimensions retained through the Delphi rounds were specifically constructed to be cluster-level groupings: each dimension is a coherent competency theme, and the indicators within a dimension are conceptually closer to one another than to indicators in other dimensions. Cross-dimension dependencies at the indicator level (for example, between Problem-Solving Ability and Equipment Operation and Maintenance when diagnosing faults in cyber-physical systems) are not modeled by AHP and constitute a known scope boundary of the present framework. Network-based MCDA methods such as the Analytic Network Process (ANP) and DEMATEL relax this scope: ANP supports priority derivation under arbitrary dependence structures, while DEMATEL produces dependence-strength matrices that can be used to test whether a given criterion set is decomposable in the first place. We treat ANP and DEMATEL extensions as future work (Section 5.2) and we acknowledge the assumption explicitly in the Limitations subsection (Section 4.5).

2.5. Agreement and Sensitivity Diagnostics

• Kendall’s Coefficient of Concordance

Agreement among experts is evaluated using Kendall’s W. Let m denote the number of raters and n the number of ranked items. If $R_i$ is the sum of ranks assigned to item i and $\overline{R}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{R}_i$, then

$$W={\displaystyle \frac{12{\sum }_{i=1}^n{(R_i-\overline{R})}^2}{m^2(n^3-n)}},$$

(6)

with $W\in [0,1]$ and larger values indicating stronger agreement.

• Perturbation-Based Sensitivity

Sensitivity is evaluated by perturbing the top-level dimension weights by a small relative factor ($\pm 10\%$ in this study), renormalizing the perturbed weights to the simplex, and recomputing global priorities to quantify rank stability under bounded shocks. This reporting convention is consistent with robustness measurement approaches used in comprehensive evaluation settings [30].

2.6. AI-Assisted Consistency Check Protocol

To improve traceability during statement consolidation in the Delphi process, an AI tool (Google Gemini 3.1 Pro) was used as a bounded consistency-check aid. The role of this step was threefold:

• Ambiguity Detection: Flagging vague or inconsistent wording across expert statements.
• Terminology Harmonization: Proposing neutral and standardized phrasing for panel review.
• Coverage Check: Identifying potentially underrepresented themes for expert reconsideration.

The workflow followed an expert–AI–expert loop (Algorithm 1): (1) expert input collection, (2) AI-generated consistency suggestions, (3) researcher screening, and (4) expert-panel confirmation. No item was added, removed, or reweighted by AI alone. All final decisions were made by the human panel under predefined consensus criteria, and an audit trail of AI-assisted revisions was maintained for reproducibility.

Algorithm 1 AI-assisted consistency check protocol

Require:  Raw Expert Inputs $E=\{e_1,e_2,\dots ,e_n\}$ Require:  AI text-consistency tool M Ensure:  Harmonized Item List C 1:  Step 1: Input Aggregation 2:      Compile raw inputs E from Delphi Round 1. 3:  Step 2: AI Analysis 4:      Prompt M: “Analyze E for semantic duplicates and ambiguous terms. Propose standardized labels.” 5:      Receive AI Suggestions S. 6:  Step 3: Human Review 7:      Researchers review S for domain accuracy. 8:      Filtered suggestions $S^{\prime }$ are presented to Expert Panel. 9:  Step 4: Panel Validation 10:     Experts accept, reject, or modify $S^{\prime }$. 11:     Finalize Item List C. 12: return  C

2.7. Workflow Overview and Algorithmic Summary

The overall workflow consists of four phases: (1) protocol design and item elicitation, (2) Delphi consensus rounds for item retention under the median–IQR rule and stability checks, (3) AHP-based weighting using pairwise comparisons with consistency screening, and (4) synthesis and validation of the resulting priority hierarchy. The bounded AI-assisted consistency-check step is used only during statement consolidation to harmonize terminology and flag ambiguity; it does not introduce items, make acceptance decisions, or compute weights.

Algorithms 2 and 3 provide pseudocode representations of the Delphi and AHP procedures, outlining each step to support inspection and reproducibility.

Algorithm 2 Delphi consensus protocol with stability check

Require:  Expert panel; Initial item pool; Max rounds $R_{max}$ Require:  Consensus rule: Median $\ge 4$ and $\mathrm{IQR}\le 1$ (5-point scale) Require:  Stability rule: Max per-item median change $\le 0.5$ between rounds Ensure:  Final item set organized into decision-hierarchy dimensions 1:  Preparation: Draft items from literature; pilot wording; revise. 2:  Round 1 (Qualitative): Collect open-ended inputs; code, merge, and clarify items. 3:  for  $r=2$  to  $R_{max}$  do 4:       Administer Likert survey on current items. 5:       For each item, compute Median and $\mathrm{IQR}$. 6:       Mark items meeting consensus; edit, merge, or drop remainder via feedback. 7:       Provide anonymized distributions and synthesized comments to panelists. 8:       Compute max change in per-item medians relative to previous round. 9:       if all items meet consensus or stability rule is satisfied then 10:           break loop 11:      else 12:           Refine items and proceed to next round. 13:      end if 14: end for 15: Organize final items into hierarchical dimensions for AHP. 16: return Final items and dimension assignments.

Algorithm 3 AHP prioritization with consistency check and expert aggregation

Require:  Decision criteria (dimensions) and sub-criteria (indicators); Expert panel Require:  Consistency Ratio threshold $CR\le 0.10$; Max 2 revisions per matrix Ensure:  Dimension weights $W_D$, Indicator weights $W_I$, Global priorities $W\left(i\right)$ 1:   Level 1: Dimension Weighting 2:   for each expert do 3:         Construct pairwise comparison matrix (Saaty 1–9 scale). 4:         Compute Eigenvector and Consistency Ratio ($CR$). 5:        while $CR>0.10$ and revisions $<2$ do 6:              Revise most inconsistent judgments; recompute. 7:         end while 8:   end for 9:   Aggregate individual matrices via geometric mean (AIJ) to obtain group matrix. 10:  Compute group eigenvector to obtain $W_D$. 11:  Level 2: Indicator Weighting 12:  for each dimension d do 13:        Construct pairwise matrix for indicators within d. 14:        Compute local weights and $CR$. 15:        while $CR>0.10$ and revisions $<2$ do 16:              for each expert do 17:                    Adjust inconsistent entries; recompute weights and $CR$. 18:              end for 19:        end while 20:        Aggregate indicator weights across experts to form $W_I$ for dimension d. 21:  end for 22:  Global Priority and Robustness Analysis 23:  for each indicator i in dimension d do 24:        Compute global weight: $W\left(i\right)=W_D\left[d\right]\times W_I\left[d\right]\left[i\right]$ 25:  end for 26:  Rank all $W\left(i\right)$ values; compute Kendall’s W for agreement. 27:  Conduct sensitivity analysis (perturb $W_D$ by $\pm 10\%$). 28:  return $W_D,W_I,W\left(i\right)$.

3. Application: Maintenance Competency Prioritization

3.1. Context, Expert Panel, and Data Collection

The application reported in this paper is based on an advanced-manufacturing maintenance-competency elicitation study. Fifteen experts were purposively selected, including senior maintenance engineers, operations managers, and academic researchers with experience in advanced-manufacturing operations. Selection criteria required at least ten years of relevant professional experience and domain expertise. The panel produced the Delphi ratings and AHP matrices used in the analysis.

Data collection coincided with the iterative Delphi and AHP stages. Qualitative responses from Round 1 were analyzed thematically to identify emergent competency indicators, while quantitative responses in Rounds 2–3 were evaluated using descriptive statistics (mode, mean, and standard deviation) to assess consensus levels. Pairwise comparison data from the AHP phase were synthesized and validated for consistency before deriving the final weights. Ethical principles were strictly observed, including informed consent, voluntary participation, confidentiality of expert identities, and secure data management. Ethical clearance was obtained from the relevant institutional review authority.

3.2. Delphi Consensus Results

The Delphi process comprised three iterative rounds:

• Round 1: Open-ended questions were distributed to elicit potential competencies and performance indicators. Redundant responses were removed, and conceptually similar statements were consolidated into an initial list. A bounded AI-assisted consistency check was used only to flag semantic overlap and ambiguous wording during this consolidation; it did not add items or change any weights without expert acceptance.
• Round 2: Experts rated the importance of each item using a five-point Likert scale. Median and interquartile range (IQR) statistics were computed to evaluate agreement and identify contentious items.
• Round 3: Feedback summaries were provided to participants, who re-evaluated items based on the group consensus. Items meeting the retention criteria (median $\ge 4$, IQR $\le 1$) were retained as final indicators.

In Round 1, the expert panel produced an initial pool of 45 candidate items. After consolidation and panel validation, 29 retained items remained, organized into five dimensions:

• Basic Knowledge and Skills;
• Professional Literacy;
• Equipment Operation and Maintenance;
• Safety and Regulation Awareness;
• Problem-Solving Ability.

Round 3 confirmed stability under the max-median-change criterion ($max|\Delta m|\le 0.5$).

Table 2. Delphi rating outcomes for representative indicators (retention rule: median $\ge 4$, IQR $\le 1$). Human-readable labels are shown with original codes.

Competency Indicator (Code)	Mode	Mean	SD
Equipment startup procedures (A-1-1)	5	4.67	0.62
Tooling changeover and setup (A-1-2)	5	4.67	0.49
Basic fault isolation (A-1-3)	5	4.53	0.64
Preventive maintenance scheduling (A-2-1)	5	4.67	0.62
Safety lockout/tagout compliance (B-2-1)	5	4.67	0.49
Incident reporting and lessons learned (D-2)	5	4.73	0.46

summarizes key statistical outcomes for representative competency indicators.

The items retained through Delphi (exemplified in Table 2) form the leaf nodes of the AHP hierarchy. Each item belongs to exactly one of the five dimensions listed above, which serve as the top-level criteria in the subsequent AHP analysis. Figure 1 shows the resulting three-level decision hierarchy.

3.3. AHP Prioritization Results

Following the Delphi phase, AHP was applied to determine the relative importance of each competency dimension. Experts completed pairwise comparison matrices using the Saaty 1–9 scale. Individual matrices were screened for consistency ($CR\le 0.10$); matrices exceeding the threshold were revised (at most two guided revisions per expert). The aggregated expert judgments were processed using Expert Choice software (version 11.5; Expert Choice Inc., Arlington, VA, USA) and verified against manual eigenvector computations.

Table 3. Aggregated pairwise comparison matrix for competency dimensions (geometric mean of $K=15$ expert matrices). BK = Basic Knowledge and Skills; PL = Professional Literacy; EOM = Equipment Operation and Maintenance; SRA = Safety and Regulation Awareness; PS = Problem-Solving Ability.

	BK	PL	EOM	SRA	PS
BK	1.00	3.24	2.05	1.35	1.78
PL	0.31	1.00	0.68	0.51	0.58
EOM	0.49	1.47	1.00	0.72	0.86
SRA	0.74	1.96	1.39	1.00	1.25
PS	0.56	1.72	1.16	0.80	1.00

presents the aggregated pairwise comparison matrix for the five top-level dimensions, obtained by computing the geometric mean of all 15 individual expert matrices (AIJ method).

Table 4. AHP dimension weights and priority ranking. Group-matrix consistency diagnostics: ${\lambda }_{max}=5.143$, $CI=0.036$, $CR=0.032$ (threshold $CR\le 0.10$; $RI=1.12$ for $n=5$). All fifteen individual expert matrices satisfied $CR\le 0.10$ after at most one guided revision.

Competency Dimension	Local Weight	Rank
Basic Knowledge and Skills (BK)	0.330	1
Safety and Regulation Awareness (SRA)	0.228	2
Problem-Solving Ability (PS)	0.185	3
Equipment Operation and Maintenance (EOM)	0.154	4
Professional Literacy (PL)	0.103	5

reports the eigenvector-derived dimension weights, consistency diagnostics, and priority ranking computed from the aggregated matrix in Table 3.

The results show that Basic Knowledge and Skills carries the highest priority weight (0.330), followed by Safety and Regulation Awareness (0.228) and Problem-Solving Ability (0.185). All individual expert matrices and the aggregated group matrix satisfied the consistency threshold of $CR\le 0.10$.

3.4. Agreement and Sensitivity Results

• Agreement (Kendall’s W)

For each expert k, the AHP procedure induces a dimension-weight vector and therefore an implied ranking over the top-level dimensions. Using the rank-based formulation defined in Section 2, Kendall’s coefficient of concordance was computed as $W=0.72$ ($p<0.001$, chi-square test), indicating moderate-to-strong agreement among the fifteen panelists on the relative ordering of competency dimensions. Individual-level pairwise matrices are not reproduced in full to preserve panel confidentiality.

• Sensitivity (Bounded Perturbation)

Sensitivity was evaluated as a stability diagnostic by applying a bounded relative perturbation of $\pm 10\%$ to each top-level dimension weight, renormalizing the perturbed vector to the simplex, and recomputing the resulting ranks.

Table 5. Bounded sensitivity diagnostic. Each row shows the post-renormalization dimension weights and the resulting rank order when the indicated dimension’s baseline weight is perturbed by the indicated relative shock. Baseline row uses the AHP weights from Table 4.

Perturbed	Shock	BK	SRA	PS	EOM	PL	Rank Order	$\mathbf{\Delta }$Rank
–	baseline	0.330	0.228	0.185	0.154	0.103	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
BK	$-10\%$	0.307	0.236	0.191	0.159	0.107	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
BK	$+10\%$	0.351	0.221	0.179	0.149	0.100	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
SRA	$-10\%$	0.338	0.210	0.189	0.158	0.105	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
SRA	$+10\%$	0.323	0.245	0.181	0.151	0.101	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
PS	$-10\%$	0.336	0.232	0.170	0.157	0.105	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
PS	$+10\%$	0.324	0.224	0.200	0.151	0.101	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
EOM	$-10\%$	0.335	0.231	0.188	0.141	0.105	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
EOM	$+10\%$	0.325	0.225	0.182	0.167	0.101	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
PL	$-10\%$	0.333	0.230	0.187	0.156	0.094	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0
PL	$+10\%$	0.327	0.226	0.183	0.152	0.112	BK ≻ SRA ≻ PS ≻ EOM ≻ PL	0

reports the post-renormalization weights and rank order for each of the ten single-dimension perturbation scenarios. The rank order $\mathrm{BK}\succ \mathrm{SRA}\succ \mathrm{PS}\succ \mathrm{EOM}\succ \mathrm{PL}$ is preserved in all ten scenarios, with no rank changes in any position. We follow the reviewers in noting that bounded single-dimension perturbation is a stability diagnostic rather than a strong robustness claim: the gap between the top weight (0.330) and the second weight (0.228) is large enough that a $\pm 10\%$ shock cannot, by construction, invert the top rank. The reported diagnostic therefore documents the absence of rank inversions in the lower three positions, where the weight gaps are smaller and the perturbation has the most room to act.

3.5. Workflow Summary and Table Integration

The tables presented in this section follow a sequential workflow that traces the complete pipeline from item retention through prioritization to operational interpretation:

• Delphi retention (Table 2): Items were screened using the median–IQR consensus rule. The 29 retained items, exemplified by the six representative indicators in Table 2, were organized into five competency dimensions forming the top level of the AHP hierarchy.
• AHP aggregation (Table 3): The pairwise comparison matrices of 15 experts were aggregated via geometric mean (AIJ), producing the group consensus matrix shown in Table 3.
• AHP prioritization (Table 4): Eigenvector derivation from the aggregated matrix yielded the dimension weights and consistency diagnostics reported in Table 4.
• Operational mapping (

  Table 6. Competency linkage to operational KPIs and resource-efficiency indicators. Note. The “Expected operational/resource effect” entries are interpretive linkages between the competency dimensions and asset-management KPI categories synthesized from the cited maintenance literature; they are not direct empirical effect sizes from the present study.

  Competency Dimension	Indicative Indicators (Examples)	Expected Operational/ Resource Effect
  Basic Knowledge and Skills	Setup stability; standard work adherence; parameter control	Fewer adjustments and retries; reduced scrap and energy waste
  Equipment Operation and Maintenance	Unplanned downtime (%); MTTR; first-time-fix; PM compliance	Higher asset utilization; lower waste and rework; improved yield stability
  Safety and Regulation Awareness	Lockout/tagout compliance; incident rate; risk-assessment adherence	Fewer safety incidents; regulatory compliance; reduced intervention risk
  Problem-Solving Ability	Root-cause identification rate; diagnostic accuracy; escalation frequency	Faster fault recovery; fewer repeat failures
  Professional Literacy	Documentation quality; cross-shift handover; communication effectiveness	Faster containment and recovery; knowledge transfer across teams

  ): The weighted dimensions were mapped to indicative operational KPIs and resource-efficiency indicators (Table 6) to support practical interpretation.

This sequential structure ensures that each table builds on the preceding one, and that the complete path from expert input to operational interpretation is traceable and auditable.

3.6. Framework Validation and Operational KPI Mapping

The competency set derived from Delphi and AHP was validated through combined quantitative and qualitative assessment. Experts evaluated clarity, applicability, coverage, and operational relevance using structured Likert-scale instruments. Scores were consistently high (generally above 4.5 on a 5-point scale), indicating strong acceptance of the framework across panel members. Qualitative feedback further supported practical usability, particularly the integration of technical, safety, and cross-functional competencies.

To support interpretation of how competency development can affect operational outcomes, each competency dimension was mapped to resource-efficiency-relevant indicators. Table 6 provides the linkage.

4. Discussion

4.1. Interpretation for Industry 4.0 Asset Management

The prioritization results indicate that maintenance competency in Industry 4.0 settings is driven by five mutually reinforcing dimensions rather than isolated technical skills. Basic Knowledge and Skills emerged as the highest-priority dimension (weight 0.330), reflecting the operational reality that fundamental technical understanding—equipment startup procedures, parameter control, and standard work adherence—forms the necessary foundation for all downstream maintenance activities. Without this base, more advanced competencies cannot be exercised effectively.

This finding is not the trivial restatement that fundamentals matter; it is consequential in the Industry 4.0 context specifically. A common narrative in the I4.0 literature anticipates that as cyber-physical systems and predictive analytics absorb diagnostic work, the relative weight of advanced data-interpretation and digital-tool skills should rise above fundamentals. The panel reaches the opposite conclusion in advanced manufacturing: cyber-physical environments amplify rather than diminish the operational cost of fundamental errors, because a misread parameter or a missed lockout step propagates faster and farther in a connected system than in a standalone machine, and because the Industry 4.0-to-5.0 transition reintroduces the operator as the human-in-the-loop decision authority for adaptive maintenance [16]. The implication for training resource allocation runs counter to the prevailing narrative: organizations should not deprioritize fundamentals when investing in digital-tool training, because the marginal value of fundamentals increases rather than decreases under cyber-physical operating conditions.

Safety and Regulation Awareness ranked second (0.228), consistent with the safety-critical nature of advanced-manufacturing operations where lockout/tagout procedures, hazard identification, and regulatory compliance are non-negotiable requirements. Problem-Solving Ability ranked third (0.185), capturing the diagnostic and troubleshooting capabilities needed to address novel fault conditions in automated and cyber-physical systems. The remaining dimensions—Equipment Operation and Maintenance (0.154) and Professional Literacy (0.103)—are complementary but received lower priority weights, suggesting that the panel views hands-on operational skills and documentation/communication competencies as necessary but relatively less differentiating.

These results have direct implications for asset management in Industry 4.0 environments. Organizations can align recruitment screening with the highest-weight dimensions to improve role fit. Training curricula can be structured around the weighted competency hierarchy to allocate resources where they have the greatest expected impact on maintenance performance and reliability. Performance management can use the indicator set as a transparent, data-driven rubric for periodic evaluation.

4.2. Role of AI-Assisted Preprocessing

The bounded AI-assisted consistency-check step (Algorithm 1) served a specific and limited role: reducing terminology-induced variance during Delphi statement consolidation. Ambiguous definitions were flagged and alternative phrasings were returned to the panel for human validation. The preprocessing step was governed by explicit constraints—it supported wording harmonization and coverage checking but did not replace expert decisions, introduce new items, or compute weights. This design is consistent with emerging guidance on responsible AI integration in decision-support systems, where AI serves as an aid within human-controlled workflows rather than as an autonomous decision-maker.

We also acknowledge that large language models can carry domain-specific biases, including in advanced engineering and reliability terminology, and that uncritical use of an LLM in expert elicitation would be inappropriate. Two safeguards mitigate this risk in the present design. First, the AI tool returns wording suggestions only; researchers screen the suggestions for domain accuracy before they reach the expert panel; the panel then accepts, rejects, or modifies each suggestion under the same consensus rule applied to all Delphi items (Algorithm 1, step 4). The chain of acceptance is therefore expert-driven, with the AI serving as a flag-only aid. Second, an audit trail of AI-suggested labels and panel-final wording was maintained: a qualitative review of this trail showed that the panel modified or rejected a substantial proportion of AI-proposed labels, indicating that the panel exercised independent judgment rather than deferring to the AI. We do not claim that AI harmonization improved the Delphi process over traditional qualitative coding; we claim only that AI was used as a bounded aid with clear human authority, and that the bias risk was managed through the validation step rather than through trust in the AI itself. Generalization of this protocol to other LLMs, domains, or panel compositions remains an open question and is noted in the Section 4.5.

4.3. Comparison with Prior Delphi–AHP Implementations

As documented in Table 1 (Section 1.3), prior Delphi–AHP studies typically report consistency checking but often omit the identity of the aggregation operator, explicit consensus thresholds, and sensitivity analysis. The present study addresses all four dimensions of procedural transparency by (a) specifying the median–IQR retention rule with an explicit stability criterion, (b) identifying the geometric-mean AIJ operator with theoretical justification (Aczél–Saaty theorem), (c) enforcing the standard $CR\le 0.10$ threshold with a documented revision protocol, and (d) reporting perturbation-based sensitivity analysis alongside Kendall’s W for agreement. This level of specification enables other researchers to replicate and audit the complete workflow.

4.4. Practical Implications

The quantitative hierarchy offers three direct applications for maintenance managers:

• Recruitment Screening: The high weight of Basic Knowledge and Skills indicates that role-specific fundamentals should be assessed at hiring through targeted evaluations reflecting the operating context, procedures, and documentation practices.
• Training Resource Allocation: Since Safety Awareness and Problem-Solving ranked highly, training plans should allocate comparable emphasis to hazard control and diagnosis practice, rather than treating safety as a secondary topic. Structured feedback methods drawn from related learning-science work—for example, video-based feedback approaches developed for skill acquisition in other domains [31]—offer a transferable design pattern for procedural-skill rehearsal in maintenance training programs.
• Standardized Evaluation: The 29 indicators can serve as a weighted checklist for performance reviews, supporting a data-driven approach to competency scoring and skill-gap analysis.

The framework also supports the integration of new digital tools—predictive analytics, IoT-based monitoring, and AI-driven diagnostics—by ensuring maintenance technicians possess the adaptive learning and diagnostic abilities needed in smart manufacturing environments. Connecting competency indicators to operational KPIs such as MTTR, unplanned downtime, first-time-fix rate, and rework-related waste (Table 6) allows organizations to evaluate whether competency interventions are associated with measurable changes in reliability, safety, and resource efficiency over time.

4.5. Limitations

The study is limited by its expert panel composition and application context. The panel comprised fifteen experts drawn predominantly from a Taiwan advanced-manufacturing maintenance setting. A panel of this size sits within the 10–30 range commonly used for purposive Delphi panels in MCDA studies, which is appropriate for structured expert elicitation but does not by itself support a generalized Industry 4.0 competency framework. The reported dimension and indicator weights should therefore be read as example-specific weights for the Taiwan advanced-manufacturing maintenance context, not as universal weights for I4.0 maintenance; the framework specification (notation, operators, diagnostics, and pseudocode in Section 2.1, Section 2.2, Section 2.3, Section 2.4, Section 2.5 and Section 2.6) is the part proposed as transferable.

Two further limitations bound the methodological claims. First, AHP requires hierarchical decomposability of the criteria rather than full independence, and we acknowledge that cross-dimension dependencies at the indicator level are not modeled by the present framework. Network-based MCDA methods (ANP, DEMATEL) are appropriate when interaction-rich criteria must be modeled and are listed as concrete extensions in Section 5.2. AHP also remains a compensatory method, so strong performance on one criterion can offset weak performance on another within the aggregated priority vector—a property that may not be appropriate for all decision contexts. Second, the bounded AI-assisted preprocessing step was applied using a single AI tool (Google Gemini 3.1 Pro), and the generalizability of the harmonization outcomes to other LLMs, different domain terminologies, or different expert panel compositions was not systematically tested.

5. Conclusions

5.1. Summary and Contributions

This paper applied an integrated Delphi–AHP framework to prioritize maintenance competencies in Industry 4.0 advanced-manufacturing settings, targeting a domain where multi-criteria decision analysis has seen limited formal application. A panel of fifteen experts participated in a three-round Delphi process that retained twenty-nine indicators across five competency dimensions, followed by AHP-based weighting with full consistency, agreement, and sensitivity diagnostics. The highest-priority dimension was Basic Knowledge and Skills, followed by Safety and Regulation Awareness and Problem-Solving Ability.

The study makes two complementary contributions:

• Application contribution: It demonstrates that integrated Delphi–AHP with formal diagnostics is effective for maintenance-competency prioritization in Industry 4.0—providing evidence-based guidance for workforce development, training allocation, and performance evaluation in asset-intensive operations.
• Procedural contribution: It closes common specification gaps in published Delphi–AHP implementations by making the complete procedural chain explicit at the operator level: consolidation and retention rules, geometric-mean group aggregation with theoretical justification, consistency screening, and agreement/robustness reporting. A bounded AI-assisted consistency-check step is formalized as preprocessing for terminology harmonization, with explicit constraints and expert-panel validation.

5.2. Limitations and Future Work

The primary limitation lies in the regional scope of the expert panel and the single application domain. Future studies should replicate and validate the framework across different industrial, cultural, and regulatory contexts to enhance generalizability. The rapid pace of technological advancement in automation and cyber-physical systems also necessitates regular updates to the competency model.

Future work can extend the present framework in several directions: (a) replacing AHP with the Analytic Network Process (ANP) to model arbitrary dependence among competency dimensions, or pairing AHP with DEMATEL or Interpretive Structural Modeling (ISM) to first quantify dependence-strength matrices and then derive weights only over decomposable clusters—this directly addresses the criterion-coupling concern raised during peer review; (b) applying fuzzy AHP or hesitant fuzzy sets to capture uncertainty in expert judgments more explicitly; (c) comparing alternative aggregation strategies (e.g., AIP vs. AIJ) and their effects on the resulting priority structure; and (d) evaluating longitudinally whether competency-based interventions informed by the resulting priorities are associated with measurable changes in operational performance. Cross-cultural and cross-industry replication with expert panels from different manufacturing regions would also strengthen the evidence base for the framework’s transferability.