A Digitalized Quality-Management Framework and Automation-Ready Compliance Architecture for Cybersecurity Testing Laboratories: An ISO/IEC 17025:2017 Crosswalk and Exploratory Case Study

Featured Application

A case-based governance, evidence-traceability, and automation-ready compliance blueprint for ISO/IEC 17025-accredited cybersecurity testing laboratories supporting IEC 62443 assurance activities and EU RED/CRA conformity evidence.

Abstract

Cybersecurity testing laboratories must produce auditable conformity evidence while operating with rapidly changing toolchains, conditional requirements, and qualitative PASS/FAIL/INCONCLUSIVE outcomes. ISO/IEC 17025:2017 is widely used to demonstrate laboratory competence, yet its operationalisation in cybersecurity testing remains under-specified for software- and tool-driven security assessments. This paper separates an architectural contribution from an empirical contribution. The architectural contribution is a digitalized quality-management framework and automation-ready compliance architecture that translate ISO/IEC 17025 clauses into cybersecurity-specific artefacts, decision rules, controlled toolchains, evidence bundles, and review workflows. The empirical contribution is an exploratory single-laboratory case study based on unpublished, anonymised, and confidentiality-constrained laboratory artefacts: an ETSI TS 103 701 workbook with 68 provision-level test groups, including 41 claimed/applicable rows for ambiguity analysis; an IEC 62443 corrective-action plan; and ISO/IEC 17025 governance records. Within this case, structured decision rules and evidence traceability reduced the Conformity Statement Ambiguity Index from 0.976 to 0.049 and converted 37 previously INCONCLUSIVE provisions into PASS determinations. These results are reported as descriptive within-case evidence only; they do not establish predictive validity or cross-laboratory generalisability. The study contributes a clause-to-artefact crosswalk, a concrete evidence-traceability architecture, and candidate cyber-maintenance indicators for future multi-laboratory validation.

1. Introduction

Cybersecurity has become an operational and regulatory concern for industrial and quality-driven environments [1,2]. Uptime, availability, and incident response are no longer determined only by mechanical or electrical failures [3,4]; connected industrial assets can also experience cybersecurity events that behave like unplanned downtime. This makes cybersecurity assurance relevant to maintenance-quality management and to the evidence practices used by testing laboratories.

At the same time, product cybersecurity requirements are increasingly embedded in certification schemes and regulations, including the Radio Equipment Directive Delegated Act [5,6,7] and the Cyber Resilience Act [8]. These developments require laboratories to produce conformity evidence that is repeatable, auditable, and supported by transparent decision rules. The detailed regulatory background is therefore given in Section 1.2, while this introduction focuses on the research gap.

ISO/IEC 17025 [2] is widely used to demonstrate laboratory competence, but cybersecurity testing stresses parts of the standard that are less visible in classical measurement laboratories: qualitative verdicts, rapidly changing tools, conditional requirements, and expert interpretation of dynamic systems. In this setting, a technically correct test is not sufficient unless the method baseline, evidence package, review authority, and decision rule are also controlled.

Existing LIMS-centred digitization and compliance-as-code approaches improve record control, schema validation, and machine-readable control exchange, but they do not adequately address the accreditation-specific governance of qualitative cybersecurity conformity statements. In particular, they under-specify how reviewer authority, decision-rule ownership, evidence-bundle traceability, nonconforming work, and controlled report re-issuance should be managed inside an ISO/IEC 17025-accredited cybersecurity laboratory [9].

This paper addresses that gap through an exploratory single-laboratory case study and an automation-ready compliance architecture. The contribution is not a claim of fully automated conformity assessment; rather, it is a case-derived framework for structuring the governance objects, data flows, and evidence packages that make digitised cybersecurity conformity statements auditable. The remainder of the paper is structured as follows. Section 1.1 clarifies the research gap, scope, and contributions. Section 1.2 positions the work in relation to standards, certification schemes, and academic literature. Section 2 presents the materials and methods, including the case study design, artefact inventory, data analysis strategy, workflow blueprint, and reference compliance-as-code pattern. Section 3 reports the case study results, Section 4 discusses the findings and limitations, and Section 5 concludes the paper.

1.1. Research Gap, Scope, and Contributions

Analytically, the proposed framework is positioned against three reference approaches. Relative to conventional ISO/IEC 17025 [9] implementations in measurement-centric laboratories, it shifts the primary control objects from uncertainty budgets and instrument calibration alone to versioned toolchains, qualitative decision rules, and software-generated evidence. Relative to generic LIMS-centred laboratory digitalization, it extends record management with machine-readable requirement-to-evidence links, explicit review states, and amendment traceability for qualitative conformity statements. Relative to compliance-as-code and software-process automation approaches, it adds accreditation-specific control objects such as sign-off authority, nonconforming-work handling, and controlled report re-issuance. This analytical comparison clarifies the design space addressed by the case study and the reason why a digitised governance layer is needed in cybersecurity laboratories.

The specific contributions are: (i) an ISO/IEC 17025:2017 clause-to-artefact crosswalk tailored to cybersecurity testing laboratories and aligned with IEC 62443 [10,11] and ETSI EN 303 645 [12]; (ii) a digitised workflow, evidence-traceability model, and automation-ready compliance architecture that make the automation boundary technically explicit; (iii) a focused literature positioning and comparative analysis across accredited laboratory systems, machine-readable control frameworks, and automated compliance-checking approaches; (iv) an exploratory within-case indicator (CSAI) with exact descriptive statistics and sensitivity interpretation for ambiguity reduction between draft and reviewed assessments; and (v) lessons learned on how these governance controls can feed a cyber-maintenance assurance loop and candidate KPIs for connected industrial assets.

This work is explicitly framed as an exploratory single-laboratory case study conducted in an ISO/IEC 17025-accredited cybersecurity testing laboratory. The aim is not statistical generalisation across laboratories, but the extraction of operational patterns—governance artefacts, decision rules, and evidence structures—that can be examined in future multi-site studies. In particular, the study focuses on translating ISO/IEC 17025 clauses into cybersecurity-specific artefacts that remain auditable under accreditation constraints while remaining compatible with digitised, automation-ready workflows.

Accordingly, the study is guided by two research questions. RQ1 asks how ISO/IEC 17025 [13] requirements can be translated into cybersecurity-native artefacts and an automation-ready evidence workflow without weakening accreditation controls. RQ2 asks which governance mechanisms, within a real laboratory case, are associated with the reduction in ambiguous conformity outcomes between draft and reviewed assessments.

1.2. Background and Related Work

1.2.1. Maintenance Engineering, Quality, and Resilience

Classical maintenance engineering treats availability as an outcome of structured failure management: functions are defined, failure modes are analysed, and maintenance tasks are selected and periodically re-optimised according to consequences and the operating context. Reliability-centred maintenance (RCM) formalises this logic and is described in IEC 60300-3-11 as a guideline for developing failure management policies and maintenance programmes, including feedback and continuous improvement [14,15]. Risk-based maintenance and inspection extend the same principle by prioritising tasks through a probability–consequence view of failure and by allocating limited resources to the highest-risk assets first [16]. Within Maintenance 4.0, this planning is increasingly coupled with Industrial IoT data streams and analytics such as predictive maintenance and condition monitoring, although recent reviews also emphasise limitations related to data quality, model drift, and operational integration [17]. More recent Maintenance 5.0 discussions shift the emphasis toward resilience and recovery capacity, while adding sustainability and human-centricity as explicit design constraints [18]. The present study draws on this lineage by treating cybersecurity incidents as a form of unplanned downtime for connected industrial assets [19] and by interpreting laboratory governance controls—risk review, corrective actions, and evidence traceability—as maintenance-like signals.

1.2.2. Industrial Cybersecurity Standards: IEC 62443 Series

Industrial cybersecurity practice is frequently structured around the ISA/IEC 62443 series, which defines lifecycle requirements and stakeholder-specific responsibilities for securing Industrial Automation and Control Systems (IACSs), spanning governance, system design and component/product assurance. Within that series, IEC 62443-4-1 [20] specifies secure product development lifecycle requirements and explicitly covers activities that matter for post-market security sustainment, including defect handling, patch management, and end-of-life practices. Complementing this, IEC TR 62443-2-3 [21] addresses patch management in the IACS environment and describes requirements for asset owners and product suppliers who establish and maintain a patch management programme, an area where operational constraints (availability, safety, and legacy platforms) often make “IT-style” patching infeasible without compensating for the controls. In practice, these standards create a direct pull for cybersecurity testing laboratories, because demonstrating conformance (or supporting regulatory conformity evidence) requires repeatable, auditable test outcomes, and traceable handling of vulnerabilities and updates across product versions, conditions that are governance-heavy even when the underlying tests are highly technical.

1.2.3. EU Cyber Resilience Act and Regulatory Context

The EU Cyber Resilience Act (Regulation (EU) 2024/2847) establishes horizontal cybersecurity requirements for “products with digital elements” and explicitly frames the policy problem as widespread vulnerabilities and inconsistent security updates across the market [22]. European Commission implementation information indicates that the CRA entered into force on 10 December 2024; the main obligations apply from 11 December 2027; and reporting obligations apply earlier, from 11 September 2026 [23,24]. From a vulnerability-handling perspective, the CRA operationalises two evidence expectations that affect testing and assessment work: (i) manufacturers must handle vulnerabilities throughout the product lifecycle, and (ii) for actively exploited vulnerabilities and severe incidents, manufacturers must report via a Single Reporting Platform using short timelines (early warning within 24 h and full notification within 72 h, with a final report following corrective measures) [24]. This shifts conformity evidence away from a single “point-in-time” report toward a lifecycle argument: test evidence, vulnerability triage artefacts, update/support commitments, and reporting/communication records become part of what regulators and market surveillance authorities may expect to see as credible proof of ongoing compliance.

1.2.4. ISO/IEC 17025 in Cybersecurity Testing

ISO/IEC 17025 [2] is widely used to demonstrate laboratory competence, but cybersecurity testing stresses parts of the standard that are less dominant in classical measurement laboratories: qualitative outcomes, expert judgement, and rapidly changing toolchains. In such contexts, decision rules become central because conformity statements can no longer be defended by numerical uncertainty alone; instead, the laboratory must specify what evidence is sufficient, how exceptions are treated, and who carries the risk of false acceptances/rejections. ILAC G8:09/2019 provides practical guidance on how laboratories define, agree, and communicate decision rules when issuing statements of conformity, including the treatment of shared risk, uncertainty, and customer expectations [25]. In principle, this allows laboratories to use guard-band style reasoning to reduce false acceptance and false rejection risks where specification limits are involved. The TELIGENCIA Labs assessment summary illustrates why this matters in cybersecurity: the case laboratory used a contract review and reporting workflow with a dedicated decision-rule field, yet the assessment noted that clients were effectively “self-declaring” compliance via checklists in a way that is not equivalent to an ISO/IEC 17025 laboratory declaration and recommended alignment with ILAC G8.

A second recurring issue in non-traditional domains is the misfit of conventional “measurement validity” mechanisms when results are qualitative. In the case laboratory, the procedure for ensuring validity applied the normalised error (En) as a performance criterion [26], even though the cybersecurity results were qualitative (PASS/FAIL) and did not require uncertainty estimation; the same assessment record references a bilateral inter-laboratory comparison [9] that used PASS/FAIL as the evaluation criterion and also noted the absence of a participation plan for proficiency testing/comparisons. This points to a practical research gap: cybersecurity laboratories need fit-for-purpose inter-comparison designs, repeatability targets, and validity criteria that respect judgement-based methods while remaining auditable.

Finally, competence and method control are amplified in cybersecurity because “methods” include tools, configurations, and environments that evolve continuously. ENISA has published EUCC-related guidance describing how ISO/IEC 17025 is interpreted for IT Security Evaluation Facilities (ITSEFs), recognising that additional interpretation is needed when applying 17025 to security evaluation [27]. In the TELIGENCIA assessment, “points sensibles” explicitly recommend structured competence evidence (including training aligned with ISO/IEC 19896-3 [28] for security testers/evaluators) and retaining proper competence and test-domain expertise; the same record highlights document control weaknesses (revision status not systematically identified) and risk-rating method gaps (undefined scoring criteria), which are directly relevant to toolchain baselining, method validation/verification, and the traceability of cybersecurity test conclusions over time. To improve readability for readers without direct access to proprietary standards, clause numbers are paired throughout the paper with the practical control topic they denote (e.g., signatory authorization, report amendment traceability, or external service governance).

1.2.5. Digitalized Laboratory Systems and Compliance Automation

ISO/IEC 17025 explicitly requires laboratories to control data and information management (Clause 7.11) and to ensure technical records remain complete and traceable (Clause 7.5). As laboratories digitalize, these controls increasingly depend on software systems (e.g., LIMS, scripted data processing, versioned templates, and evidence repositories). EUROLAB has provided recent guidance on managing digitalized systems in ISO/IEC 17025-accredited laboratories, emphasising validation, access control, change management, and audit trails [13]. In parallel, compliance automation approaches based on machine-readable control catalogues and assessment results (e.g., NIST OSCAL) [29,30] enable policy-as-code, structured evidence exchange, and schema-level validation of compliance artefacts [30].

This perspective is consistent with the software-process compliance literature, which reports that effective automated compliance checking depends on explicit rule representations, process models, and traceable evidence structures rather than on the wholesale replacement of human judgement [31,32]. Ontology-based information-security compliance research similarly shows that formal control descriptions can support automated compliance determination while keeping the rationale inspectable to decision makers [33]. Empirical software engineering studies on test-automation maturity also suggest that disciplined automation practices are associated with better product quality and shorter release cycles, which reinforces the importance of controlled, reusable automation components in accredited cybersecurity testing workflows [34].

1.2.6. Quality-Management Positioning and Test-Automation Maturity

A recurring source of confusion in laboratory digitization projects is the assumption that ISO 9001-style quality digitalization is sufficient for an accredited cybersecurity testing laboratory. Metrology research treats ISO 9001 and ISO/IEC 17025 as complementary rather than interchangeable: ISO 9001 strengthens organisation-wide process control and customer focus, whereas ISO/IEC 17025 adds laboratory-specific requirements for technical competence, validated methods, defensible results, and auditable decision rules [35]. For cybersecurity testing, this distinction matters, because a generic QMS can digitalize forms and approvals without necessarily controlling toolchain baselines, sign-off authority, amendment traceability, or evidence sufficiency.

The software testing literature also provides a useful boundary condition for the present work. Test-automation maturity is associated with better product quality and shorter release cycles in continuous integration contexts [34], but those studies do not resolve accreditation-specific concerns such as nonconforming work, reviewer authority, controlled records, or externally provided services. The contribution of the present paper is therefore not a generic automation claim; it is an accreditation-aware governance model that shows how automation can be introduced without dissolving the laboratory controls that make conformity statements defensible.

1.2.7. Focused Literature Positioning Across Adjacent Automation Domains

To sharpen the positioning of the present study, this subsection provides a focused literature positioning across four adjacent domains: (i) accredited and digitalized laboratory quality-management systems, (ii) machine-readable control frameworks and compliance-as-code, (iii) automated compliance checking using process models, ontologies, NLP, or knowledge graphs, and (iv) formal verification and empirical test-automation quality research. It is not presented as a PRISMA-style systematic review; instead, it maps the design space most relevant to accredited cybersecurity testing laboratories and highlights the unresolved intersection addressed by this paper.

Table 1. Comparative positioning of representative related work streams and the gap addressed by the present study.

Approach Stream	Representative Sources	Primary Automation Focus	What It Supports Well	Limitation for Accredited Cybersecurity Labs
Accredited laboratory quality systems	[13,36,37]	Digital records, validation, audit trails, success factors	General ISO/IEC 17025 governance and system implementation	Usually not focused on cybersecurity toolchains, qualitative verdict logic, or machine-readable evidence links
Generic LIMS-centred digitization	[13,35,36]	Workflow, samples, records, approvals	Operational record control and audit trails	Often document-centric; limited treatment of decision rules, evidence bundles, and evolving test scripts as controlled methods
Software-process compliance checking	[32,41]	Rule checking against process models and trace links	Timely detection of missing QA steps and process deviations	Does not directly address accreditation sign-off, report issuance, or externally provided services
Ontology/NLP/semantic automated compliance checking	[33,38,39]	Formal control representation, semantic reasoning, text-to-rule mapping	Machine-assisted compliance determination and requirement extraction	Usually optimises rule matching, not laboratory method control or auditable reviewer responsibility
Machine-readable control frameworks/compliance-as-code	[30,42,43]	Structured control catalogues, schema validation, CI-supported compliance artefacts	Interoperable machine-readable plans, assessments, and evidence metadata	Strong on representation, weaker on how qualitative conformity judgements are governed under ISO/IEC 17025
Proposed framework	This paper	Accreditation-aware evidence architecture with explicit automation boundary	Links machine-readable artefacts, evidence bundles, review states, and ISO/IEC 17025 governance	Currently demonstrated as a single-laboratory exploratory case; predictive validity remains future work

summarises this comparison. The pattern is consistent across the reviewed streams: recent work offers strong automation primitives—e.g., schema-validated control artefacts, ontology-backed compliance reasoning, NLP-assisted clause extraction, or process-centric quality checks—but typically under-specifies accreditation-specific concerns such as reviewer authority, report amendment traceability, controlled toolchain change, and the treatment of qualitative PASS/FAIL/INCONCLUSIVE states [36,37,38,39,40,41]. The technical novelty claimed in this paper is therefore not a generic automation framework but an accreditation-aware evidence architecture that connects these automation primitives to ISO/IEC 17025 decision rules, technical records, and nonconforming-work controls in a cybersecurity laboratory.

2. Materials and Methods

2.1. Research Design

This work follows an exploratory case study research design based on accreditation-style implementation evidence from an operational cybersecurity testing laboratory. The empirical setting is TELIGENCIA Labs, a Tunisian Accreditation Council “TUNAC” International Laboratory Accreditation Cooperation Mutual Recognition Arrangement “ILAC-MRA” ISO/IEC 17025-accredited Tunisian based cybersecurity laboratory, affiliated to Teligencia Labs in Germany that conducts cybersecurity evaluation, conformity assessment, and related assurance activities for connected products and industrial systems.

During the cybersecurity testing activities, the following software and hardware tools were used: Wireshark version 4.2.4 (Wireshark Foundation, Beaverton, OR, USA), Burp Suite Community Edition version 2024.4.1 (PortSwigger Ltd., Knutsford, UK), Kali Linux version 2023 (Offensive Security, New York, NY, USA), Nmap version 7.95-2 (Insecure.Org, Lyon, France), SEGGER J-Link debugger (SEGGER Microcontroller GmbH, Monheim am Rhein, Germany), and Bus Pirate v3.6a (SparkFun Electronics, Boulder, CO, USA).

The unit of analysis is the laboratory’s unpublished and confidentiality-constrained governance and technical-record artefacts used to substantiate ISO/IEC 17025 conformity, complemented by anonymised project datasets aligned with ETSI EN 303 645 [12] and IEC 62443-related assurance activities. The study combines artefact analysis with a mixed quantitative–qualitative evaluation of assessment outcomes to identify how decision rules, method governance, and traceable records reduce ambiguity in conformity statements. The case is treated as analytically revelatory rather than statistically representative: its value lies in exposing operational patterns that can later be tested across multiple laboratories.

2.2. Data Analysis

2.2.1. Dataset Preparation and Anonymization

The case study dataset comprises (i) project-level cybersecurity assessment artefacts and (ii) laboratory governance artefacts used to demonstrate ISO/IEC 17025 conformity in a cybersecurity testing context. To preserve confidentiality, client/product identifiers were removed and replaced with neutral labels (e.g., Client A, Product Family A). Requirement identifiers (e.g., “5.3–11”) were retained because they are standard-defined and needed for traceability and reproducibility.

The analysed evidence included:

• ETSI TS 103 701 assessment workbook [44] (EN 303 645 aligned), including an Implementation Conformance Statement (ICS) view, an assessment view with verdicts per test case and per test group, and a reviewed “final review” version.
• IEC 62443 programme audit action plan: A nonconformity-driven improvement log (NCR-based) capturing findings and corrective actions related to methodology completeness, TRF/report integrity, and evidence traceability.
• ISO/IEC 17025 internal audit report: Internal audit scope, nonconformities, corrective actions, and closure dates used as a proxy for management-system “maintenance” responsiveness.
• Laboratory method and toolchain governance artefacts (ISO/IEC 17025 clause instantiations), including a method selection/verification/validation procedure, a method validation form, a controlled list of cybersecurity test methods, and a Used Computer Master List supporting controlled toolchains.

Together, these artefacts represent the “scope-design artefacts” listed in

Table 2. Inventory of analysed artefacts, analytical roles, and anonymization levels.

Analysed Artefact	Type	Period Covered	Role in Analysis	Anonymization Level
ETSI TS 103 701 workbook	Project-level conformity assessment workbook	2023–2025 case study snapshot; exact project dates suppressed	Verdict distribution, CSAI, transition matrix, and evidence-traceability subset	Client/product identifiers removed; standard requirement IDs retained
IEC 62443 programme action plan	Corrective-action and nonconformity log	2023–2025 case study snapshot; exact project dates suppressed	Qualitative analysis of governance failure modes, scope errors, report integrity, and evidence traceability	Organisation-specific names removed; NCR grouping retained
ISO/IEC 17025 internal audit report	Laboratory governance record	2024–2025 accreditation cycle; exact dates suppressed	Clause-linked analysis of governance drift, corrective actions, and management-system maintenance signals	Personal names removed; roles and clause topics retained
Method and toolchain governance artefacts	Controlled procedures, method lists, validation forms, and tool registers	2023–2025 controlled-document snapshot	Clause-to-artefact crosswalk and automation-ready architecture extraction	Internal identifiers generalised; sensitive customer and tool details removed

The dates are reported as ranges because exact project and client dates were suppressed during anonymization. The retained standard requirement identifiers allow traceability without exposing product or client identity.

referenced in the proposal and enable analysis across planning, execution, checking, and corrective-action stages.

2.2.2. Quantitative Analysis

Quantitative analysis focused on verdict stability, ambiguity reduction, and scope discipline. The ETSI assessment workbook provides a natural before/after structure, because it contains both a draft assessment state and a reviewed assessment state.

Unit of analysis (ETSI dataset): A “test group”/“provision” row (e.g., 5.6–3). Variables extracted included:

• Applicability/claim status (claimed: yes/no/blank for conditional not applicable);
• Requirement type (mandatory vs. recommended; conditional flags);
• Support indicator (evidence present vs. not);
• Verdict at test group level (PASS/FAIL/INCONCLUSIVE/NO = explicitly not claimed/NA = not applicable);
• Reviewer comment presence (notes and review columns).

The primary quantitative indicators computed were:

• Conformity Statement Ambiguity Index (CSAI): Measures how often a conformity statement cannot be made because decision rules and evidence are insufficient [45].
• Verdict transition counts (draft → reviewed): A transition matrix capturing how many provisions changed from INCONCLUSIVE/FAIL to PASS (or remained unresolved).
• Evidence traceability ratio (ETSI review items): Proportion of reviewer-flagged items where the final review explicitly references concrete evidence artefacts (e.g., document references, captures) [46].

Throughout the analysis, PASS and FAIL denote determinate conformity states; NO denotes explicitly unclaimed functionality; NA denotes conditional requirements that do not apply to the architecture under assessment; and INCONCLUSIVE denotes an applicable requirement for which the available evidence or the decision-rule boundary is not yet sufficient for a determinate statement. This distinction was preserved during coding and verdict transition analysis to avoid conflating scope exclusions with unresolved evidence gaps.

For the IEC 62443 audit action plan, the unit of analysis was an action item row. Rows were forward filled by NCR identifier (to group multi-row NCRs) and then categorised by theme (scope definition, TRF/report integrity, evidence traceability, and methodology/work instruction completeness). Counts were computed per NCR and per theme.

Because the ETSI workbook covers the full set of provisions assessed in this case rather than a random sample drawn for population inference, the quantitative analysis is primarily descriptive. To characterise the observed shift more rigorously, the analysis is supplemented with exact binomial confidence intervals, a paired baseline comparison, and local sensitivity calculations. CSAI is therefore treated as a within-case heuristic that visualises ambiguity reduction under explicit decision rules and improved evidence traceability rather than as a universally validated benchmark for all laboratories.

2.2.3. Qualitative Analysis

Qualitative analysis used directed thematic coding [47] aligned with ISO/IEC 17025 technical and management requirements and the proposed Assurance Maintenance Loop (Plan–Do–Check–Act) [48].

Two coding frames were applied:

ISO/IEC 17025 clause-oriented frame (operationalisation lens):

• Method governance and validation depth (method selection, verification, validation, and deviations);
• Technical records and traceability (what evidence supports each verdict, how it is referenced);
• Reporting integrity and decision rules (how verdicts are derived and stated);
• Nonconforming work/corrective action (how gaps are tracked and closed).

Assurance Maintenance Loop frame (maintenance lens):

• Plan: Scope definition, requirement selection, decision rules, and milestones;
• Do: Controlled execution, toolchain baselines, and evidence collection;
• Check: Peer review, consistency checks, and re-evaluation triggers;
• Act: Corrective actions, template updates, and governance refinements.

Reviewer notes in the ETSI workbook and NCR texts in the 62443 action plan were coded to identify recurring causes of ambiguity and to characterise which PDCA stage produced (or resolved) them.

Coding was conducted using a directed procedure. The corresponding author performed the initial coding because of direct familiarity with the laboratory artefacts; the two co-authors then reviewed the coding categories and the interpretation of representative excerpts. Disagreements were resolved through discussion until consensus was reached. No personal or client-identifying text was retained in the coding examples. Illustrative examples include reviewer notes requesting more detailed evidence, which were coded under technical records and evidence traceability; NCR statements concerning ambiguous certification scenarios, which were coded under scope definition and plan-stage control; and findings on report numbering or re-issuance, which were coded under reporting integrity and amendment traceability.

2.2.4. Within-Case Baseline, Statistical Descriptors, and Sensitivity Analysis

The quantitative comparison is anchored in a within-case baseline design: the draft ETSI assessment state is treated as the observational baseline and the reviewed state as the post-review comparator. For CSAI, we use the descriptive formulation CSAI = I/C, where I is the number of INCONCLUSIVE applicable/claimed provisions and C is the number of applicable/claimed provisions for which a laboratory statement is sought. To improve transparency, we report exact binomial confidence intervals for the observed proportions and recode the paired states into ambiguous versus determinate verdicts (PASS or FAIL) for an exact McNemar comparison between draft and reviewed assessments.

A simple local sensitivity analysis is also applied to the candidate indicators. For a fixed denominator C, a single unresolved provision changes CSAI by 1/C; likewise, for the reviewed-comment subset, one additional evidence-linked item changes the evidence-traceability ratio by 1/R, where R is the number of reviewer-flagged items. These perturbation calculations do not establish predictive validity, but they show how stable—or unstable—the indicators are under small changes in case-level evidence. Predictive validation against laboratory failure rates, reopened reports, or downstream vulnerability outcomes is reserved for future longitudinal studies with larger datasets.

Algorithm 1 summarises the calculation of CSAI and the transition matrix. The procedure treats NA and NO as scope states rather than unresolved evidence states; only applicable/claimed rows are used as the denominator for CSAI.

Algorithm 1. CSAI and verdict transition calculation.

Input: provision rows p = 1...68; draft_verdict[p]; reviewed_verdict[p]; applicable_claimed[p]. Initialise C = 0; I_draft = 0; I_review = 0; transition_count[(x,y)] = 0. For each provision p: transition_count[(draft_verdict[p], reviewed_verdict[p])] + = 1. If applicable_claimed[p] is true: C + = 1; if draft_verdict[p] = INCONCLUSIVE then I_draft + = 1; if reviewed_verdict[p] = INCONCLUSIVE then I_review + = 1. Return CSAI_draft = I_draft/C, CSAI_review = I_review/C, and transition_count.

2.3. Digitised Workflow Blueprint and Evidence Traceability Model

To make the automation claim technically explicit, the paper derives an automation-ready workflow blueprint from the case laboratory processes. Figure 1 summarises a reference architecture that links requirement sources (standards and regulations) to method and decision-rule catalogues, workflow orchestration, tool-controlled execution environments, evidence packaging, independent review, and report generation. The model focuses on evidence traceability (ISO/IEC 17025 Clauses 7.5 and 7.11) and repeatability under changing toolchains.

In the case laboratory, evidence packages were stored as immutable “evidence bundles” comprising (a) raw tool outputs (logs, reports, and screenshots); (b) tool configuration and version information; (c) the system-under-test identifier and input artefact versions (e.g., ICS/IXIT); (d) the applied decision-rule identifiers; and (e) a machine-readable manifest file (e.g., JSON) containing file hashes and metadata. A lightweight packaging script can generate this manifest automatically at the end of each test run and deposit it in a controlled repository, enabling auditors to trace each report statement back to specific evidence files and toolchain states [13,30].

Table 3. Workflow stages, digitised evidence outputs, and ISO/IEC 17025 clause anchors (case-derived blueprint).

Workflow Stage	Digitised/Automatable Elements (Evidence Outputs)	Manual/Expert Activities	ISO/IEC 17025 Clause Anchor
(1) Scope & test planning	Controlled test plan template; scope ticket; mapping to scheme requirements; versioned inputs (ICS/IXIT, SUT identification).	Define testing objectives and constraints; agree acceptance criteria; approve scope changes.	7.1, 7.2
(2) Method selection & decision rules	Versioned method list; decision-rule catalogue; traceable link between requirement, test method, and decision rule ID.	Select fit-for-purpose methods; define judgement criteria where automation is insufficient; approve decision-rule deviations.	7.2, 7.8.6
(3) Environment provisioning & toolchain control	Tool registry; tool version capture; configuration scripts; environment baselines; access control and change logs.	Approve toolchain updates; validate new tool versions; ensure competence for specialised tools.	6.2, 6.4, 7.11
(4) Test execution	Automated test scripts (where applicable); raw logs; timestamped results; linkage to SUT and configuration baseline.	Exploratory testing; interpret ambiguous tool outputs; manage safety and operational constraints during testing.	7.2, 7.5
(5) Evidence packaging & traceability	Evidence package (manifest + hashes); stored artefacts (logs, screenshots, configs); immutable identifiers for audit retrieval.	Select relevant evidence; ensure completeness and relevance; document rationale for exclusions.	7.5, 7.11, 8.4
(6) Technical review & nonconforming-work handling	Review checklist; reviewer comments; change-log between draft and reviewed verdicts; nonconformance records.	Peer review; resolve inconsistencies; decide on re-testing or additional evidence generation.	7.7, 7.10
(7) Report generation & issuance	Templated reporting; automatic insertion of verdict tables, evidence IDs, and toolchain metadata; controlled report versions.	Write interpretive narrative; sign-off; apply decision rules and communicate limitations to customers.	7.8, 7.8.6
(8) Governance feedback and continuous improvement	CAPA register; internal audit findings; management review minutes; metrics dashboard (e.g., CSAI).	Prioritise improvements; allocate resources; update procedures and competence plans.	8.7–8.9

makes the automation boundary explicit: Several steps can be digitised (and partially automated) through templates, versioned catalogues, toolchain registries, and structured evidence packages; however, qualitative judgement and risk acceptance remain necessary in method selection, interpretation of tool outputs, and final conformity statements.

2.4. Reference Compliance-As-Code Implementation Pattern

The workflow blueprint in Section 2.3 can be operationalised through a concrete compliance-as-code stack rather than through narrative document handling alone. A machine-readable option is to encode applicable requirements, profiles, component definitions, system security plans, assessment plans, assessment results, and plans of action in NIST OSCAL [30,49], which provides XML, JSON, and YAML representations for control-based compliance artefacts [30]. In a laboratory setting, this allows requirement mappings, evidence manifests, and assessment outcomes to move between authoring, validation, execution, and reporting tools without repeated manual transcription.

A practical authoring and governance layer can then be implemented with Git-based tooling such as Compliance Trestle, which is explicitly designed for the creation, validation, and governance of OSCAL artefacts in CI/CD pipelines [42]. In such a pattern, analysts’ author-scoping rationales, procedure metadata, and decision-rule identifiers in human-readable source files are created; CI jobs transform them into schema-valid OSCAL documents, check provenance, and preserve change history. This turns method updates, template revisions, and evidence-schema changes into controlled, reviewable records rather than informal document edits.

The execution layer can then connect controlled toolchains to configuration and policy validators such as Lula, which bridges expected compliant configuration to observed system state and can emit machine-readable validation results [43]. Raw outputs from scanners, captures, scripts, and review tools are deposited in an evidence repository, together with hashes, timestamps, input artefact versions, and tool versions. This does not imply full automation of conformity decisions: method selection, interpretation of complex outputs, sign-off authority, and residual-risk acceptance remain outside the automated core. However, the pattern makes scripts, APIs, data flows, and evidence traceability explicit enough to evaluate automation claims technically rather than rhetorically.

The present case study does not benchmark a full end-to-end deployment of this stack against a manual baseline. Instead, the architecture is used as a concrete reference implementation pattern that is compatible with the evidence bundles and governance artefacts observed in the case laboratory and that can be evaluated quantitatively in future multi-laboratory studies.

3. Results

3.1. Coverage of ISO/IEC 17025 Requirements with Cybersecurity Artefacts

The analysed artefact set indicates that ISO/IEC 17025 requirements can be instantiated into cybersecurity-specific records without forcing cybersecurity testing into a purely “measurement uncertainty” model. Key operationalisations observed in the case laboratory include:

• Method identification and control: A controlled List of Methods assigns internal identifiers to cybersecurity activities (e.g., ETSI/RED-aligned testing, IEC 62443 process/product assessments, and vulnerability testing/pentesting). This supports repeatability by anchoring each project to a defined method baseline rather than relying on informal “test approach” narratives.
• Method verification/validation governance: A formal method procedure explicitly requires validation depth proportional to changes in scope and deviations and recognises performance characteristics relevant to cybersecurity testing (e.g., robustness, repeatability/reproducibility, and uncertainty in results interpretation).
• Toolchain control: The presence of a Used Computer Master List and authorised usage records provides a lightweight mechanism to document test environment identity and software update state—critical in cybersecurity testing where scanner versions, firmware images, and tool configurations can change outcomes.
• Project workflow integration: The RED/ETSI/IEC 62443 work instruction includes explicit steps from application evaluation to conformity assessment, report packaging, and technical review/closeout. This creates a direct governance bridge between ISO/IEC 17025 management controls and cybersecurity assurance deliverables.

These artefacts collectively enable a clause-to-record “crosswalk” in practice: ISO/IEC 17025 requirements become concrete cybersecurity records (ICS/IXIT completion, evidence packages, controlled templates, review logs, and corrective actions).

3.2. ETSI TS 103 701 (EN 303 645 Aligned) Assessment Outcomes

3.2.1. Dataset Structure

The ETSI assessment workbook contains 68 provision-level test groups. For the draft assessment state, 41 rows were in the applicable/claimed assessment scope (40 INCONCLUSIVE and one FAIL), while 27 rows were classified as NA. During review, 22 rows remained NA, and five rows were clarified as NO (explicitly not claimed). The reviewed in-scope subset therefore still contained 41 rows (38 PASS, one FAIL, and two INCONCLUSIVE). This distinction separates applicability/scope classification from unresolved evidence gaps and prevents NA or NO rows from being counted as ambiguous conformity statements.

3.2.2. Draft vs. Reviewed Verdict Distributions

The draft assessment state exhibited a high reliance on INCONCLUSIVE outcomes, while the reviewed state produced a substantially larger share of PASS outcomes (

Table 4. Verdict distribution before and after review (ETSI assessment, n = 68 provisions).

Verdict Category	Draft Assessment	Reviewed Assessment
PASS	0	38
INCONCLUSIVE	40	2
FAIL	1	1
NO (explicitly not claimed)	0	5
NA (not applicable)	27	22
Total	68	68

and Figure 2).

When restricted to applicable/claimed provisions, the change is more pronounced:

Draft applicable/claimed subset (n = 41): 40 INCONCLUSIVE, one FAIL.

Reviewed applicable/claimed subset (n = 41): 38 PASS, one FAIL, and two INCONCLUSIVE.

This indicates that the review cycle and the associated evidence/decision-rule refinement converted many unresolved assessment states into determinate conformity outcomes.

3.2.3. Verdict Transition Analysis

A transition analysis from a draft to reviewed format shows the mechanism of improvement:

Items that were not applicable as mentioned in

Table 5. Verdict transition counts (draft → reviewed).

Transition	Count
INCONCLUSIVE → PASS	37
FAIL → PASS	1
INCONCLUSIVE → INCONCLUSIVE	2
INCONCLUSIVE → FAIL	1
NA → NA	22
NA → NO	5
Total	68

mostly remained not applicable (NA), with some clarified as explicitly not claimed (NO) cases during review.

This pattern indicates that ambiguity was primarily caused by documentation/evidence sufficiency and decision-rule clarity. The aggregate FAIL count remained one across draft and review, but the identity of the failing clause changed: the original draft FAIL became a PASS, while one previously INCONCLUSIVE item became a FAIL after scope and decision-rule correction.

3.2.4. Ambiguity Reduction Indicator

Using CSAI (INCONCLUSIVE rate among claimed/applicable items):

Draft CSAI: 40/41 = 0.976;

Reviewed CSAI: 2/41 = 0.049.

This reduction indicates a shift from an assessment state dominated by unresolved evidence to one capable of producing determinate conformity statements. The result is consistent with the interpretation that formal decision rules and controlled records can reduce ambiguity within the case dataset. To provide descriptive statistical context, exact binomial intervals place the draft CSAI at 0.976 (95% CI: 0.871–0.999) and the reviewed CSAI at 0.049 (95% CI: 0.006–0.165). Recoding the paired provision states into ambiguous versus determinate verdicts yields 38 ambiguity-resolving transitions and no reverse transitions, corresponding to an exact McNemar p-value < 1 × 10⁻¹⁰. This is only reported as evidence of a strong within-case shift between the draft and reviewed states, not as inferential proof beyond the case laboratory. The indicator is also sensitive to small changes: with C = 41 applicable/claimed provisions, each additional unresolved provision changes CSAI by 1/41 = 0.024. Accordingly, resolving the two remaining INCONCLUSIVE items would reduce CSAI to 0.000, whereas five additional unresolved items would raise it to 0.171. This behaviour confirms that CSAI is useful for tracking review maturity within a project, but too sensitive to support predictive claims about incidents or vulnerability escape rates without larger longitudinal datasets.

3.2.5. Reviewer Comments and Evidence Traceability

In the reviewed assessment, 12 provisions contained explicit reviewer notes requiring clarification (e.g., increased detail, missing evidence, or scope justification). In the final review notes for these items:

• Approximately ~75% referenced IXIT updates (documentation additions to support repeatable testing);
• Approximately ~83% referenced specific evidence artefacts (e.g., named documents, captures), indicating strengthened traceability.

Two items remained INCONCLUSIVE, because the review identified evidence gaps that were documented but not closed within the dataset snapshot, illustrating the governance value of preserving an INCONCLUSIVE verdict as a controlled outcome rather than forcing a pass/fail judgement.

One item became FAIL because a mandatory expectation was incorrectly treated as “not claimed,” demonstrating the importance of explicit decision rules for when “not applicable” is permitted.

The evidence-traceability ratio shows a similar sensitivity. With 10 of 12 reviewer-flagged items linked to specific evidence artefacts, the observed ratio is 0.833 (95% CI: 0.516–0.979); because the denominator is only 12, each additional closed or unresolved item changes the ratio by 0.083. This reinforces the interpretation of the metric as a case-level governance signal rather than a stable benchmark.

3.3. IEC 62443 Audit Action Plan Results

The IEC 62443 action plan considered in this study was developed under the IECEE (IEC System of Conformity Assessment Schemes for Electrotechnical Equipment and Components) (Certification Body) CB Scheme, an international system for the mutual recognition of electrotechnical test reports and certificates [50]. Within this context, the CB Scheme provides the procedural basis for recognising test evidence across participating bodies, which makes report integrity, scope definition, and traceability especially important.

Dominant themes include:

• Methodology/workflow incompleteness (NCR 3-dominant): The methodology was in draft status and missing explicit steps (e.g., identifying requirements in scope and maturity level for the chosen certification scenario).
• Scope and scenario definition errors (NCR 4-dominant): Plan of Evaluation contained incorrect/ambiguous certification scenarios, indicating insufficient control of the “Plan” stage and its downstream impact on reporting and conformity interpretation.
• TRF/report integrity issues (NCR 4- and NCR 5-dominant): Report numbering conflicts, combining multiple certification scenarios into one report and modifying template sections not intended for modification, directly affecting comparability and credibility of conformity statements.
• Evidence traceability requirements: Repeated emphasis that evidence must be described with sufficient metadata (type/version/chapter/date), consistent with ISO/IEC 17025 expectations for technical records and reproducibility.

Overall, the action plan supports the claim that cybersecurity assurance quality is often constrained less by the absence of security expertise and more by the absence of a structured conformity evidence discipline (scope, decision rules, evidence referencing, and reporting integrity).

3.4. ISO/IEC 17025 Internal Audit Findings as Governance “Maintenance” Signals

ISO/IEC 17025 internal audits can be interpreted as periodic governance “maintenance” events: they detect early drift in confidentiality, authorisation, and documentation controls before such drift propagates into invalid test outcomes or the exposure of sensitive client information. In the case study laboratory, the internal audit cycle recorded minor nonconformities in (i) confidentiality governance (e.g., incomplete coverage of employee non-disclosure agreements) and (ii) the timely update of documented responsibilities and authorisations when personnel roles change. Corrective actions were assigned with explicit due dates and were tracked to closure in the audit log.

Although these items are not “cyber-technical” vulnerabilities, they are material in cybersecurity testing laboratories, because confidentiality, integrity of authorisation, competence continuity, and traceability are prerequisites for trustworthy security results and for protecting client artefacts (e.g., firmware images, vulnerability details, and proprietary interfaces). Consistent with this pattern, the 2025 assessment dataset shows that governance-centric deviations tend to cluster around management-system controls rather than test execution. Reported minor findings include: incomplete specification of responsibilities and authorities for report signatories and supervision (Clause 6.2.4); incomplete records for externally provided cloud services (Clause 6.6.2); misspecified performance comparison criteria and lack of a participation plan for inter-laboratory comparisons for qualitative security outcomes (Clause 7.7.2); report templates containing inapplicable elements for qualitative cybersecurity tests and unclear terminology around outsourced/subcontracted work (Clause 7.8.3); ambiguity in contract review and reporting regarding decision rules and conformity statements, with an explicit recommendation to align with ILAC G8 [25] (Clauses 7.1.3 and 7.8.6); insufficient traceability when amending or re-issuing reports (Clause 7.8.8); and an under-specified risk-and-opportunity register and review cadence, including residual-risk treatment (Clause 8.5). Additional minor governance gaps noted in the dataset relate to the handling of internal nonconformities (Clause 7.10), the clarity of test-request templates to avoid confusion between testing and product certification (Clause 7.1), and the completeness of equipment registers for standard-specific tooling (Clause 6.4).

Framed through an industrial maintenance lens, each minor nonconformity functions as a leading indicator of governance degradation. Tracking recurrence, clause concentration, and mean time-to-closure provides measurable “cyber-maintenance indicators” [51] that complement technical test KPIs and support the sustained comparability and defensibility of conformity statements in fast-evolving security test domains.

4. Discussion

4.1. Why the Results Support ISO/IEC 17025 Operationalisation for Cybersecurity Testing

Cybersecurity testing differs from traditional laboratory testing because it combines:

• Tool-driven measurements (scanner outputs, captures, and logs);
• Expert judgement (triage, exploitability interpretation, and applicability decisions);
• Rapidly changing methods (tool versions, threat patterns);
• Conditional applicability (requirements depend on architecture, configuration, and interfaces).

Across the case evidence, the limiting factor for defensible conformity statements is rarely “test execution capability” alone; it is the governance layer that makes qualitative security results auditable, repeatable, and comparable. This is consistent with the 2025 accreditation/assessment findings, where recorded nonconformities were minor yet concentrated in management-system controls that directly shape test trustworthiness—e.g., incomplete definition of signatory authorities and supervision responsibilities (Clause 6.2.4), incomplete control records for externally provided services such as cloud hosting (Clause 6.6.2), a missing fit-for-purpose approach to proficiency testing/inter-laboratory comparisons for qualitative PASS/FAIL outcomes (Clause 7.7.2), report template misalignment with qualitative testing (Clause 7.8.3), ambiguity around decision rules and conformity statements (Clauses 7.1.3 and 7.8.6), weak amendment traceability when reports are re-issued (Clause 7.8.8), and an under-specified risk/opportunity register and review mechanism (Clause 8.5).

Taken together, these findings suggest that ISO/IEC 17025 can govern cybersecurity testing effectively only when clauses are translated into cybersecurity-native artefacts (e.g., ICS/IXIT, controlled method lists, evidence packages with versioned toolchains, decision-rule catalogues, review logs, and re-issuance traceability records). Compared with generic LIMS-centred digitization, the case evidence shows the additional need for explicit decision-rule ownership and controlled evidence manifests; compared with software-process compliance automation, it shows the added need for accreditation sign-off, nonconforming-work handling, and report amendment traceability. In the ETSI-aligned dataset, the main barrier to issuing conformity statements was not technical failure but insufficiently structured evidence and decision rules, visible in the initial dominance of INCONCLUSIVE outcomes. After review and evidence integration, most outcomes became determinate, suggesting that ambiguity is reduced primarily through governance discipline rather than through additional testing alone.

From a research perspective, the case contributes more than a local workflow description. It identifies an explanatory mechanism: ambiguity decreases when decision-rule ownership, method control, and evidence binding are governed as interdependent control objects. Framed in this way, the study offers an analytical model for accreditation-aware compliance automation that can be examined in future comparative and longitudinal studies.

4.2. Decision Rules as the Primary Lever for Ambiguity Reduction

The largest measurable effect remains the reduction in INCONCLUSIVE outcomes among claimed provisions (CSAI from ~0.98 to ~0.05 in the paper’s dataset). This aligns with ILAC-style logic: if evidence is incomplete, a laboratory should either (a) declare the result INCONCLUSIVE with documented rationale or (b) apply a predefined decision rule for the sufficiency, acceptance, and treatment of uncertainty/limitations.

The 2025 assessment findings strengthen this interpretation by showing that decision-rule weaknesses surface early and remain auditable. In the case material, a client-facing checklist had drifted toward implicit self-declaration, whereas the ISO/IEC 17025 report required a laboratory-owned statement of conformity and an explicit decision rule aligned with ILAC G8 (Clauses 7.1.3 and 7.8.6). Accordingly, the decision rule is not merely an administrative field; it determines ownership of the conformity claim, the threshold for evidence sufficiency, and the treatment of exceptions.

Two discussion implications follow:

• “INCONCLUSIVE” is not a failure of testing; it is a controlled quality state, where evidence expectations are recognised but not met, preserving an INCONCLUSIVE outcome prevents false passes and enables later reassessment without reconstructing the context.
• “Not applicable” and “not claimed” require strict constraints. Mandatory expectations cannot be de-risked by re-labelling them as “unclaimed.” This boundary condition is precisely where formal decision rules add value: they make the line between “out of scope” and “nonconforming” explicit and auditable. In practice, this argues for a decision-rule structure that is standard-aware (ETSI/EN 303 645, EN 18031 [52,53,54], IEC 62443) and interface-aware (ICS/IXIT conditions), and that explicitly encodes when NA is permissible.

4.3. Traceability and Comparability: Why Controlled Toolchains and Evidence Packages Matter

The IEC 62443 action plan issues highlighted earlier—scenario selection errors, mixed scenarios in one report, incomplete TRF fields, inconsistent report numbering, and insufficient evidence metadata—are not “mere documentation problems.” They determine whether two laboratories (or two assessments at different times) can be compared and whether a regulator can treat the report as credible compliance evidence.

The 2025 assessment dataset adds concrete, clause-linked examples of comparability failure modes that are governance-driven:

• Report template misfit for qualitative cybersecurity tests (e.g., references to uncertainty estimation where qualitative PASS/FAIL is used; confusion between outsourced vs. externalised work terminology), which can introduce interpretive noise and inconsistent reporting expectations (Clause 7.8.3).
• Weak amendment and re-issuance traceability, where a re-issued report retains the original identifier instead of using a unique identifier and explicit linkage, reducing auditability over time (Clause 7.8.8).
• Incomplete governance records for external service providers, notably cloud hosting, which creates both confidentiality risk and reproducibility ambiguity when the execution environment is externalised (Clause 6.6.2).
• Document control sensitivity points, including periodic review and consistent revision-state identification, which directly affects whether a given test was executed under the correct controlled method and template revision (Clauses 8.2/8.3—points sensibles).

In cybersecurity testing, “toolchain control” is not only version pinning, it is evidence interpretability: the environment state, tool versions, configuration, and artefact provenance must be sufficiently specified so that results remain meaningful as tools evolve. Evidence packages (captures, logs, configuration snapshots, versioned scripts, and controlled templates) therefore function as repeatability scaffolding for qualitative verdicts.

4.4. The Assurance Maintenance Loop as a Cyber-Maintenance Control System

Interpreting the empirical findings through the proposed PDCA loop clarifies how ISO/IEC 17025 governance behaves like an industrial maintenance system—detecting, correcting, and preventing “assurance downtime” (ambiguous or non-defensible conformity outcomes):

• Plan (scope, decision rules, and risk model): Assessment findings show that when decision rules are under-specified and conformity statements are blurred with client self-declarations, the entire assurance chain becomes fragile (Clauses 7.1.3 and 7.8.6). The same applies to risk and opportunity management: missing cyber-specific risks (e.g., IT system operational blockage, data quality/non-quality, and external equipment integrity/security) and the absence of residual-risk treatment or periodic re-evaluation limit the management system’s ability to anticipate and prevent governance failures (Clause 8.5).
• Do (controlled execution and technical records): Execution quality depends on controlled templates, fit-for-purpose reporting for qualitative outcomes, controlled external provider interfaces (including cloud services), and disciplined technical records. The findings around reporting content and external provider records indicate that “doing” in cyber labs includes maintaining the trust boundary around tooling, environments, and outsourced/externalised components.
• Check (review, internal audit, and inter-lab validity mechanisms): The ETSI dataset demonstrates that completeness checks and peer review materially change outcomes (INCONCLUSIVE → PASS) by resolving ambiguity rather than by adding tests. The assessment dataset reinforces that “Check” must also include fit-for-purpose validity mechanisms: qualitative PASS/FAIL domains require different inter-lab comparison criteria than quantitative measurement domains, and a participation plan is expected (Clause 7.7.2). Additionally, competence assurance in this stage includes demonstrable auditor competence and relevant cybersecurity testing expertise for the internal audit function (Clause 8.8.2—points sensibles).
• Act (corrective actions as governance maintenance actions): Nonconformity action plans (programme level) and internal audit corrective actions are the “maintenance actions” that update procedures, templates, decision rules, competence records, and risk registers. The key maintenance property is closure with verification—the governance analogue of restoring availability after downtime. The assessment process also makes closure timeliness explicit (action plan submission and evidence windows), which can be treated as a measurable maintenance parameter.

4.5. Proposed Measurable Cyber-Maintenance Indicators

Building on the observed patterns (ETSI/IEC 62443 dataset behaviour + 2025 assessment findings), the following indicators are proposed for accredited cybersecurity laboratories and asset owners:

• Conformity Statement Ambiguity Index (CSAI): Rate of INCONCLUSIVE outcomes among claimed items. Interpretation: Evidence/decision-rule maturity; reflects “assurance uptime.”
• Scope Misclassification Count: Number of mandatory requirements incorrectly marked NA/unclaimed (or moved outside of the scope without an auditable rule). Interpretation: Decision-rule boundary failures with direct compliance impact.
• Decision Rule Coverage Ratio: Share of projects/reports in which the decision rule is explicitly documented, traceable to the applicable specification, and applied consistently across provisions (contract review → report). Motivation: Assessment findings explicitly flag decision-rule ambiguity and the need to align conformity reporting with ILAC-style expectations (Clauses 7.1.3 and 7.8.6).
• Evidence Traceability Ratio: Share of provisions whose verdict includes explicit references to versioned artefacts (captures/logs/config snapshots/tool versions). Interpretation: Reproducibility and audit readiness; predictor of comparability across labs/time.
• Report Amendment Integrity Rate: Proportion of amended/re-issued reports that use a unique identifier and explicit linkage to the original report they replace. Motivation: Assessment findings show that weak amendment traceability degrades longitudinal comparability and audit defensibility (Clause 7.8.8).
• External Service Governance Coverage: Share of externally provided services (including cloud hosting) with documented evaluation, risk treatment, and retained records. Motivation: Assessment findings show governance records may omit cloud providers despite their security and confidentiality implications (Clause 6.6.2).
• Corrective Action Lead Time: Time from NCR creation to verified closure (internal audit + programme action plans). Interpretation: Governance MTTR; measures how quickly the lab restores “assurance availability.”

Together, these metrics connect cybersecurity testing quality to maintenance-quality management and enable asset owners to incorporate cybersecurity assurance capacity into broader uptime/resilience planning. At this stage, however, they should be interpreted as candidate process-quality indicators rather than as validated predictors of laboratory failure rates or field vulnerabilities. Establishing predictive validity would require repeated observations across laboratories and explicit linkage to downstream outcomes such as audit nonconformities, reopened reports, missed vulnerabilities, or post-release incident findings.

4.6. Limitations and Future Work

This study has the limitations typical of exploratory case study research. The empirical material is drawn from a single accredited cybersecurity testing laboratory and from a limited artefact set; therefore, the findings are intended as analytic generalisations rather than statistical claims. CSAI and the related governance indicators are introduced as descriptive within-case measures and are not yet validated as predictive indicators of laboratory failure, vulnerability escape, or incident rates. Likewise, although the manuscript now specifies a concrete automation-ready evidence architecture, it does not report a controlled experiment against a manual workflow, a generic LIMS implementation, or an alternative compliance-as-code deployment.

Future work should evaluate the proposed workflow blueprint across multiple laboratories and certification contexts and should include controlled or quasi-experimental comparisons against explicit baselines such as manual document-centric workflows, generic LIMS-centred digitalization, and OSCAL/CI-based compliance-as-code stacks. Useful benchmark variables include effort, traceability completeness, reviewer rework, amendment integrity, and reproducibility under toolchain change. In addition, predictive validation studies should test whether repeated observations of CSAI, the evidence-traceability ratio, decision-rule coverage, and corrective-action lead time correlate with downstream outcomes such as reopened reports, audit findings, or discovered vulnerabilities.

5. Conclusions

This paper presented an exploratory single-laboratory case study of how ISO/IEC 17025:2017 can be operationalised for cybersecurity testing laboratories through a clause-to-artefact crosswalk, a digitised evidence workflow, and an automation-ready compliance architecture. The main conceptual contribution is an accreditation-aware governance model in which decision rules, toolchain control, evidence bundles, review states, and report traceability are treated as coupled control objects rather than isolated implementation details.

Empirically, the ETSI-aligned case showed a strong within-case shift from ambiguous to determinate outcomes: INCONCLUSIVE verdicts decreased from 40 to 2, and 37 previously INCONCLUSIVE provisions transitioned to PASS. The aggregate number of FAIL verdicts remained one, although the identity of the failing clause changed during review. These results should be interpreted as descriptive evidence from a single laboratory and a limited artefact set. They show how explicit decision rules and traceable evidence can reduce ambiguity within this case, but they do not establish predictive validity or cross-laboratory generalisability.

Future work should evaluate the proposed framework across multiple laboratories and certification contexts and should include controlled or quasi-experimental comparisons against manual document-centric workflows, generic LIMS-centred digitization, and compliance-as-code stacks. Longitudinal studies are also needed to test whether CSAI and related governance indicators correlate with downstream outcomes such as reopened reports, audit findings, missed vulnerabilities, or post-release incident observations.