EPCIS-MutBench: A Controlled Benchmark for Security-Relevant Semantic and Lifecycle Integrity Validation in EPCIS 2.0.1 Supply-Chain Histories

Abstract

Electronic Product Code Information Services (EPCIS) standardizes inter-organizational supply-chain visibility events, yet structurally conformant histories can contain contradictory lifecycle claims that undermine information integrity. We present EPCIS-MutBench, a controlled benchmark and layered validator for evaluating security-relevant semantic and lifecycle integrity in EPCIS 2.0.1 histories. Mutations are treated as integrity violations independently of accidental or adversarial origin; the benchmark evaluates detection and localization rather than attacker attribution. The evaluation uses 2000 deterministic clean histories and 20,000 primary mutants, including 16,000 single-fault and 4000 compound cases, with validator-independent typed ground truth. Across all primary mutants, JavaScript Object Notation (JSON) Schema plus compatibility-projected GS1 Shapes Constraint Language (SHACL) detected 18.27%, whereas the full event-local and role-aware lifecycle validator detected 95.29%. Among 13,429 single-fault mutants that passed both structural baselines, the full validator achieved 92.98% recall, with end-to-end intended-rule, event, and field/edge localization of 92.90%, 92.19%, and 92.14%, respectively. No false positives occurred on the 2000 generated-clean controls; however, in a separate sensitivity analysis that provisionally treated all 37 parseable external-source fixtures as clean, 19 were marked invalid. A separate attestation layer used cryptographic hashes, Merkle commitments, a digital signature, and public-testnet receipts to make post-freeze evidence modification detectable. The results demonstrate controlled, reproducible evaluation of EPCIS information-integrity assurance beyond structural conformance.

1. Introduction

GS1 is an international not-for-profit standards organization that develops standards for identifying, capturing, and sharing supply-chain data [1]. It maintains Electronic Product Code Information Services (EPCIS), a standardized event model for exchanging supply-chain visibility information across organizations and software systems [2]. The associated Core Business Vocabulary (CBV) standardizes business steps, dispositions, source and destination types, and other terms used in EPCIS events [3].

In this article, an EPCIS history is a set of causally or temporally related events that jointly describe the lifecycle of one or more products, logistics units, or materials. Each event can record what happened, when and where it occurred, which objects or quantities were involved, and the associated business context. A history can therefore represent commissioning, aggregation, transformation, shipping, receiving, correction, repair, return, and destruction.

EPCIS 2.0.1 supports JavaScript Object Notation (JSON), JavaScript Object Notation for Linked Data (JSON-LD), sensor evidence, and linked-data validation artifacts [4,5]. These capabilities make EPCIS relevant to traceability systems and Digital Product Passport (DPP) infrastructures. In such infrastructures, interoperability, semantic consistency, and verifiable data quality are increasingly important requirements [6,7,8,9].

Because EPCIS exchanges machine-readable lifecycle claims across organizational and software boundaries, the integrity of those claims becomes security-relevant when downstream systems rely on them for custody, provenance, traceability, compliance, or automated decisions. A faulty or maliciously manipulated event can remain syntactically valid while inducing a false cross-event state, so representation-level acceptance does not by itself establish trustworthy lifecycle information. Prior supply-chain cybersecurity and blockchain-supported traceability research has likewise emphasized trustworthy shared evidence, secure traceability, and tamper resistance [10,11,12,13]. The cybersecurity focus of this study is therefore the integrity of machine-readable EPCIS lifecycle information.

However, structural conformance does not guarantee the semantic integrity of an EPCIS history. The GS1 EPCIS and CBV Implementation Guideline distinguishes technical and content validation from integrity validation, which is performed retrospectively over end-to-end event sequences [14]. Consider a product identifier recorded in a receiving event at a warehouse even though the history contains no compatible shipping event from the expected source. The receiving event may contain a valid event type, timestamp, location, transaction identifier, and business vocabulary, so it can remain syntactically and structurally conformant when examined in isolation. Nevertheless, under the declared custody-flow contract evaluated in this study, the combined history is operationally inconsistent because the claimed receipt is unsupported by the preceding custody flow. Detecting this fault requires reasoning across events, identifiers, locations, transactions, participation roles, and causal or temporal relationships rather than checking individual fields alone. Similar cross-event inconsistencies represented in the benchmark include use before commissioning, simultaneous attachment of one child object to two active parents, and transformation quantities that do not balance. These inconsistencies are semantic-integrity failures: a structurally valid record can still contain contradictory or operationally invalid lifecycle claims. Their security relevance follows from the consequences of accepting inconsistent lifecycle information in downstream custody, provenance, traceability, compliance, or automated-decision processes.

Prior studies have advanced EPCIS-based traceability, repositories, semantic representation, compliance validation, privacy, and blockchain-supported provenance [10,15,16,17,18,19,20,21]. However, the reviewed literature does not provide a reusable EPCIS-specific framework for controlled evaluation of integrity validators. Such a framework must introduce known lifecycle-integrity violations and confirm each intended violation independently of the validator under test. It must also preserve typed ground truth, control equivalent and ambiguous cases, and evaluate localization on an end-to-end basis so that undetected violations count as failures. Without these elements, reproducible comparison of validators that analyze cross-event integrity violations remains difficult. EPCIS-MutBench was developed to address this experimental gap.

Mutation testing addresses this evaluation gap by introducing known faults into otherwise valid inputs and observing whether a validator detects and localizes them correctly [22,23]. A mutation operator is a deterministic rule that introduces a specified fault into a clean EPCIS history. The altered history is a mutant. A single-fault mutant contains one introduced fault, whereas a compound mutant combines two registered faults. In this study, a clean history is a history that satisfies the declared generator contracts.

Synthetic benchmarks can become circular when the same assumptions influence generation, mutation, validation, and evaluation. EPCIS-MutBench reduces this risk by separating mutation truth from validator output. It also rejects equivalent or ambiguous candidates before primary evaluation and preserves deterministic replay and complete denominators. Section 3 specifies the corresponding benchmark-construction and admission procedures.

To address this gap, we developed and evaluated EPCIS-MutBench for EPCIS 2.0.1 histories. The framework provides a controlled mutation benchmark together with a layered validator. It measures whether semantic and lifecycle-integrity faults are detected and localized correctly.

The benchmark generates deterministic clean histories and applies controlled EPCIS-specific mutations. It establishes fault truth independently of the validator and records typed localization targets at multiple levels. Validation is then performed through separate structural, event-local, and lifecycle-aware layers.

For each admitted mutant, we produced a semantic witness without consulting the validator under evaluation. This independent witness establishes the intended rule violation. We separately recorded typed ground truth for the intended rule, affected events, document fields represented by JSON Pointer paths [24], graph nodes, and graph edges.

The evaluated validator performs JSON parsing and applies the official EPCIS JSON Schema [4,25]. It then executes event-local rules and role-aware graph/lifecycle reasoning. Shapes Constraint Language (SHACL) evaluation uses a documented compatibility projection of the GS1 SHACL graph [4,26]. This projection is performed in memory and does not modify the byte-preserved source graph.

The evaluation also examines dependence on source histories and source origin. Additional analyses cover conditional and end-to-end localization, clean-control specificity, scalability, reproducibility, and tamper-evident result attestation.

The security property evaluated in this study is information integrity, considered at two complementary levels. Semantic and lifecycle integrity concerns violations of declared structural, causal, role, quantity, custody, correction, and lifecycle constraints. The validator is evaluated on its ability to detect and localize these violations in EPCIS histories.

Evidence integrity concerns modification or substitution after the experimental artifacts have been frozen. The protected artifacts include benchmark data, typed truth, software, rules, results, and manifests. Hashes, signatures, Merkle commitments [27], and blockchain receipts make such changes detectable. The benchmark therefore evaluates security-relevant integrity assurance rather than cybersecurity in general. Confidentiality, availability, live-partner authentication, network-layer defense, and the physical truth of source observations are outside the evaluated scope.

The study addresses five research questions (RQs):

RQ1.

To what extent can event-local and graph/lifecycle reasoning detect security-relevant EPCIS semantic and lifecycle integrity violations that are not detected by JSON Schema and compatibility-projected GS1 SHACL validation?

RQ2.

How accurately can the validator localize the intended rule, affected events, and affected fields or graph relations when undetected integrity violations are counted as localization failures?

RQ3.

Which mutation operators and semantic preconditions account for the residual false negatives in integrity-violation detection and for diagnostic multiplicity?

RQ4.

How do full per-history validation and bounded-state streaming scale with event volume under the evaluated workloads?

RQ5.

To what extent can the attestation protocol detect post-freeze modification or substitution of frozen experimental evidence under the declared evidence-integrity threat model?

The study makes five principal contributions:

1.

We construct a controlled EPCIS 2.0.1 mutation benchmark with 2000 deterministic clean histories, 16,000 primary single-fault mutants, and 4000 primary compound mutants. The mutants span declared semantic and lifecycle integrity-fault families, and their security interpretation focuses on the consequences of accepting inconsistent histories.

2.

We define an admission process that checks applicability, confirms the intended violation independently of the validator, rejects equivalent or ambiguous candidates, verifies deterministic replay, and suppresses duplicates. Each admitted mutant receives typed ground truth for its rule, events, document fields, graph nodes, and graph edges.

3.

We implement a layered validator that separates EPCIS JSON Schema and compatibility-projected GS1 SHACL findings from event-local and role-aware graph/lifecycle reasoning. The graph/lifecycle layer distinguishes observed, consumed, produced, parent, and child participation roles.

4.

We evaluate detection recall and both conditional and end-to-end localization. The evaluation also includes complete operator-level results, generated-clean controls, compound faults, source-history uncertainty, source-origin sensitivity, ablations, and scalability experiments.

5.

We provide an evidence-integrity mechanism for the frozen benchmark, software, truth, and result artifacts. It uses 256-bit Secure Hash Algorithm (SHA-256) values [28], Merkle commitments [27], a digital signature, local Ethereum Virtual Machine (EVM) tests [29], and Ethereum Sepolia public-testnet receipts [30].

The remainder of this article is organized as follows. Section 2 reviews the cybersecurity context, EPCIS validation, semantic interoperability, traceability, integrity and provenance assurance, and mutation testing. Section 3 presents the materials and methods, including the overall EPCIS-MutBench workflow, formal problem formulation, benchmark construction, layered validation framework, and experimental protocol. Section 4 reports the SHACL compatibility audit, detection, localization, source-sensitivity, clean-control, miss, compound-fault, scalability, ablation, and integrity-attestation results. Section 5 discusses the findings and their interpretation. Section 6 summarizes the principal findings, their scope of interpretation, and the conclusions.

2. Security Context, Standards, and Related Work

This section establishes the security, standards, and research context for the benchmark. It first distinguishes structural EPCIS validation from cross-event lifecycle-integrity reasoning, then reviews supply-chain security, integrity and provenance assurance, semantic interoperability, traceability, and mutation-testing research to define the specific security-relevant evaluation gap addressed by EPCIS-MutBench.

2.1. EPCIS and Structural Validation

EPCIS defines event representations and interfaces for visibility data, while the Core Business Vocabulary (CBV) standard supplies controlled terms for business steps, dispositions, source and destination types, and related concepts [2,3]. The normative artifacts include a JSON Schema, a JSON-LD context, ontologies, and a SHACL graph [4]. JSON Schema evaluates representation-level requirements, whereas SHACL, a World Wide Web Consortium (W3C) Recommendation, validates constraints over Resource Description Framework (RDF) graphs [25,26]. Neither mechanism is, by itself, a complete operational lifecycle model. This distinction is consistent with the GS1 implementation guideline, which separates technical and content validation from retrospective integrity validation [14].

2.2. Supply-Chain Security, Integrity, Provenance, and Evidence Assurance

Prior EPCIS and adjacent supply-chain security research has examined privacy-aware event linking, blockchain-supported authentication and traceability, cybersecurity monitoring with blockchain-recorded evidence, and distributed product-trust mechanisms [10,11,12,17,20,21,31]. Related studies have explored Ethereum-compatible on-chain/off-chain traceability, blockchain-referenced assurance of software artifacts against modification or substitution, and Merkle-tree-based blockchain mechanisms for data integrity and accountability [32,33,34]. Broader reviews identify transparency, tamper resistance, trust, scalability, and cost as recurring concerns in blockchain-supported information traceability [35]. These mechanisms primarily protect authenticity, provenance, storage integrity, or trust in shared records. A complementary security problem arises when an accepted EPCIS record is syntactically well formed yet participates in a cross-event history that violates custody, causal, quantity, transformation, aggregation, correction, or lifecycle constraints. Recent work has also demonstrated reproducible blockchain anchoring of canonical records and Merkle commitments for tamper-evident auditability  [36]. EPCIS-MutBench targets a different primary problem—semantic and lifecycle information-integrity validation—while its separate attestation component similarly uses compact cryptographic commitments to provide tamper-evident binding of the frozen experimental evidence.

2.3. EPCIS, Semantic Interoperability, and Traceability

Prior work has studied standards-based tracking exchange, cloud-supported EPCIS transparency, cold-chain monitoring, privacy-aware event linking, sensor-data sharing, open-source EPCIS platforms, and blockchain-supported traceability [10,15,16,17,18,37,38]. Recent studies connect EPCIS or related event models with knowledge graphs, DPPs, semantic interoperability, governance, and distributed ledgers [7,19,31,39,40,41,42,43,44].

Adjacent semantic-information-system research further illustrates how standards-based and ontology-based data can be subjected to executable graph validation. Platt et al. transformed resources based on Fast Healthcare Interoperability Resources (FHIR) into a Web Ontology Language (OWL) knowledge graph and evaluated structural conformance with SHACL together with ontology reasoning, while Montecchiari operationalized ontology-based architectural principles through SHACL and SPARQL Protocol and RDF Query Language (SPARQL) checks that produced traceable violation reports [45,46]. Humm likewise incorporated executable SHACL/SPARQL quality checks into an industry-oriented ontology, demonstrating how semantic constraints can be maintained alongside evolving graph representations [47]. In adjacent industrial information-systems research, Li et al. constructed an industrial-chain knowledge graph to semantically organize cross-domain information, while Gorgs et al. implemented an ontology within a graph architecture to support semantic querying and consistency checking in knowledge-based production [48,49].

A closely related distinction appears in recent work on sequence-aware digital trust. Kim and Jang showed that validity of individual interaction steps does not necessarily establish interaction-level trust when violations emerge only through dependencies across a complete execution sequence [50]. Although their model concerns verifiable digital interactions rather than EPCIS histories, the underlying distinction is relevant here: local validity and cross-step consistency are separate assurance properties.

Recent EPCIS-oriented integrity work further sharpens the distinction between provenance assurance and lifecycle-integrity-validator evaluation. Tolcha et al. integrated EPCIS/CBV 2.0 with on-chain and off-chain Hyperledger Fabric storage for blockchain-backed traceability [20], while Malamas et al. coupled EPCIS-modeled events with append-only blockchain token histories and cryptographic event references for granular multi-firm traceability [21]. A recent preprint by Cochinescu maps contradiction constraints, including spatial uniqueness, temporal monotonicity, and quantity conservation, to EPCIS 2.0 event semantics and constructs contradiction proofs over causally ordered signed claims [51]. These studies address provenance, integrity, or contradiction accountability, but they do not provide the combined controlled EPCIS mutation population, validator-independent typed fault truth, mutation-admission controls, and end-to-end localization evaluation targeted by EPCIS-MutBench.

Mutation-based testing has also been applied to software that consumes knowledge graphs. John et al. introduced domain-specific graph mutation operators, SHACL-based robustness masks, input-coverage measures, automated operator extraction, and test-case minimization for failures caused by changing knowledge graphs [52,53]. Their objective was to evaluate the robustness of software applications and reasoners that consume mutable graph inputs. EPCIS-MutBench addresses a different but related problem: evaluating EPCIS semantic validators using lifecycle-specific mutations, validator-independent semantic witnesses, typed ground truth at multiple localization levels, and end-to-end detection and localization measures. Thus, the present novelty is not mutation of graph data itself but an EPCIS-specific benchmark and evaluation protocol for standards-governed multi-event lifecycle validation.

Table 1 and Table 2 compare the reviewed studies without ranking their overall quality. Table 1 summarizes their distinct objectives and domains, whereas Table 2 focuses narrowly on capabilities needed for controlled validator evaluation. A dash in the capability table means that the capability was outside the reported study scope, not that the study was deficient.

Table 1. Representative related work, research objective, and evaluated scope. EPCIS denotes Electronic Product Code Information Services.

Work	Primary Objective	Evaluated Scope
Främling et al. [15]	Inter-organizational product-information exchange	Standards-based product tracking and information sharing
Kassahun et al. [16]	Chain-wide meat transparency	EPCIS-supported traceability integration
Thakur and Forås [37]	Cold-chain temperature monitoring	EPCIS event and sensor evidence in an application case
Byun et al. [17]	Privacy-aware EPCIS event linking	Event linkage and privacy protection
Vieira et al. [54]	Simulation-supported semantic validation	Selected semantic validation scenarios
Li et al. [10]	Blockchain-supported EPCIS authentication	Record authentication and distributed evidence
Ahn et al. [18]	Open-source EPCIS 2.0 repository	Standards-compliant repository and service implementation
Tolcha et al. [7]	EPCIS-to-knowledge-graph mapping	Semantic representation and graph integration
Tolcha et al. [20]	Hybrid EPCIS blockchain traceability	On-chain/off-chain EPCIS/CBV traceability with Hyperledger Fabric
Wintercorn and van Deventer [19]	Ontology- and SHACL-based compliance validation	Case-based graph constraint validation
Neto et al. [31]	Hyperledger Fabric traceability	Distributed-ledger traceability architecture
Malamas et al. [21]	Granular blockchain traceability	EPCIS-modeled events with append-only token histories and cryptographic anchoring
Cochinescu [51]	Contradiction-based supply-chain accountability	EPCIS-mapped contradiction detection over causally ordered signed claims
John et al. [53]	Mutation-based testing of knowledge-graph software	Robustness of software and reasoners under graph changes
EPCIS-MutBench	Controlled evaluation of security-relevant EPCIS integrity validation	Known integrity violations, typed truth, detection, localization, and evidence attestation

Table 2. Capabilities relevant to controlled EPCIS integrity-validator evaluation. A check mark indicates an explicitly implemented and evaluated capability; “Case” indicates validation on selected cases rather than a reusable mutation benchmark; a dash denotes a capability outside the study’s reported evaluation scope and is not a judgment of study quality.

Work	Systematic Controlled Mutation	Admission or Validity Controls	Typed Multi-Level Ground Truth	Lifecycle-Violation Detection	End-to-End Fault Localization
Främling et al. [15]	–	–	–	–	–
Kassahun et al. [16]	–	–	–	–	–
Thakur and Forås [37]	–	–	–	–	–
Byun et al. [17]	–	–	–	–	–
Vieira et al. [54]	–	–	–	Case	–
Li et al. [10]	–	–	–	–	–
Ahn et al. [18]	–	–	–	–	–
Tolcha et al. [7]	–	–	–	–	–
Tolcha et al. [20]	–	–	–	–	–
Wintercorn and van Deventer [19]	–	–	–	Case	–
Neto et al. [31]	–	–	–	–	–
Malamas et al. [21]	–	–	–	–	–
Cochinescu [51]	–	–	–	Case	–
John et al. [53]	✓	✓	–	–	–
EPCIS-MutBench	✓	✓	✓	✓	✓

Taken together, the reviewed literature exposes a specific experimental gap rather than a lack of useful EPCIS technologies. Prior studies provide traceability applications, repositories, semantic and knowledge-graph representations, case-based compliance validation, privacy and supply-chain cybersecurity mechanisms, and blockchain-supported provenance, while mutation-based research provides controlled changes for evaluating software and knowledge-graph application robustness. However, among the works reviewed above, including those compared in Table 1 and Table 2, none jointly provide an EPCIS-specific controlled mutation population, explicit mutation-admission controls, validator-independent typed fault truth, multi-event lifecycle-violation detection, and end-to-end fault localization. EPCIS-MutBench addresses this combined evaluation gap by providing a controlled, typed, and end-to-end framework for measuring the detection and localization of security-relevant semantic and lifecycle integrity violations in EPCIS histories.

Mutation testing evaluates a system with controlled, known changes and has long been used to study fault-detection effectiveness, equivalent mutants, and higher-order faults [22,23]. Mutation analysis has also been combined with model-based test generation specifically to support fault detection and fault localization [55], while recent software-engineering work has used targeted mutation operators and automatically generated mutants to quantify the effectiveness of code inspection [56]. The present work applies this principle to multi-event EPCIS histories. Its novelty lies in using EPCIS-specific controlled mutations with validator-independent typed truth to measure not only whether lifecycle faults are detected but also whether the intended rule and affected events, fields, and graph relations are localized correctly on an end-to-end basis.

3. Materials and Methods

This section describes the design and evaluation methodology of EPCIS-MutBench. It proceeds from the overall framework and formal problem formulation to benchmark construction, layered validation, and the experimental protocol used for detection, localization, scalability, sensitivity, reproducibility, and integrity evaluation.

3.1. Overview of the EPCIS-MutBench Framework

Figure 1 presents the complete workflow of the proposed EPCIS-MutBench framework. The process begins with EPCIS 2.0.1 and Core Business Vocabulary standards, official GS1 examples [57], fixtures from the open-source OpenEPCIS test-data generator [58], and deterministic clean-history generation. These source assets support construction of the clean benchmark population, primary single-fault mutants, and compound mutants used in the reported evaluation.

Figure 1. End-to-end workflow of the proposed EPCIS-MutBench framework. Source assets support benchmark construction and controlled mutation admission; admitted cases receive validator-independent typed ground truth and are processed through the layered B0–B5 and B7 validation configurations. Detection, localization, source-sensitivity, clean-control, compound-fault, and scalability analyses are followed by a separate integrity-attestation stage that binds the frozen evidence to reproducible article-facing outputs. Abbreviations: CBV, Core Business Vocabulary; EPCIS, Electronic Product Code Information Services; EVM, Ethereum Virtual Machine; JSON, JavaScript Object Notation; SHA-256, 256-bit Secure Hash Algorithm; SHACL, Shapes Constraint Language. GS1 is a proper name rather than an acronym.

Candidate mutants subsequently undergo applicability, validator-independent semantic witness, non-equivalence, deterministic replay, ambiguity, and duplicate suppression controls. Each admitted mutant receives typed ground truth identifying the intended rule, affected events, JSON Pointer locations, graph nodes, and graph edges. The ground-truth store remains separate from the validator under evaluation so that mutation labels cannot influence detection.

For compact reporting, the evaluated validation configurations are denoted B0–B5 and B7: B0 is parse-only processing; B1 is EPCIS JSON Schema validation; B2 is compatibility-projected GS1 SHACL validation; B3 combines the B1 and B2 invalid decisions; B4 comprises event-local semantic rules; B5 comprises graph/lifecycle rules; and B7 is the full per-history validator, combining the B1–B5 findings with deterministic root-cause suppression and typed diagnostics. The evaluated configurations are specified in detail in Section 3.4.

As illustrated in Figure 1, the central scientific contribution is the controlled mutation benchmark comprising the clean histories, mutation operators, admission controls, and typed ground truth. The layered validator supports evaluation of that benchmark through structural, event-local, and graph/lifecycle reasoning. The evaluation stage measures detection recall, conditional and end-to-end localization, operator-level behavior, source sensitivity, clean-control specificity, compound-fault behavior, and scalability. The integrity and attestation stage is intentionally separate from semantic validation: its purpose is to make post-freeze modification or substitution of the benchmark, software, truth records, and reported results detectable through hashes, Merkle roots, signatures, local Ethereum Virtual Machine tests, and Ethereum Sepolia receipts.

3.2. Formal Problem Formulation

This subsection establishes the formal objects and decision criteria used throughout the evaluation. It defines histories and mutations, role-aware causal relations, detection and localization measures, and the associated security and integrity objectives.

3.2.1. Histories, Mutations, and Decision States

Let an EPCIS history be denoted by H, with

$$H=(E,R,S),$$

(1)

where E is the set of events, R is the set of derived relations, and S is source and scenario metadata. An event $e\in E$ contains an identifier, event type, event time, record time, business terms, locations, transactions, object-role assignments, quantities, and optional sensor evidence.

A mutation operator m transforms a clean history H into a candidate mutant $H^{\prime }=m\left(H\right)$. The admission process assigns a truth object

$$T\left(H^{\prime }\right)=(r^{*},E^{*},P^{*},V^{*},G^{*}),$$

(2)

where $r^{*}$ is the intended violated rule, $E^{*}$ is the affected event set, $P^{*}$ is the affected JSON Pointer set, $V^{*}$ is the affected graph-node set, and $G^{*}$ is the affected graph-edge set. Truth is generated and stored outside the validator under evaluation.

Every validation layer returns one of five operational states:

$$D_b\left(H\right)\in \{\mathrm{VALID},\mathrm{INVALID},\mathrm{INDETERMINATE},\mathrm{NOT}\text{-}\mathrm{APPLICABLE},\mathrm{ERROR}\}.$$

(3)

Indeterminate is used when the available partial order does not justify a unique causal conclusion. Not-applicable means that a rule’s preconditions are absent. Error is reserved for engine or execution failure and is never counted as a successful detection.

3.2.2. Role-Aware Relations and Causal Order

For identifier x and event e, the role relation is

$$\rho (x,e)\in \{\mathrm{observed},\mathrm{consumed},\mathrm{produced},\mathrm{parent},\mathrm{child},\mathrm{source},\mathrm{destination}\}.$$

(4)

The values observed, consumed, produced, parent, and child denote object participation roles, whereas source and destination denote custody, location, or organizational relation endpoints. The causal relation $e_i\prec e_j$ is established only by evidence such as explicit transformation flow, compatible transaction continuity, parent–child state transitions, correction references, or unambiguous event-time precedence. Equal timestamps are unordered unless another relation establishes causality. File position and record time are not treated as physical order.

3.2.3. Detection and Localization Metrics

Let $F\left(H^{\prime }\right)$ be the set of validator findings. Each finding contains a rule identifier and typed locations. A mutant is detected when at least one invalid finding is produced:

$$\det \left(H^{\prime }\right)=\mathbf{1}[\exists f\in F\left(H^{\prime }\right):f.\mathrm{state}=\mathrm{INVALID}].$$

(5)

Here, $\mathbf{1}[·]$ denotes the indicator function, which equals one when the enclosed condition is true and zero otherwise. For an eligible set $\mathcal{M}$, recall is

$$\mathrm{Recall}\left(\mathcal{M}\right)={\displaystyle \frac{{\sum }_{H^{\prime }\in \mathcal{M}}\det \left(H^{\prime }\right)}{\left|\mathcal{M}\right|}}.$$

(6)

Let $\widehat{\mathcal{R}}\left(H^{\prime }\right)$ denote the set of rule identifiers reported for mutant $H^{\prime }$. The intended-rule hit metric asks whether the intended rule occurs among those reported identifiers:

$$L_r\left(H^{\prime }\right)=\mathbf{1}[r^{*}\in \widehat{\mathcal{R}}\left(H^{\prime }\right)].$$

(7)

Strict rule-set exactness is a different endpoint:

$$L_{R,\mathrm{strict}}\left(H^{\prime }\right)=\mathbf{1}[\widehat{\mathcal{R}}\left(H^{\prime }\right)=\left\{r^{*}\right\}].$$

(8)

Thus, a correct intended-rule finding accompanied by a valid downstream consequence receives an intended-rule hit but not strict rule-set exactness.

Let ${\widehat{E}}_{r^{*}}$ and ${\widehat{Q}}_{r^{*}}$ be the predicted event set and combined field/edge set selected only from findings for $r^{*}$, with $Q^{*}=P^{*}\cup G^{*}$. Graph-node truth $V^{*}$ is retained in the audit artifacts and used to interpret graph-edge endpoints, but it is not scored as a separate primary endpoint. Intended-rule event and field/edge exactness are

$$L_E\left(H^{\prime }\right)=\mathbf{1}[{\widehat{E}}_{r^{*}}=E^{*}],$$

(9)

$$L_Q\left(H^{\prime }\right)=\mathbf{1}[{\widehat{Q}}_{r^{*}}=Q^{*}].$$

(10)

Strict all-finding versions compare the union of locations from every unsuppressed finding with the truth. Conditional localization averages over detected mutants. Define the end-to-end (E2E) localization score for endpoint k as

$${\tilde{L}}_k\left(H^{\prime }\right)=\left\{\begin{array}{cc}{L}_k\left(H^{\prime }\right),\hfill & \mathrm{if}\det \left(H^{\prime }\right)=1,\hfill \\ 0,\hfill & \mathrm{if}\det \left(H^{\prime }\right)=0.\hfill \end{array}\right.$$

(11)

The end-to-end mean over all eligible mutants is then

$${\overline{L}}_k^{\mathrm{E}2\mathrm{E}}={\displaystyle \frac{1}{\left|\mathcal{M}\right|}}\sum _{H^{\prime }\in \mathcal{M}}{\tilde{L}}_k\left(H^{\prime }\right),k\in \{r,E,Q,R_{\mathrm{strict}}\}.$$

(12)

Table 3 summarizes the operational notation and analysis populations used throughout the evaluation.

Table 3. Operational notation and analysis populations used in the evaluation.

Symbol or Term	Meaning	Population or Denominator Definition
${\mathcal{M}}_{20k}$	All primary single and compound mutants	20,000 predefined primary mutants
${\mathcal{M}}_{\mathrm{elig}}$	Primary single mutants that remain valid under both JSON Schema and compatibility-projected GS1 SHACL	Determined by the baseline-validity criterion; realized size reported in Results
Generated clean	Clean histories produced under the declared benchmark contracts	2000 predefined histories
Intended-rule localization	Locations selected from findings for $r^{*}$	Conditional and end-to-end
Strict all-finding localization	Union of all unsuppressed validator findings	Conditional and end-to-end
Engine error	Validator execution failure	Excluded from successes and reported separately

3.2.4. Security Property and Threat Model

The security property evaluated in this study is information integrity, with two distinct threat surfaces.

For semantic and lifecycle integrity, the relevant threat is the introduction of EPCIS claims that remain parseable or structurally conformant but produce a history that violates a declared causal, role, quantity, custody, aggregation, transformation, correction, sensor, or lifecycle constraint. Such claims may result from accidental data error, defective or compromised software, or deliberate manipulation by an actor capable of creating or altering EPCIS event content before validation. The benchmark does not attempt to distinguish these origins and does not assign malicious intent to individual mutants. Instead, it evaluates the security-relevant consequence: whether an integrity violation that could survive representation-level validation is detected and localized.

For evidence integrity, the threat model permits post-freeze modification, replacement, selective omission, reordering, or substitution of committed benchmark documents, typed truth, software, rule registries, result artifacts, or manifests, including attempts to associate the manuscript with a different artifact set. Hashes, signatures, Merkle commitments, and public-testnet receipts are used to make such post-freeze changes detectable.

The trusted boundary includes the cryptographic primitives, signing credentials, execution environment, validator implementation, and recorded blockchain receipt data. The evaluated threat model does not establish the physical truth of source observations, source authentication, resistance to a compromised signing key or validator host, confidentiality, availability, or network-layer intrusion resistance. Accordingly, EPCIS-MutBench is scoped to information-integrity assurance over EPCIS lifecycle evidence.

3.3. Benchmark Construction and Scope

This subsection defines the benchmark populations and the controls used to construct them. It describes the clean-history and source populations, the mutation taxonomy and coverage, and the admission procedure used to establish the final controlled mutant population.

3.3.1. Clean Histories and Populations

The benchmark generator produced eight declared lifecycle scenarios: manufacture and distribution; transformation; repair and reuse; recall and quarantine; recycling and circularity; cold-chain transport; aggregation hierarchy; and error correction. Each generator-controlled history contains exactly six EPCIS events. Accordingly, the 2000 generated clean histories comprise 12,000 events in total, and the generated-origin mutation population is constructed from these fixed-length six-event source histories. The external-source GS1 examples and OpenEPCIS fixtures retain their native document structures and are analyzed separately rather than being constrained to this six-event template. The scenarios were instantiated deterministically into 2000 clean histories that serve as controlled test fixtures under the declared benchmark contracts. Table 4 summarizes the benchmark populations and limits the interpretation permitted for each population.

Table 4. Benchmark populations used in the reported evaluation and their permitted interpretation.

Population	Cases	Operators/Templates	Permitted Interpretation
Generated clean controls	2000	–	Consistency evaluation under the declared generator contracts
Primary single mutants	16,000	27	Main single-fault detection and localization evidence
Primary compound mutants	4000	5	Any-fault detection and all-component rule localization

The primary benchmark uses controlled histories to preserve exact mutation truth and reproducible denominators, while EPCIS documents not produced by the benchmark generator were incorporated to assess whether the principal behavior was specific to generator-origin inputs. These external-source documents comprised official GS1 examples and OpenEPCIS fixtures. In the primary mutant population, 19,819 mutants were derived from generated histories, 170 from official GS1 examples, and 11 from OpenEPCIS fixtures. A separate set of 37 parseable external-source fixtures was used in the external-fixture denominator-sensitivity analysis reported in Section 4.8. The primary clean false-positive endpoint was defined on the 2000 generator-controlled clean histories, whereas external-source mutants were analyzed separately by source origin.

3.3.2. Mutation Taxonomy and Standards-Derived Coverage

The reported evaluation uses 27 primary single-fault operators and five compound templates. Registered mutation operators use identifiers beginning with M, followed by the frozen registry number and a descriptive operator name. The 27 primary single-fault operators span 11 semantic families: vocabulary, identity, temporal, location, aggregation, transformation, quantity, custody, lifecycle, correction history, and sensor evidence. Operator selection was derived from EPCIS/CBV constraints, declared lifecycle invariants, and scenario-specific contracts. The frozen primary operator quotas define the controlled evaluation population; accordingly, operator- and family-level recall characterize performance within that declared experimental population.

Table 5 summarizes the grouped semantic coverage of the primary single-fault operators and compound templates. Complete operator-level detection and localization results are reported in Section 4.

Table 5. Grouped semantic coverage of primary single-fault families and compound templates.

Family/Template Group	Representative Fault Semantics	Principal Evidence Type
Vocabulary and identity	Unknown business terms, invalid action, duplicate or malformed identifiers	Local fields and identity relations
Temporal and location	Reversed causal time, record-before-event, teleportation, simultaneous locations	Partial order and location state
Aggregation	Orphan disaggregation, cycle, child under two active parents	Parent–child graph
Transformation and quantity	Input equals output, missing output, non-conservation, missing unit	Role-aware flow and quantities
Custody and lifecycle	Transaction mismatch, receive without ship, use before commission, activity after destruction	Transaction and lifecycle state
Correction and sensor	Missing or self-referential correction target, excursion, sensor time outside event	Reference graph and sensor interval
Compound	Two coordinated faults from different families	Multiple intended rules

From a cybersecurity perspective, the registered mutations exercise integrity properties whose violation can render lifecycle state or evidence untrustworthy if the altered history is accepted. Table 6 summarizes these properties and their security-relevant consequences.

Table 6. Security-relevant integrity properties and consequences exercised by the benchmark mutation groups.

Benchmark Group	Integrity Property Stressed	Security-Relevant Consequence If Accepted
Vocabulary and identity	Semantic identity and business-context integrity	Untrustworthy identifiers, actions, business steps, or dispositions may enter the shared event history
Temporal and location	Causal and provenance integrity	The history may assert impossible chronology, unsupported movement, or mutually inconsistent location state
Aggregation	Relationship and containment integrity	Parent–child state may be corrupted through orphaned, cyclic, or conflicting aggregation claims
Transformation and quantity	Material-lineage and accounting integrity	Input–output lineage, quantities, or units may become inconsistent with the declared transformation state
Custody and lifecycle	Custody and process-state integrity	Unsupported transfers, receipts, commissioning state, destruction state, or subsequent activity may be accepted
Correction and sensor	Historical and evidence integrity	Correction relationships or sensor evidence may no longer support the event history that they are intended to qualify
Compound	Multi-fault integrity	Multiple coordinated or co-occurring violations may affect more than one integrity property in the same history

3.3.3. Mutation Admission

Algorithm 1 formalizes primary mutation admission and typed truth construction. Admission begins by verifying that the source history satisfies its declared clean-scenario contract and that the registered mutation operator is applicable. The operator is then applied once, after which a validator-independent semantic witness is constructed to determine whether the candidate exhibits the intended contract violation. The witness is an admission oracle rather than a validation rule: it may compare the clean source history H with the mutated history $H^{\prime }$, together with the registered operator contract and exact mutation difference, to establish benchmark truth. It therefore answers whether the mutation successfully introduced the intended violation, not whether the validator under evaluation can infer that violation from $H^{\prime }$ under its own applicability and evidence requirements. If the witness fails to establish that violation, the candidate is returned as Oracle-rejected; the validator under evaluation is never consulted in this decision.

For candidates with an established semantic witness, typed ground truth is derived from the operator contract, the exact syntactic difference, and the witness. Canonical-hash equality separately rejects exact no-op transformations, deterministic replay verifies reproducibility, and candidates with ambiguous causal interpretations under the registered partial-order policy are excluded from the reported primary benchmark. Duplicate candidate hashes are also suppressed before admission.

Algorithm 1 Primary mutation admission and typed truth construction

Require: Clean history H, registered operator m, operator contract $C_m$

Ensure: Admitted pair $(H^{\prime },T)$ or a recorded rejection reason

1: Verify that H satisfies the clean scenario contract    2: if $C_m$ is not applicable to H then    3:      return Not-applicable    4: end if    5: Apply m once to obtain candidate $H^{\prime }$ and an exact syntactic difference $\Delta$    6: Construct validator-independent admission witness $W_m(H,H^{\prime },C_m,\Delta )$ from the source–mutant pair, operator contract, and exact mutation difference    7: if $W_m$ does not establish the intended contract violation then    8:     return Oracle-rejected    9: end if    10: Derive typed truth $T=(r^{*},E^{*},P^{*},V^{*},G^{*})$ from $C_m$, $\Delta$, and $W_m$    11: if canonical hash$\left(H^{\prime }\right)=$ canonical hash$\left(H\right)$ then    12:     return Equivalent    13: end if    14: Re-run mutation and truth construction from the same seed    15: if history or truth differs from the first run then    16:     return Non-deterministic    17: end if    18: if causal interpretation is ambiguous under the partial-order policy then    19:     return Ambiguous    20: end if    21: if canonical hash$\left(H^{\prime }\right)$ already exists in the operator-specific candidate pool then    22:     return Duplicate    23: end if    24: Store $H^{\prime }$ and T in separate validation and truth stores    25: return Admitted

Duplicate screening was performed at two stages. During candidate construction, canonical hashes were checked within each operator-specific candidate pool to prevent repeated candidates from entering that pool. During frozen quota selection, a second canonical-hash check was applied globally across operators and benchmark classes. A duplicate identified at this stage was excluded and, where necessary, replaced by another eligible candidate so that the predefined class and operator quotas could be preserved. The resulting mutation screening and quota-selection flow were evaluated through the mutation-screening and quota-flow accounting audit described in Section 3.5.2.

3.4. Validation Framework

This subsection specifies the validation and diagnostic architecture used to evaluate the benchmark. It describes the layered execution model, the SHACL compatibility projection and audit protocol, and the causal replay, suppression, and typed-diagnosis procedures.

3.4.1. Layered Execution

Table 7 defines the per-history validation configurations used in the reported experiment.

Table 7. Validation layers used in the experiment.

Layer	Name	Operation
B0	Parse only	JSON/JSON-LD loading and basic document accessibility
B1	EPCIS JSON Schema	Official EPCIS 2.0.1 JSON Schema validation
B2	Compatibility-projected GS1 SHACL	Compatibility projection of the GS1 SHACL graph executed with pySHACL
B3	Schema + compatibility-projected SHACL	Union of B1 and B2 invalid decisions
B4	Event-local semantic rules	Vocabulary, local time, quantities, corrections, and sensor constraints
B5	Graph/lifecycle rules	Role-aware causal, aggregation, custody, transformation, and lifecycle constraints
B7	Full validator	B1–B5 findings followed by deterministic root-cause suppression and typed diagnostics

The reported per-history semantic evaluation uses 11 event-local rules and 13 graph/lifecycle rules. Validation never reads mutation labels. Root-cause suppression removes only registered duplicate or deterministic downstream consequences; the pre-suppression finding set remains available for audit.

3.4.2. SHACL Compatibility Projection and Audit Protocol

The official GS1 SHACL source file was preserved byte-for-byte. Direct execution and a documented compatibility projection were evaluated separately so that the behavior of the unmodified source graph could be distinguished from that of the projected comparative baseline.

The compatibility projection was performed in memory and normalized shape roles without modifying the preserved source artifact. It involved 23 shapes, comprising 58 node-to-property role normalizations and one property-to-node role normalization. The resulting comparative baseline is therefore described throughout this article as compatibility-projected GS1 SHACL, rather than as direct execution of the unmodified graph.

The engine audit comprised 2712 documents: 2037 clean documents and 675 primary single-fault mutants. The clean population contained 2000 generated lifecycle histories, 35 official GS1 examples [57], and two fixtures from the open-source OpenEPCIS test-data generator [58]. The mutant population contained 25 documents for each of the 27 primary single-fault operators; 668 were derived from generated histories and seven from official GS1 examples.

Four execution modes were evaluated: direct and projected pySHACL [59], and direct and projected Apache Jena [60]. For modes that completed document-level validation, cross-engine reproducibility was assessed using conformance status and normalized finding signatures. Engine-loading failures were preserved explicitly rather than converted into document-level invalid decisions. The audit outcomes are reported in Section 4.

The byte-preserved epcis-shacl.ttl source has SHA-256 7db2547a7a55e7d6e6f19c0ad472d1f0987ef19f3d04a43b195bca55c0cc7f42. The reproducibility records preserve the complete 23-shape role-normalization mapping, all 59 reference-rewrite operations, the pinned pySHACL, RDFLib [61], Apache Jena, Java, offline-context, and execution settings, representative execution logs, the audit receipt and summary, document-level results, and the baseline audit report. These non-sensitive records are available as described in the Data Availability Statement.

3.4.3. Causal Replay, Suppression, and Typed Diagnosis

Algorithm 2 gives the operational sequence. Findings are never silently converted from Error or Indeterminate to Invalid. The intended-rule selection used for one localization endpoint is performed only by the evaluator after validator findings have been generated.

Algorithm 2 Layered validation, causal replay, suppression, and typed diagnosis

Require: EPCIS history H

Ensure: Decision states and auditable validator findings F

1: Parse H; on failure emit B0 Error and stop    2: Execute official JSON Schema and record B1 findings    3: Build JSON-LD graph and execute compatibility-projected GS1 SHACL; preserve engine errors explicitly    4: Normalize events and assign object participation roles    5: Evaluate event-local rules with explicit applicability states    6: Construct the event-causality relation and the separate transaction, aggregation, transformation, correction, and lifecycle relations    7: Replay state in a topological order of the event-causality relation; keep equal-time incomparable events unordered    8: for all graph/lifecycle rules do    9:     emit Invalid, Valid, Indeterminate, or Not-applicable    10: end for    11: Save the complete pre-suppression finding set    12: Apply only registered root-cause and duplicate suppression mappings    13: Attach event identifiers, JSON Pointers, graph nodes, and graph edges to remaining findings    14: Return layer decisions, pre-suppression findings, suppression records, and final findings

For n events, q object/transaction references, and f emitted findings, normalization and event-local validation are $O(n+q)$. Construction of the derived relations and topological replay of the event-causality graph are $O(n+|E_g\left|\right)$, where n is the number of events and $E_g$ is the event-causality edge set. Aggregation-cycle validation operates on the separate active parent–child relation and therefore does not define the event replay order. The implemented suppression pass is $O(flogf)$ because findings are deterministically ordered before keyed suppression. JSON Schema and SHACL costs depend on document size and engine behavior; they are measured empirically rather than assigned a benchmark-independent asymptotic bound.

3.5. Experimental Protocol

This subsection defines the experimental endpoints and procedures used to evaluate the benchmark and validator. It covers the primary statistical endpoints and uncertainty analyses, scalability and sensitivity experiments, integrity-attestation protocol, and reproducibility environment.

3.5.1. Primary Endpoints and Uncertainty

The primary RQ1 endpoint uses ${\mathcal{M}}_{\mathrm{elig}}$, defined as the primary single mutants that remain valid under both JSON Schema and compatibility-projected GS1 SHACL. This conditioning directly operationalizes RQ1 by isolating controlled integrity violations that escape both structural baselines. From a security-assurance perspective, this population represents the cases of greatest interest because representation-level validation has accepted the document while the declared cross-event integrity contract has been violated. The all-primary analysis is reported in parallel to retain full-population context. The realized eligible-subset size is reported in Section 4. Detection results on all 20,000 primary single and compound mutants are therefore additionally reported as a descriptive full-population coverage analysis. The generated-clean endpoint measures false positives only on the 2000 generated controls. The RQ2 endpoint reports both conditional and E2E localization, and the compound-fault endpoint evaluates the 4000 compound mutants. The scalability analysis addresses RQ4, while the evidence-integrity analysis addresses RQ5.

B3 is used as a standards-conformance baseline rather than as a competing lifecycle-integrity validator. Accordingly, the B3–B7 comparison is intended to quantify the additional integrity-violation coverage obtained when event-local and cross-event lifecycle reasoning is added beyond representation- and graph-constraint validation. It is not interpreted as a head-to-head comparison between systems designed to provide identical validation capabilities.

For the represented mutation-family set $\mathcal{F}$, the equally weighted macro-family recall difference between B7 and B3 is

$${\Delta }_{\mathrm{macro}}={\displaystyle \frac{1}{\left|\mathcal{F}\right|}}\sum _{f\in \mathcal{F}}\left(R_{\mathrm{B}7,f}-R_{\mathrm{B}3,f}\right),$$

(13)

where $R_{b,f}$ is recall for validation configuration b within family f. Equal family weighting prevents large operator quotas from dominating this endpoint.

Mutant-level proportions retain two-sided 95% Wilson confidence intervals (CIs) as descriptive binomial intervals [62]. Because multiple mutants descend from the same clean source history, seven principal endpoints were additionally evaluated with 10,000 source-origin-stratified percentile-bootstrap resamples of clean_history_id clusters [63]. Each resample kept every descendant mutant of a sampled clean history together. The seven endpoints were B7 and B3 recall on all primary mutants, the paired B7–B3 recall gain, B7 recall on eligible primary single mutants, and end-to-end intended-rule, event, and field/edge localization. The corresponding source-history cluster counts are reported alongside the resulting estimates in Section 4. The macro-family recall difference separately retains its frozen 10,000-resample scenario/history cluster bootstrap. Every major proportion is accompanied by its denominator and a Wilson interval where a binomial interpretation is meaningful. The corresponding Wilson and source-history cluster-bootstrap sensitivity results are reported in Section 4. Count-based flow audits and analytical Ethereum gas ratios, where gas is the EVM execution-resource accounting unit [29], are non-binomial endpoints and therefore do not receive Wilson intervals.

3.5.2. Scalability, Ablation, and Integrity-Attestation Protocol

Scalability was evaluated in two separate tracks. Full validation was measured over five history volumes spanning 1000–16,000 generated six-event clean histories, with five non-warm-up runs per scale. Runtime, throughput, per-history 95th-percentile latency, and peak resident set size were recorded. Bounded-state streaming was evaluated over multiple event volumes up to one million events, again using five non-warm-up runs per scale, with runtime, throughput, batch 95th-percentile latency, maximum active state, and peak resident set size recorded. The streaming track evaluated event-local and bounded graph/lifecycle processing without JSON-LD expansion, JSON Schema, or SHACL, so its measurements are reported separately from full validation.

The evaluation also included eight computational ablation and sensitivity analyses: removal of event-local rules (A1), removal of graph/lifecycle rules (A2), collapse of participation roles (A3), mutation-screening and quota-flow audit (A4), collapse of typed localization (A5), a counterfactual external-fixture clean-label denominator (A6), disabling root-cause suppression (A7), and comparison of commitment strategies (A8).

For A8, the six commitment strategies were labeled C0–C5: C0, off-chain only; C1, per-event registration; C2, batch-hash commitment; C3, event-Merkle commitment; C4, dual-root commitment; and C5, complete-manifest commitment.

For evidence integrity, frozen artifacts were deterministically serialized as canonical JSON, hashed with SHA-256, and bound through Merkle roots, an off-chain signature, local EVM tests, and Ethereum Sepolia receipts [27,28,29,30,64]. The threat model permits post-freeze modification, replacement, selective omission, or substitution of committed digital artifacts, including attempts to associate the manuscript with a different artifact set. The integrity claim assumes trustworthy cryptographic primitives, signing credentials, execution environment, and blockchain receipt data. The protocol included artifact-modification tamper controls, secret-redaction boundary testing, local commitment-strategy measurement, and Sepolia deployment and registration measurements. Raw EPCIS histories were not placed on-chain.

The artifact-integrity controls were labeled T1–T7 for event-content change, event reordering, report-root change, rule-set-hash change, software-hash change, truth-record change, and manifest-root substitution or reuse, respectively. T8_SECRET_LEAKAGE was a boundary assertion for secret exposure and was complemented by the separate secret_redaction self-test.

3.5.3. Reproducibility and Execution Environment

The experiment ran on Windows 11 build 10.0.26200 with Python 3.12.10, a 13th Gen Intel(R) Core(TM) i9-13900H processor, 64 gibibytes (GiB) of memory, and an NVIDIA RTX 4070 Laptop graphics processing unit (GPU). The validation workload was central-processing-unit (CPU) bound. The principal validation packages included jsonschema 4.21.1 [65], RDFLib 7.6.0 [61], and pySHACL 0.40.1 [59]; Apache Jena 6.1.0 supplied the second SHACL engine [60] and was executed with Java 21. Resource usage was recorded as resident set size (RSS).

Reproducibility was assessed through two complementary procedures. First, the four principal clean and mutant findings/summary artifacts were re-evaluated from the frozen inputs and compared using SHA-256 values. Second, a reuse-only verification recalculated the metric fingerprints without regenerating benchmark data or resending blockchain transactions. The outcomes of these reproducibility checks are reported with the integrity-attestation results in Section 4.9.

4. Results

This section reports the empirical outcomes of the evaluation protocol. The results cover the SHACL compatibility audit, mutation detection and localization, source and clean-control sensitivity, residual misses, compound faults, scalability, computational ablations, and integrity-attestation experiments.

4.1. SHACL Compatibility Audit

Both tested engines rejected the unmodified official GS1 SHACL graph during shape loading, before document-level conformance evaluation. pySHACL raised a ShapeLoadError stating that a shape defined as a NodeShape cannot be the subject of an sh:path predicate. Apache Jena raised a ShaclParseException stating that the object of sh:node must be a node shape rather than a property shape. Consequently, neither direct mode produced document-level conformance results. In the audit output, the mode-level graph-load failure was propagated as an engine-error outcome to each of the 2712 document entries in each direct mode; these entries therefore do not represent 2712 independent document-level conformance failures.

Both compatibility-projected modes completed without engine errors. pySHACL and Apache Jena each classified 2637 of the 2712 audit documents as valid and 75 as invalid. They agreed on conformance status and normalized finding signatures for all 2712 documents. Table 8 summarizes the audit outcomes.

Table 8. SHACL execution audit across 2712 documents.

Mode	Valid	Invalid	Errors	Comparable	Interpretation
pySHACL, direct official graph	0	0	2712	no	graph-load failure
Apache Jena, direct official graph	0	0	2712	no	graph-load failure
pySHACL, compatibility projection	2637	75	0	yes	projected baseline
Apache Jena, compatibility projection	2637	75	0	yes	projected baseline
Projected cross-engine comparison	100% status, conformance, and signature agreement			yes	reproducible projection

4.2. Security-Relevant Integrity-Violation Detection Coverage

The principal detection analysis quantified controlled integrity violations that escaped the structural baselines and were recovered by lifecycle-aware validation. Across all 20,000 primary mutants, B3 (JSON Schema plus compatibility-projected GS1 SHACL) detected 3654 cases, corresponding to 18.27% recall (mutant-level Wilson 95% CI 17.74–18.81%). B7 detected 19,057 cases, corresponding to 95.29% recall (95% CI 94.98–95.57%), an absolute difference of 77.02 percentage points.

Within the predefined RQ1 population of 13,429 primary single mutants that remained valid under both structural baselines, B7 detected 12,486 cases (92.98%). All 943 B7 misses in the complete 20,000-mutant population occurred within this conditioned subset; no additional misses occurred outside it. The equally weighted macro-family recall difference between B7 and B3 was 0.9460 (95% bootstrap CI 0.9437–0.9485). These results quantify the additional security-relevant integrity coverage provided by event-local and role-aware cross-event reasoning within the benchmark population. Table 9 summarizes detection recall across all evaluated validation configurations.

Table 9. Detection recall on all 20,000 primary mutants.

Layer	Detected	Recall	95% CI
B0 parse only	0	0.0000	[0.0000, 0.0002]
B1 JSON Schema	1883	0.0942	[0.0902, 0.0983]
B2 compatibility-projected GS1 SHACL	3151	0.1576	[0.1526, 0.1627]
B3 Schema + compatibility-projected SHACL	3654	0.1827	[0.1774, 0.1881]
B4 event-local rules	9110	0.4555	[0.4486, 0.4624]
B5 graph/lifecycle rules	12,701	0.6351	[0.6284, 0.6417]
B7 full validator	19,057	0.9529	[0.9498, 0.9557]

Table 10 reports the source-history dependence sensitivity for the seven principal endpoints. The cluster-bootstrap estimates closely tracked the mutant-level Wilson results. In particular, B7 all-primary recall remained 0.9529 with a cluster-bootstrap 95% CI of 0.9504–0.9554, and the paired B7–B3 gain was 0.7702 with a cluster-bootstrap 95% CI of 0.7656–0.7747.

Table 10. Mutant-level Wilson intervals and source-origin-stratified clean-history cluster-bootstrap sensitivity for principal endpoints. The paired gain is a paired mean difference and therefore has no Wilson interval.

Endpoint	Source			Wilson	Cluster-Bootstrap
	Mutants	Histories	Estimate	95% CI	95% CI
B7 recall, all primary mutants	20,000	2037	0.9529	[0.9498, 0.9557]	[0.9504, 0.9554]
B3 recall, all primary mutants	20,000	2037	0.1827	[0.1774, 0.1881]	[0.1784, 0.1868]
Paired B7–B3 recall gain	20,000	2037	0.7702	–	[0.7656, 0.7747]
B7 recall, eligible primary singles	13,429	2035	0.9298	[0.9253, 0.9340]	[0.9261, 0.9335]
End-to-end intended-rule hit	13,429	2035	0.9290	[0.9246, 0.9333]	[0.9254, 0.9327]
End-to-end event exactness	13,429	2035	0.9219	[0.9172, 0.9263]	[0.9180, 0.9258]
End-to-end field/edge exactness	13,429	2035	0.9214	[0.9167, 0.9258]	[0.9174, 0.9253]

Figure 2 visualizes the corresponding recall estimates and their mutant-level 95% Wilson intervals.

Figure 2. Mutation recall of the separately evaluated validation configurations on 20,000 primary mutants. Error bars show 95% Wilson intervals. B2 and B3 use the compatibility-projected GS1 SHACL graph.

Table 11 separates the 19,819 generated-origin primary mutants from the 181 external-source primary mutants and further reports the official GS1 and OpenEPCIS source strata individually. The generated-only estimates were almost identical to the all-primary estimates, showing that the aggregate result was not driven by the small external-source stratum. The external-source aggregate had an eligible denominator of 136. Because the individual external-source strata are small, their estimates are interpreted as source-origin sensitivity results rather than independent external-validation evidence.

Table 11. Source-origin sensitivity for generated-origin and external-source primary mutants, including the individual official GS1 and OpenEPCIS source strata.

(a) Detection on all primary mutants
Source Scope	All-Primary $\mathit{n}$		B3 Recall		B7 Recall
Generated-origin only	19,819		0.1828		0.9529
Official GS1 examples	170		0.1882		0.9412
OpenEPCIS fixtures	11		0.0000		1.0000
External-source aggregate	181		0.1768		0.9448
(b) Eligible-single detection and end-to-end localization
Source Scope	Eligible $\mathit{n}$	B7 Recall	E2E Rule	E2E Event	E2E Field/Edge
Generated-origin only	13,293	0.9298	0.9298	0.9229	0.9229
Official GS1 examples	126	0.9206	0.8492	0.8413	0.7937
OpenEPCIS fixtures	10	1.0000	0.9000	0.6000	0.5000
External-source aggregate	136	0.9265	0.8529	0.8235	0.7721

Family-level results in Figure 3 localize the residual error concentration. B7 recall was 0.8338 for aggregation and 0.6351 for quantity, while most other semantic families exceeded 0.99. These values characterize performance under the frozen benchmark-family composition and operator quotas.

Figure 3. Family-level recall of Schema plus compatibility-projected GS1 SHACL and the full validator, with 95% Wilson intervals and family sample counts.

Table 12 reports the complete operator-level detection and intended-rule end-to-end localization results for all 27 primary single-fault mutation operators.

Table 12. Complete operator-level B7 detection and intended-rule end-to-end localization for the 27 primary single-fault mutation operators. M09_EQUAL_CAUSAL_TIME is a registered non-primary temporal control operator and is therefore not included in this primary-performance table; the resulting numbering gap is intentional. Primary n denotes the frozen single-fault population for each operator, and eligible n denotes mutants that remained valid under both JSON Schema and compatibility-projected GS1 SHACL. B7 detection, recall, misses, and localization are evaluated on the eligible denominator. The localization triplet reports intended-rule hit/event exactness/field-edge exactness. A dash denotes that the corresponding performance quantity is not applicable because the operator had no eligible mutants. UOM denotes unit of measure.

Operator	Family	Primary	Eligible	B7		Misses	E2E Localization
		$\mathit{n}$	$\mathit{n}$	Detected	Recall		Rule/Event/Field-Edge
M03_INVALID_ACTION	Vocabulary	689	0	–	–	–	–/–/–
M04_UNKNOWN_BIZSTEP	Vocabulary	689	689	689	1.0000	0	1.0000/1.0000/1.0000
M05_UNKNOWN_DISPOSITION	Vocabulary	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M06_DUPLICATE_EVENT_ID	Identity	688	0	–	–	–	–/–/–
M07_MALFORMED_EVENT_ID	Identity	688	0	–	–	–	–/–/–
M08_REVERSE_CAUSAL_TIME	Temporal	688	688	688	1.0000	0	0.9985/0.9985/0.9985
M10_RECORD_BEFORE_EVENT	Temporal	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M11_LOCATION_TELEPORT	Location	688	688	687	0.9985	1	0.9985/0.9985/0.9985
M12_SIMULTANEOUS_TWO_LOCATIONS	Location	688	688	688	1.0000	0	1.0000/0.9956/0.9956
M13_ORPHAN_DISAGGREGATION	Aggregation	501	501	501	1.0000	0	1.0000/1.0000/1.0000
M14_AGGREGATION_CYCLE	Aggregation	502	502	502	1.0000	0	1.0000/1.0000/1.0000
M15_CHILD_TWO_PARENTS	Aggregation	501	501	251	0.5010	250	0.5010/0.4990/0.4990
M16_INPUT_EQUALS_OUTPUT	Transformation	503	503	503	1.0000	0	1.0000/1.0000/1.0000
M17_REMOVE_TRANSFORMATION_OUTPUT	Transformation	504	501	501	1.0000	0	0.9980/0.9980/0.9980
M18_DUPLICATE_TRANSFORMATION_OUTPUT	Transformation	503	0	–	–	–	–/–/–
M19_NEGATIVE_QUANTITY	Quantity	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M20_NONCONSERVING_QUANTITY	Quantity	504	504	504	1.0000	0	1.0000/1.0000/1.0000
M21_REMOVE_UOM	Quantity	688	688	2	0.0029	686	0.0000/0.0000/0.0000
M22_TRANSACTION_MISMATCH	Custody	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M23_RECEIVE_WITHOUT_SHIP	Custody	688	688	685	0.9956	3	0.9942/0.8605/0.8605
M24_DESTROY_THEN_ACTIVE	Lifecycle	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M25_USE_BEFORE_COMMISSION	Lifecycle	688	688	688	1.0000	0	1.0000/1.0000/1.0000
M26_REUSE_WITHOUT_REPAIR	Lifecycle	250	250	250	1.0000	0	1.0000/1.0000/1.0000
M27_CORRECTIVE_TARGET_MISSING	Correction history	688	688	688	1.0000	0	1.0000/1.0000/0.9971
M28_CORRECTION_SELF_REFERENCE	Correction history	688	688	688	1.0000	0	0.9971/0.9971/0.9971
M29_SENSOR_EXCURSION	Sensor evidence	267	267	264	0.9888	3	0.9775/0.9775/0.9625
M30_SENSOR_TIME_OUTSIDE_EVENT	Sensor evidence	267	267	267	1.0000	0	1.0000/1.0000/0.9963

The frozen registry numbering is not contiguous within this primary subset: M09_EQUAL_CAUSAL_TIME is a registered non-primary temporal control operator and is therefore excluded from the primary single-fault detection and localization population reported in Table 12. Its absence from the table consequently reflects the frozen operator classification rather than a missing primary result.

Four primary operators—M03_INVALID_ACTION, M06_DUPLICATE_EVENT_ID, M07_MALFORMED_EVENT_ID, and M18_DUPLICATE_TRANSFORMATION_OUTPUT—had no mutants that remained valid under both JSON Schema and compatibility-projected GS1 SHACL; they are retained with an eligible denominator of zero and the corresponding performance quantities marked not applicable rather than being omitted. This preserves complete operator accounting even when the conditioned RQ1 population contains no eligible cases for a registered primary operator. Because eligibility requires a mutant to remain valid under both JSON Schema and compatibility-projected GS1 SHACL, B3 detection is necessarily zero within this conditioned subset and is therefore not repeated in the table.

4.3. Generated Clean Controls

No B7 false positives occurred on the 2000 generated clean histories: 0.0000 with a 95% Wilson upper bound of 0.0019. This endpoint measures consistency between the declared generator contracts and validator behavior on the predefined generated-clean control population. External-source cases are reported separately in the source-origin sensitivity analysis.

4.4. Conditional and End-to-End Localization

Table 13 and Figure 4 separate the two reporting scopes. Conditional metrics use only the 12,486 eligible mutants detected by B7. End-to-end metrics use all 13,429 eligible mutants and assign zero localization success to the 943 undetected cases. This reduces intended-rule hit from 0.9992 conditionally to 0.9290 end-to-end, event exactness from 0.9915 to 0.9219, and field/edge exactness from 0.9909 to 0.9214. Strict rule-set exactness is substantially lower because valid downstream findings count as extras.

Table 13. Conditional and end-to-end localization on primary single mutants that remained valid under both JSON Schema and compatibility-projected GS1 SHACL. Conditional intervals are Wilson intervals. For the first three principal end-to-end endpoints, the displayed intervals are source-history cluster-bootstrap intervals; strict end-to-end endpoints retain Wilson intervals.

Endpoint	Conditional $\mathit{n}=\mathrm{12,486}$	95% CI	End-to-End $\mathit{n}=\mathrm{13,429}$	95% CI
Intended-rule hit	0.9992	[0.9985, 0.9996]	0.9290	[0.9254, 0.9327]
Event exact	0.9915	[0.9897, 0.9930]	0.9219	[0.9180, 0.9258]
Field/edge exact	0.9909	[0.9891, 0.9925]	0.9214	[0.9174, 0.9253]
Strict rule-set exact	0.7708	[0.7633, 0.7781]	0.7167	[0.7090, 0.7242]
Strict all-finding event exact	0.8512	[0.8448, 0.8573]	0.7914	[0.7845, 0.7982]
Strict all-finding field/edge exact	0.7630	[0.7555, 0.7704]	0.7094	[0.7017, 0.7171]

Figure 4. Conditional and end-to-end localization. End-to-end estimates count every undetected eligible mutant as unsuccessful. Points show proportions, and figure error bars show mutant-level 95% Wilson intervals. Table 10 additionally reports source-history cluster-bootstrap sensitivity for the three principal end-to-end endpoints.

4.5. Miss Analysis

Residual errors were highly concentrated rather than distributed across the rule set. Importantly, a validator miss does not mean that the admitted mutation lacked a confirmed violation. Admission and detection answer different questions: the validator-independent witness establishes the intended violation from the source–mutant pair, registered operator contract, and mutation difference, whereas B7 must infer an invalid state from the evidence available in the history being validated under the applicability conditions of its implemented rules. Consequently, an oracle-confirmed mutant can remain undetected when those validator-side context requirements are not established.

Two operators accounted for 936 of the 943 misses (99.26%). For M21_REMOVE_UOM, the admission witness confirmed that the registered mutation removed the required unit information and thereby established the intended benchmark violation; however, 686 of 688 eligible mutants were not detected because the implemented unit-scoped conservation rule required sufficient quantity-and-unit context in the validated history before issuing an Invalid decision. For M15_CHILD_TWO_PARENTS, the admission witness established the intended conflicting-parent mutation, while 250 eligible cases did not provide the active-parent state required by the validator’s conservative aggregation rule. The remaining seven misses were distributed across three operators. Thus, these cases are false negatives of the evaluated validator rather than failures of mutation admission, and they identify two specific context-gating limitations for subsequent validator extension. Table 14 summarizes the operator-level distribution and diagnostic categories of the 943 residual misses.

Table 14. Operator-level causes of the 943 missed eligible primary mutants.

Operator	Family	Misses	Diagnostic Category
M21_REMOVE_UOM	Quantity	686	unit, scope, or conservation context insufficient
M15_CHILD_TWO_PARENTS	Aggregation	250	active aggregation-state precondition not established
M23_RECEIVE_WITHOUT_SHIP	Custody	3	no matching validator finding
M29_SENSOR_EXCURSION	Sensor evidence	3	no matching validator finding
M11_LOCATION_TELEPORT	Location	1	no matching validator finding

4.6. Compound Faults

All 4000 compound mutants had at least one component detected: 1.0000 (95% Wilson CI [0.9990, 1.0000]). All component rules were localized in 3997 cases, or 0.99925 (95% Wilson CI [0.99780, 0.99974]). These results characterize compound-fault behavior for the five registered templates included in the benchmark.

4.7. Scalability Results

Following the scalability protocol in Section 3.5.2, points are reported as medians and error ranges span the observed minimum–maximum across the five measured runs at each scale. Full validation included JSON Schema, compatibility-projected GS1 SHACL, event-local rules, and graph/lifecycle rules on 1000–16,000 generated six-event clean histories. These measurements therefore characterize full validation on the predefined generated six-event clean-history workload across the evaluated scales. Median throughput remained between 211 and 217 events/s; median runtime increased from 28.39 s to 445.66 s, median per-history 95th-percentile (p95) latency remained near 0.054 s, and median peak resident set size (RSS) remained between 1.68 and 1.89 GiB. Figure 5 reports runtime, throughput, per-history latency, and peak memory across the five full-validation scales.

Figure 5. Full per-history validation across multiple history volumes: (a) runtime, (b) throughput, (c) latency, and (d) peak resident set size (RSS). Points are medians; error ranges span the observed minimum–maximum across five non-warm-up measured runs per scale.

The bounded streaming track excluded JSON-LD expansion, JSON Schema, and SHACL and evaluated event-local plus bounded graph/lifecycle state. Here, bounded-state processing means that the implementation retains only the active object and relation state required by the configured rules, rather than retaining the complete preceding event stream. At one million events, median runtime was 7.13 s, median throughput was 140,155 events/s, median p95 batch latency was 0.342 s, maximum active state was 6251 entries, and peak RSS was 1.70 GiB. Across the five measured runs at that scale, runtime ranged from 6.25 to 7.76 s, throughput from 128,905 to 160,083 events/s, and p95 batch latency from 0.160 to 0.391 s. Because this track performs the narrower event-local and bounded graph/lifecycle task, its throughput is reported separately from the full-validation measurements. Figure 6 reports the corresponding runtime, throughput, batch latency, and peak-memory measurements for bounded-state streaming.

Figure 6. Bounded event/graph streaming across multiple event volumes: (a) runtime, (b) throughput, (c) latency, and (d) peak RSS. Points are medians; error ranges span the observed minimum–maximum across five non-warm-up measured runs per scale. This track evaluates event-local and bounded graph/lifecycle processing without JSON-LD expansion, JSON Schema, or SHACL; its throughput is therefore reported separately from full-validator throughput.

4.8. Computational Ablation and Sensitivity Analyses

Eight computational ablation and sensitivity analyses were completed (A1–A8). Table 15 reports denominators and intervals. A1 and A2 show that neither semantic layer alone reproduces full eligible-subset recall. A3 detected 13,358 of 13,429 eligible mutants (0.9947, Wilson 95% CI 0.9933–0.9958), which was 872 more detections than the full role-aware validator on that subset. However, the same role-collapsed engine marked all 2000 generated-clean histories as invalid: false-positive rate 1.0000 (95% CI 0.9981–1.0000) and specificity 0.0000 (95% CI 0.0000–0.0019), with zero engine errors. The full role-aware validator produced 0 false positives on the same controls and specificity 1.0000 (95% CI 0.9981–1.0000). Thus, the apparent recall gain from role collapse was obtained only by sacrificing clean-case discrimination completely, demonstrating that mutation recall alone is insufficient for evaluating lifecycle-validation quality. A5 shows that collapsing typed locations into one untyped target set substantially reduces exactness. A6 evaluates a denominator counterfactual in which all 37 parseable external fixtures are assigned a clean label. Under that deliberately broadened denominator, 19 of 37 fixtures are marked invalid, yielding an apparent false-positive rate of 0.5135 and demonstrating the sensitivity of the endpoint to the clean-reference definition.

Table 15. Completed computational ablations and sensitivity analyses.

Label	Analysis	Denominator	Estimate	95% CI
A1	Remove event-local rules	13,429	0.5592 recall	[0.5508, 0.5676]
A2	Remove graph/lifecycle rules	13,429	0.4093 recall	[0.4010, 0.4177]
A3	Collapse participation roles: eligible-mutant detection	13,429	0.9947 recall	[0.9933, 0.9958]
	Collapse participation roles: generated-clean controls	2000	1.0000 false-positive rate; 0.0000 specificity	false-positive rate [0.9981, 1.0000]; specificity [0.0000, 0.0019]
A4	Mutation-screening and quota-flow audit	75,369 attempts	54,292 pre-quota; 20,050 frozen records	not applicable
A5	Collapse typed localization	12,486	0.5605 exact	[0.5517, 0.5692]
A6	Counterfactual clean-label sensitivity on external fixtures	37	19/37; 0.5135 apparent false-positive rate	[0.3589, 0.6655]
A7	Disable root-cause suppression	20,000	0.06035 affected	[0.05713, 0.06374]
A8	Compare commitment strategies	6 strategies	144,882.8×	not applicable

A4 is an accounting audit rather than a binomial performance endpoint. Across 75,369 operator–history mutation attempts, screening excluded 21,077 attempts and produced a pre-quota capacity of 54,292 viable candidates. Frozen capacity-aware selection retained 20,050 mutation records: 16,000 primary single mutants, 4000 primary compound mutants, and 50 auxiliary non-primary records. The reported detection, localization, compound-fault, and related performance analyses use only the 20,000 primary evaluation mutants; the 50 auxiliary records are retained solely for frozen construction and provenance accounting and are not part of the reported benchmark-performance population. The remaining 34,242 viable candidates were not retained because the declared class and operator quotas had been filled. During global selection, 10 duplicate hashes were suppressed and replaced, and no final primary operator quota had a shortfall. Table 16 reports the complete numerical flow from mutation attempts to frozen record selection. Selection-flow quantities are reported as counts, so no Wilson confidence interval is assigned.

Table 16. Numerical flow from mutation attempts to the frozen mutation-record selection.

Stage or Audit Quantity	Count	Interpretation
Operator–history mutation attempts	75,369	All registered mutation attempts before screening
Attempts excluded before pre-quota capacity	21,077	Attempts not included in the viable pre-quota candidate capacity after applicability, witness, equivalence, ambiguity, replay, or related screening controls
Pre-quota candidate capacity	54,292	Candidates available before frozen quota selection
Primary single mutants retained	16,000	Main single-fault evaluation population
Primary compound mutants retained	4000	Main compound-fault evaluation population
Auxiliary non-primary records retained	50	Frozen construction records excluded from the reported performance evaluation
Total frozen mutation records retained	20,050	20,000 primary evaluation mutants plus 50 auxiliary non-primary records
Viable candidates not retained after quota filling	34,242	Candidates exceeding the declared class or operator quotas
Duplicate hashes suppressed and replaced during selection	10	Global duplicates replaced without reducing final quotas
Final primary operator-quota shortfall	0	Every declared primary evaluation quota was satisfied

The 10 duplicate replacements were selection-stage audit events rather than an additional subtraction from the 54,292-to-20,050 flow. The admission, equivalence, ambiguity, replay, and duplicate controls therefore operated without reducing any predefined final quota.

A7 affected 1207 histories and introduced 1746 extra findings when root-cause suppression was disabled. The A6 value is intentionally labeled as an apparent false-positive rate because its denominator is a deliberate counterfactual that assigns a clean label to all parseable external fixtures.

A8 evaluated six commitment strategies (C0–C5); the reported 144,882.8× ratio compares the per-event strategy (C1) with the complete-manifest strategy (C5). The per-event strategy used a local-EVM estimate of 215,329 gas per event registration; extrapolation across 141,020 represented events therefore gives 30,365,695,580 gas units. The complete-manifest strategy used one experimentally measured local EVM transaction of 209,588 gas units. Their ratio is 144,882.8×, which is reported as an analytical commitment-strategy ratio.

4.9. Evidence Integrity and Result Attestation

The integrity-attestation protocol described in Section 3.5.2 was executed on the frozen evidence package. Deterministically serialized canonical JSON, SHA-256 file hashes, Ethereum Keccak-256 contract values, and Merkle roots bound the benchmark data, typed truth, rule registry, software artifacts, and result manifests [27,28,64,66].

A clean frozen-input re-evaluation reproduced the four principal clean and mutant findings/summary artifacts with identical SHA-256 values. A separate reuse-only verification recalculated the metric fingerprints without regenerating benchmark data or resending blockchain transactions. Together, these checks verified deterministic reproducibility of the reported computational outputs under the documented execution environment.

Local contract tests passed, and the off-chain signature recovered the expected signer. Seven digital-integrity tamper controls passed: T1 event-content change, T2 event reordering, T3 report-root change, T4 rule-set-hash change, T5 software-hash change, T6 truth-record change, and T7 manifest-root substitution or reuse. T8_SECRET_LEAKAGE served as a boundary assertion, and the separate secret_redaction self-test verified that sensitive values were excluded or redacted.

A separate Ethereum Sepolia experiment deployed the Solidity 0.8.30 contract at 0x85B71a4d5560303d9CDDbB3dC66Df1c39670aCa3 on Sepolia (chain identifier 11155111) [30,67]. Deployment consumed 508,597 gas. Two registration transactions were confirmed: a one-event sample used 209,552 gas, and the 141,020-event complete-manifest commitment used 209,588 gas. These values are reported as Sepolia testnet execution measurements; no Mainnet cost conversion is applied.

Complete transaction hashes, block numbers, the signature, application binary interface, bytecode hash, manifests, tamper results, and event/truth Merkle proofs were retained as non-sensitive reproducibility evidence and are available as described in the Data Availability Statement. Within the declared threat model, these results show that the committed digital artifacts can be checked for post-freeze modification or substitution and linked to the expected signer. The attestation claim concerns the integrity and provenance binding of the committed digital evidence.

5. Discussion

This section interprets the experimental findings in relation to the five research questions and the declared information-integrity security model. It considers security-relevant lifecycle-integrity detection and residual errors, robustness of the principal results, end-to-end localization, the security implications of structurally conformant integrity violations, the SHACL reproducibility boundary, scalability, and evidence integrity.

5.1. Security-Relevant Lifecycle-Integrity Detection and Residual Errors

For RQ1, the principal finding is that lifecycle-aware reasoning detects a large fraction of controlled integrity violations that survive both structural baselines. Among the 13,429 baseline-valid primary single mutants, B7 detected 12,486 cases (92.98%). In the parallel full-population analysis, JSON Schema plus compatibility-projected GS1 SHACL detected 18.27% of the 20,000 primary mutants, whereas the full event-local and role-aware lifecycle validator detected 95.29%, an absolute difference of 77.02 percentage points. These results quantify the additional integrity coverage obtained when explicit business-process and lifecycle constraints are evaluated across related events. They do not imply a deficiency in JSON Schema or SHACL, which address different validation purposes and are not, by themselves, complete operational lifecycle models.

For RQ3, the residual misses are highly concentrated: 936 of 943 (99.26%) arise from the unit-removal and two-active-parent operators. These misses concern validator detectability rather than benchmark-truth uncertainty: the admission witnesses had already established the intended violations independently of B7. The quantity result is dominated by unit removal, for which the implemented validator rule requires sufficient scope and conservation context within the validated history before issuing an Invalid decision. For example, if a clean transformation records an input quantity as 10 kg and the mutation preserves the value 10 while removing its unit of measure, the admission witness can establish the deletion from the source–mutant pair, whereas B7 observes only the mutated history and cannot establish that the affected quantity is commensurable with the other transformation quantities. The rule therefore conservatively withholds an Invalid decision rather than assuming a unit scope that is not established by the validated history. The aggregation result is similarly dominated by the conservative active-parent precondition. These results therefore expose specific context-gating limitations of the evaluated validator while preserving the independent admission truth of the corresponding mutants. The suppression sensitivity analysis further showed that disabling root-cause suppression affected 1207 histories and introduced 1746 additional findings, quantifying the contribution of registered duplicate and downstream consequences to diagnostic multiplicity.

The A3 ablation demonstrates the complementary role of typed participation semantics. Collapsing consumed, produced, observed, parent, and child roles increased eligible-mutant detection from 0.9298 to 0.9947, but the same role-collapsed configuration marked all 2000 generated-clean controls as invalid. Role distinctions are therefore not merely diagnostic metadata: under the tested contracts, they are necessary to prevent broad object overlap from becoming a universal invalidity signal. Physically, the same identifier can legitimately participate in different lifecycle relations: for example, an object consumed as an input to a transformation has a different process meaning from an object produced as its output, just as a child contained by a parent aggregation has a different relation from an object merely observed in an event. Collapsing these distinctions causes valid process participation to be interpreted as a contradictory state, which explains the 100% false-positive rate observed in A3. In an automated supply-chain system, such behavior would make otherwise valid lifecycle histories unusable for reliable exception handling, traceability, or downstream decision support. The ablation therefore shows that granular participation roles are required not only for higher-quality diagnostics but also for preserving an operationally meaningful lifecycle state.

5.2. Security Interpretation and Defense-in-Depth Implications

The security interpretation of EPCIS-MutBench centers on the operational consequences of accepting EPCIS content that violates a declared integrity contract. Such acceptance can corrupt custody, provenance, transformation, aggregation, quantity, correction, sensor, or lifecycle state available to downstream systems. The reported results therefore characterize the ability of the evaluated validation layers to detect and localize these security-relevant integrity failures within the benchmark population.

The detection results also support a defense-in-depth interpretation. JSON Schema and compatibility-projected GS1 SHACL provide representation- and graph-constraint checks, whereas the event-local and role-aware graph/lifecycle layers evaluate integrity properties that depend on relationships across events. The 77.02-percentage-point difference between B7 and B3 over the complete primary population, together with the 92.98% B7 recall on structurally baseline-valid primary single mutants, shows that representation-level acceptance does not by itself establish cross-event lifecycle integrity within the benchmark population. The separate attestation layer addresses a different security boundary by making post-freeze modification or substitution of committed experimental evidence detectable. Structural validation, lifecycle-integrity reasoning, and evidence attestation should therefore be interpreted as complementary controls rather than interchangeable security mechanisms.

5.3. Robustness of Detection and Localization Across Source Origin

The principal detection result was stable to both source-history dependence and source-origin composition, but localization showed a larger source-origin sensitivity. Source-history cluster resampling retained a B7 all-primary recall estimate of 0.9529 with a 95% interval of 0.9504–0.9554, while the paired B7–B3 recall gain remained as 0.7702 with a 95% interval of 0.7656–0.7747. The generated-origin stratum reproduced the all-primary B7 recall of 0.9529, while the external-source stratum yielded 0.9448. On the eligible-single population, generated-origin and external-source B7 recall were similarly close, at 0.9298 and 0.9265, respectively.

The source-origin comparison was less stable for end-to-end localization. For generated-origin eligible single mutants, intended-rule, event, and field/edge localization were 0.9298, 0.9229, and 0.9229, respectively. For the external-source aggregate, the corresponding values were 0.8529, 0.8235, and 0.7721. Thus, detection recall remained similar across source origin, whereas localization—particularly field/edge localization—was lower on the external-source cases. Because the external-source eligible subset contained only 136 mutants and its individual source strata were small, these differences are interpreted as source-origin sensitivity rather than as independent external-validation estimates.

5.4. End-to-End Localization Performance

For RQ2, near-perfect conditional localization does not by itself describe end-to-end performance because undetected faults are absent from the conditional denominator. Conditional intended-rule, event, and field/edge localization reached 0.9992, 0.9915, and 0.9909, respectively. Counting all 13,429 eligible mutants reduces the corresponding end-to-end values to 0.9290, 0.9219, and 0.9214 because every undetected mutant receives zero localization success. Strict rule-set and strict all-finding metrics are lower because valid downstream consequences can produce additional findings. The distinction among intended-rule hit, strict rule-set exactness, and all-finding exactness is therefore essential for interpreting diagnostic usefulness.

5.5. SHACL Projection as a Reproducibility Boundary

The multi-engine audit defines the reproducibility boundary of the SHACL baseline. Both tested engines rejected the unmodified official graph during shape loading, whereas the documented compatibility projection executed successfully in both engines. Exact cross-engine agreement after projection establishes reproducibility of the projected baseline across the tested toolchains. The reported SHACL comparisons therefore refer specifically to this documented projected configuration.

5.6. Scalability

For RQ4, full validation scaled approximately linearly over the measured range with stable throughput and memory. Median full-validation throughput remained between 211 and 217 events/s across 1000–16,000 generated six-event histories. The separate bounded streaming track processed one million events at a median throughput of 140,155 events/s while retaining a maximum active state of 6251 entries. Because the streaming track evaluates the narrower event-local and bounded graph/lifecycle task without JSON-LD expansion, JSON Schema, or SHACL, its measurements are reported separately from full-validation throughput. The reported scalability results characterize the documented single-machine configurations and evaluated event volumes.

5.7. Evidence Integrity and Reproducibility

For RQ5, the attestation experiments demonstrated completed integrity checks across local and public-testnet execution. Local contract tests passed, the off-chain signature recovered the expected signer, and all seven artifact-modification tamper controls passed. The Sepolia contract deployment and both registration transactions were confirmed, while the clean frozen-input re-evaluation reproduced the principal result artifacts with identical SHA-256 values. Together, these results show that, within the declared threat model, the frozen benchmark, truth records, software, rules, and reported results can be bound to tamper-evident commitments and checked against the recorded evidence. The attestation layer supports evidence integrity and provenance binding and is kept separate from the semantic-validation contribution.

5.8. Limitations and Threats to Validity

Several boundaries define the interpretation of the results. The primary benchmark deliberately relies mainly on generator-controlled histories so that source contracts, mutation applicability, intended violations, and multi-level localization truth can be established deterministically rather than inferred retrospectively. These generator-controlled histories have a fixed length of six EPCIS events. This design supports controlled mutation truth and reproducible cross-event reasoning, but it does not establish detection or localization performance for longer or more structurally variable individual lifecycle histories. The separate scalability experiments increase the number of six-event histories and evaluate bounded streaming at substantially larger event volumes, but they do not replace semantic evaluation on long, variable-length individual histories. To assess dependence on that controlled population, the evaluation additionally includes official GS1 examples and OpenEPCIS fixtures as source-origin sensitivity cases; these cases are not claimed to constitute a statistically independent external-validation corpus. The registered mutation operators cover declared EPCIS, CBV, and scenario-specific integrity contracts but do not exhaust every violation that may occur in operational deployments. Finally, although benchmark truth and validator output are isolated through validator-independent witnesses, separate stores, deterministic replay, ambiguity controls, and duplicate screening, the benchmark and evaluated validator were developed within the same research project. Independent validator implementations, longer and more structurally diverse lifecycle histories, and larger independently curated EPCIS corpora would provide valuable next stages of external replication.

The cybersecurity interpretation is intentionally limited to information integrity. The mutation operators are controlled integrity violations rather than observations of real-world attacks; consequently, the benchmark does not estimate attack probability, adversary prevalence, exploitability, or the operational frequency of the represented violations. The study also does not evaluate confidentiality, availability, live-partner authentication, network-layer compromise, credential theft, denial of service, or the physical truth of sensor and source observations.

The structural comparison is additionally bounded by the documented compatibility projection of the GS1 SHACL graph because the byte-preserved source graph did not load successfully in either tested engine. Scalability measurements characterize the evaluated single-machine workloads, while the bounded streaming track excludes JSON-LD expansion, JSON Schema, and SHACL. Finally, the attestation mechanism protects the post-freeze integrity and provenance binding of committed digital artifacts; it does not establish security against compromise of the trusted signing credentials, validator host, execution environment, or underlying cryptographic assumptions.

6. Conclusions

This study introduced EPCIS-MutBench, a controlled benchmark for evaluating security-relevant semantic and lifecycle information-integrity validation in EPCIS 2.0.1 histories. Across 20,000 primary mutants, JSON Schema plus compatibility-projected GS1 SHACL detected 18.27%, whereas the full event-local and role-aware lifecycle validator detected 95.29%, an absolute difference of 77.02 percentage points. Among 13,429 primary single-fault mutants that passed both structural baselines, the full validator achieved 92.98% recall, with end-to-end intended-rule, event, and field/edge localization of 92.90%, 92.19%, and 92.14%, respectively. No false positives occurred on the 2000 generated-clean controls, all 4000 compound mutants triggered at least one component detection, and source-history cluster-bootstrap analysis preserved the principal detection conclusions. The residual misses were concentrated primarily in two context-gated operators, providing specific targets for further validator development.

From a security perspective, the results show that structural conformance and cross-event lifecycle integrity are complementary assurance objectives: a document can satisfy representation-level checks while participating in a history that violates a declared integrity contract. EPCIS-MutBench contributes a reproducible test population, validator-independent typed fault truth, controlled mutation-admission procedures, and end-to-end diagnostic measures for evaluating these security-relevant integrity violations. The supporting attestation mechanism addresses a separate security boundary by making post-freeze modification or substitution of committed experimental evidence detectable. Future work can extend this foundation through independently curated EPCIS corpora, independently implemented validators, broader lifecycle contracts, and security scenarios derived from operational threat models and heterogeneous real-world workloads. Within the evaluated scope, EPCIS-MutBench demonstrates that security-relevant cross-event inconsistencies can be converted into controlled, localizable, and reproducible test cases, providing a rigorous experimental basis for evaluating information-integrity assurance beyond structural conformance in interoperable EPCIS systems.