SEMA: Self-Evolving Multi-Agent Auditing for Smart Contracts ^†

Abstract

Smart contract auditing remains challenging because vulnerabilities often emerge only under complex execution conditions, cross-transaction interactions, and environment-dependent assumptions. Existing analysis techniques, including static analysis, symbolic execution, fuzzing, and recent LLM-assisted approaches, each provide useful but incomplete coverage, and monolithic auditing pipelines often struggle to balance search breadth, reproducibility, and reporting reliability. This paper presents SEMA, a self-evolving multi-agent auditing framework for smart contracts that formulates auditing as a resource-bounded discovery of concrete counterexamples under replay-certified reporting semantics. SEMA combines heterogeneous specialized agents, an orchestrator, a shared artifact-centric knowledge base, and a replay-based referee. During auditing, agents generate and consume reusable artifacts, such as candidate invariants, refuted hypotheses, transaction templates, and coverage cues, allowing the shared search state to evolve across rounds without modifying the analyzers themselves. To ensure reporting reliability, findings are accepted only when the referee can replay the candidate scenario under a pinned execution configuration and confirm violation of an executable security property. We further evaluate SEMA on an annotated smart contract benchmark under a fixed 300 s budget per contract. The full system achieves 0.9469 instance recall, 0.9441 success rate, and 0.9445 macro-average category recall on the retained executable subset, outperforming both symbolic-only and fuzzing-only baselines, as well as multi-agent ablations that disable dynamic knowledge evolution or cross-agent artifact reuse.

1. Introduction

Smart contracts [1], first implemented on Ethereum [2], have become a foundational programming abstraction for decentralized systems. They now govern critical functionality across decentralized finance (DeFi) [3,4], Internet of Things ecosystems [5,6,7], healthcare platforms [8,9], and security infrastructures [10,11]. This broad adoption makes smart contract security unusually consequential. Because deployed contracts execute persistently on public ledgers and are difficult to patch retroactively, latent defects can be exploited rapidly and may result in irreversible financial or operational damage. As a result, contract auditing has become a central requirement for dependable blockchain systems.

Over the past several years, the auditing ecosystem has advanced substantially. Static analyzers identify recurring bug patterns and insecure code idioms [12,13]; fuzzers explore concrete executions to expose input-dependent failures [14,15]; and formal methods provide stronger guarantees for carefully encoded correctness and safety properties [16,17]. These approaches have improved the state of practice, and recent work has further extended analysis to bytecode- and intermediate-representation-level reasoning [13,15,18]. Nevertheless, practical auditing remains difficult. Modern contracts are rarely isolated artifacts: they interact with companion contracts, proxy layers, inherited libraries, and environment-dependent assumptions. Moreover, auditors must often reason under incomplete information, heterogeneous program representations, and limited analysis budgets. In such settings, no single analysis technique offers a uniformly effective trade-off between depth, coverage, and validation cost.

Large language models (LLMs) [19] have recently been explored for smart contract security tasks such as summarizing code behavior, generating candidate test cases or properties, and explaining suspected vulnerabilities [20,21,22,23]. However, directly applying an LLM as a monolithic auditor raises three persistent problems. First, coverage remains uneven because useful evidence is distributed across multiple representations, including source code, bytecode, execution traces, and inferred behavioral summaries. Second, heuristic reasoning alone is insufficiently reliable for high-stakes reporting: LLM outputs may be plausible but incorrect, and even traditional detectors can over-approximate or surface warnings that do not correspond to executable exploits. Third, many reported findings are difficult to reproduce deterministically, which weakens their value for downstream triage and remediation. For smart contract auditing, therefore, the core challenge is not only to generate vulnerability hypotheses, but also to convert them into replayable, evidence-backed counterexamples under explicit execution assumptions.

This observation motivates a different design point. Instead of treating auditing as a single-pass prediction problem, we view it as resource-bounded counterexample discovery under a replay-certified reporting semantics. Under this view, heuristic components may explore candidate vulnerabilities, transaction scenarios, invariants, and intermediate artifacts, but a finding should be reported only after it is validated by a trusted execution-and-checking procedure under a pinned reference model. This separation between exploration and reporting is important for smart contracts, where false alarms are costly and where reproducible witnesses are often more valuable than broad but weakly grounded warnings.

In this paper, we present SEMA, a self-evolving multi-agent auditing framework for smart contracts, derived from our previous work [24]. SEMA combines multiple specialized analysis agents with an orchestrator, a shared artifact-centric knowledge base, and a replay-based referee. The agents are intentionally heterogeneous in search bias: different agents prioritize distinct modes of reasoning, such as symbolic analysis [25,26], fuzzing-oriented exploration [27], invariant-guided checking [28], and differential testing [29]. The orchestrator allocates effort under a fixed budget and coordinates information flow among agents. The referee is the only component authorized to accept findings, and it does so by replaying candidate scenarios under a pinned execution configuration and evaluating executable security properties. This architecture allows heuristic exploration to remain flexible while restricting final reporting to validated counterexamples.

Compared to our prior work [24], we (i) replace the informal architecture with a rigorous formalization of agents, knowledge base, orchestrator, and referee (Section 3); (ii) introduce the concept of benchmark-aligned validated discovery and reframe all empirical claims accordingly; (iii) redesign and conduct the experiment with a controlled SmartBugs-Curated evaluation including ablation studies and category-level analysis; (iv) add new baselines including invariant-only, traditional tool comparisons, and an illustrative case study; and (v) provide a substantially expanded discussion covering scope, limitations, and threats to validity.

The term self-evolving in SEMA is used in an operational rather than anthropomorphic sense. The framework does not claim that its analyzers autonomously improve their internal reasoning procedures during an audit. Instead, it evolves the shared search state. As an audit progresses, agents produce artifacts such as candidate invariants, validated witnesses, refuted hypotheses, unsatisfiable constraints, coverage frontiers, and reusable transaction templates. These artifacts are accumulated in a shared knowledge base and then reused to guide subsequent analysis rounds. In this way, later exploration can benefit from earlier diagnostic results without requiring online training or unverifiable self-modification of the underlying analyzers. This distinction is essential to both the technical design and the claims of the paper.

We further instantiate SEMA with multiple specialized testing agents, a shared knowledge base, an orchestrator, and a replay-based referee, and we assess it on a benchmark of annotated smart contracts under a fixed wall-clock budget. Consistent with the framework design, the evaluation is framed as validated vulnerability discovery rather than unrestricted vulnerability classification: we compare accepted findings against the subset of benchmark instances that can be mapped to executable property templates supported by the referee. This framing lets us isolate whether the multi-agent design, the evolving shared state, and cross-agent artifact reuse improve benchmark-aligned validated discovery relative to weaker single-agent and ablated multi-agent variants.

The main contributions of this paper are as follows.

• We present SEMA, a self-evolving multi-agent framework for smart contract auditing that separates heuristic exploration from replay-certified reporting and targets resource-bounded discovery of concrete counterexamples.
• We formalize the framework around specialized agents, orchestrated coordination, a shared artifact-centric knowledge base, and a referee that accepts findings only after replay under a pinned execution configuration and executable property evaluation.
• We operationalize this design in an end-to-end auditing system and evaluate it on a benchmark of smart contracts, showing that the full multi-agent configuration improves benchmark-aligned validated vulnerability discovery over weaker single-agent and ablated variants under the same auditing budget.
• We provide a comparison with four established traditional analysis tools on the same benchmark to contextualize SEMA’s performance relative to the existing auditing ecosystem.

The remainder of this paper is organized as follows. Section 2 reviews related work in smart contract auditing, LLM-based analysis, and multi-agent systems. Section 3 formalizes the SEMA framework following [30]. Section 4 describes the implementation and includes an illustrative case study. Section 5 presents the evaluation. Section 6 discusses implications, limitations, and threats to validity. Section 7 concludes.

2. Related Work

Traditional smart contract auditing has been built on a combination of static analysis [12,13], symbolic execution [31], fuzzing [14,15,32,33], and formal verification [16,17,34], typically grounded in the EVM execution model [2]. These techniques remain the foundation of practical auditing because they offer complementary trade-offs. Static analyzers scale well and provide rapid feedback, but they may over-approximate behavior or miss vulnerabilities whose manifestation depends on precise multi-transaction semantics or environment interactions [12,13]. Symbolic execution improves path-sensitive reasoning and can synthesize concrete witnesses for narrow feasibility conditions, yet it remains constrained by path explosion, solver cost, and the difficulty of modeling external calls and execution environments faithfully [31,32]. Fuzzing is effective for discovering concrete failures by exploring executable behaviors, especially when guided by coverage or learned heuristics, but deep state-dependent vulnerabilities and sparsely triggered DeFi logic remain challenging [14,15,33]. Formal verification offers the strongest assurance by proving encoded properties under an explicit semantics, but its practical use depends on precise specifications, sound modeling assumptions, and substantial expert effort [16,17,34]. Recent hybrid dynamic and concolic techniques have further improved DeFi-oriented analysis [18], but their effectiveness still depends heavily on search heuristics and problem-specific guidance. Taken together, this literature shows that no single analysis paradigm provides a uniformly satisfactory balance among coverage, depth, cost, and reproducibility.

In parallel, LLM-based vulnerability analysis has emerged as a promising but still immature direction. Prior work shows that naive prompting is generally insufficient for robust vulnerability reasoning, whereas performance improves when LLMs are supplied with structured exemplars, vulnerability taxonomies, stepwise decomposition, and semantically relevant retrieval context [35,36]. Structural augmentation is also important: GRACE [37], for example, combines graph information with LLM reasoning for vulnerability detection in traditional software, while specialized fine-tuned detectors such as SecureFalcon [23] improve stability on curated vulnerability corpora by sacrificing some generality. Across these studies, a consistent lesson is that high-quality context, structural information, and task decomposition matter substantially more than treating the LLM as a standalone end-to-end detector. Surveys of this literature accordingly argue for tighter integration between LLM reasoning, program structure, retrieval, and downstream validation mechanisms [22]. However, this line of work is still centered primarily on prediction, classification, or explanation quality. It does not by itself solve the central auditing requirement emphasized in our setting: converting heuristic vulnerability hypotheses into replayable, evidence-backed counterexamples under explicit execution assumptions.

This limitation is especially important in security. Recent empirical studies report that current LLMs remain brittle for security reasoning, with hallucinations, localization errors, weak robustness under dataset shift, and unstable performance outside curated settings [38]. For smart contract auditing [39,40], these weaknesses are amplified by the need to reason about transaction ordering, contract interactions, pinned execution environments, and property-specific exploit witnesses. Consequently, LLMs are better viewed as heuristic assistants for search, prioritization, summarization, and hypothesis generation than as trusted reporting authorities. Our work adopts exactly this position. In SEMA, LLM-assisted components may help specialized agents explore the search space, but final reporting is restricted to findings accepted by a replay-based referee under a fixed validation configuration. This design differs from prior LLM-centric vulnerability-detection pipelines by framing the task as validated vulnerability discovery rather than unrestricted vulnerability classification.

In addition, while multi-agent LLM systems are increasingly explored in other software-engineering tasks [22,23,35,36,37,38], their use as a principled smart contract auditing architecture remains underdeveloped. Existing vulnerability-analysis studies largely evaluate single-model prompting, retrieval augmentation, or specialized detectors. They do not provide a framework that explicitly separates heterogeneous exploration from trusted reporting, nor do they formalize how intermediate artifacts should be accumulated and reused across rounds of analysis. In contrast, SEMA is organized around specialized agents, orchestration under a fixed budget, a shared artifact-centric knowledge base, and referee-enforced replay validation. This positioning aligns the related-work landscape with the technical contribution and evaluation target of this paper: not merely generating plausible vulnerability reports, but improving benchmark-aligned, replay-validated discovery through coordinated multi-agent exploration and cross-agent artifact reuse.

3. Self-Evolving Multi-Agent Auditing Framework

We formulate SEMA, a self-evolving multi-agent framework for smart contract auditing under replay-certified reporting. The framework is designed for resource-bounded counterexample discovery: heterogeneous analyzers may use heuristic search, symbolic reasoning, differential comparison, and LLM-assisted guidance to propose candidate vulnerabilities, but a finding is reportable only when a trusted referee reconstructs a fully specified witness and reproduces a property violation under a pinned execution configuration. This separation is the central design principle of SEMA. Heuristic components influence what is explored; they do not determine what is reported.

The term self-evolving is used in a restricted and operational sense. SEMA does not assume online learning or autonomous modification of the internal decision procedure of an analyzer during an audit. Instead, the framework evolves a shared search state: validated witnesses, replay failures, infeasible path prefixes, coverage summaries, transaction templates, and scoped behavioral abstractions are accumulated and reused by later jobs. This mechanism improves search adaptivity without enlarging the trusted computing base. This section defines the framework, its assumptions, its interaction model, and the precise scope of its reporting guarantees.

3.1. Overview of SEMA

3.1.1. Audit Targets

Let $\mathcal{C}$ denote the set of audit targets. A target $c\in \mathcal{C}$ may be given as Solidity source code with compiler metadata, as deployed bytecode with deployment context, or as a contract system comprising multiple mutually relevant components, such as a proxy–logic pair, a factory with spawned instances, or an upgrade sequence. We write $\mathsf{Comp}\left(c\right)$ for the set of components that constitute the target under the audit scope.

3.1.2. Security Objective

SEMA seeks concrete witnesses to violations of executable security properties. It is not a proof system for the absence of vulnerabilities. A successful output is therefore a validated finding: a replayable execution witness, together with a property instance and a reproducibility package, that the referee accepts under the pinned validation model.

3.1.3. Adversarial Model

The baseline adversary is transaction-level. Subject to the harness assumptions of the audit, the adversary may choose call data, caller identities, transferred value, transaction ordering within a bounded scenario, and inputs supplied by explicitly modeled external contracts. Stronger capabilities, for example adversarial callees, miner- or proposer-influenced block context, proxy upgrade actions, or bounded oracle manipulation, must be encoded explicitly in the replay harness. SEMA does not by itself model off-chain coordination, long-horizon economic strategy, mempool dynamics, chain reorganization, or unmodeled counterparties. Accordingly, any claim about classes such as front running, time manipulation, or upgrade hazards is always relative to the concrete executable property template and harness used by the referee.

3.1.4. Trusted Computing Base

The reporting guarantees of SEMA are conditional on a narrow but explicit trusted computing base:

• The pinned compiler version and build settings, when source-level replay is used;
• The reference EVM implementation and revision used for replay;
• The execution harness, including deployment parameters, proxy resolution rules, external-call models, initial balances, and chain-environment bindings;
• The executable property library used by the referee;
• The witness reduction procedure only insofar as it preserves an already reproduced violation.

Agent heuristics, LLM outputs, retrieval rankings, utility estimates, and scheduling policies are not trusted. They may affect efficiency and recall, but not the acceptance criterion for reporting.

3.2. Execution Model

For a target $c\in \mathcal{C}$, replay is interpreted under a pinned configuration

$$\kappa =(\mathsf{solc},\mathsf{evmRev},\mathsf{gas},\mathsf{harness},\mathsf{env},\mathsf{policy}),$$

where $\mathsf{solc}$ denotes compiler version and settings when applicable, $\mathsf{evmRev}$ the EVM revision, $\mathsf{gas}$ the gas schedule, $\mathsf{harness}$ the deployment and external-call harness, $\mathsf{env}$ the chain-dependent environment bindings, and $\mathsf{policy}$ the executable property library. The replay semantics induced by $\kappa$ for c is a transition system

$${\mathcal{M}}_{c,\kappa }=({\Sigma }_{c,\kappa },{\to }_{c,\kappa },{\Sigma }_{0,c,\kappa }),$$

where ${\Sigma }_{c,\kappa }$ is the set of admissible machine states, ${\to }_{c,\kappa }$ is the small-step transition relation, and ${\Sigma }_{0,c,\kappa }$ is the set of admissible initial states. A state includes at least the program counter, stack, memory, persistent storage, balances, call stack, returndata buffer, gas-related state, and all environmental variables required to replay the chosen scenario. When relevant, the state also includes proxy-resolution context, harness state for adversarial or stubbed external contracts, and instrumentation state used by property evaluation.

An audit scenario is a tuple

$$x=(T,ϵ,\mu )\in \mathcal{I},$$

where $T=\langle t_1,\dots ,t_k\rangle$ is a finite transaction sequence, $ϵ$ records the environment choices needed for replay, and $\mu$ records any harness-specific bindings not already determined by $\kappa$. Each transaction $t_i$ specifies at least a caller, a callee, calldata, transferred value, and a gas limit. The pair $(ϵ,\mu )$ may include, for example, block-number bindings, timestamp selections, oracle or stub responses, proxy resolution choices, or deployment-time constructor arguments if these are not fixed globally by $\kappa$.

Let

$${\mathsf{Init}}_{\kappa }(c,x)\subseteq {\Sigma }_{0,c,\kappa }$$

denote the initial states consistent with $(c,\kappa ,x)$. The replay operator

$${\mathsf{Replay}}_{\kappa }(c,x)$$

returns the finite set of traces reachable from ${\mathsf{Init}}_{\kappa }(c,x)$ by executing T under ${\to }_{c,\kappa }$. In the common case, the harness pins all relevant choices and ${\mathsf{Replay}}_{\kappa }(c,x)$ is a singleton. If the harness intentionally leaves a bounded family of admissible choices open, then ${\mathsf{Replay}}_{\kappa }(c,x)$ may contain multiple traces. This distinction matters for witness construction and reporting.

To avoid ambiguity at the reporting boundary, SEMA distinguishes two witness representations:

Definition 1 

(Scenario-level Witness). A scenario-level witness is a candidate tuple

$$\widehat{w}=(c,\chi ,x,\eta ),$$

where χ is the declared property instance, x is an executable scenario, and η is advisory metadata produced by an analyzer, such as a rationale, path predicate, suspected vulnerable region, solver model, coverage summary, or retrieved artifacts. The metadata η is not trusted for reporting.

Definition 2 

(Replay Witness). A replay witness for $(c,\chi )$ under κ is a tuple

$$w^{\star }=(c,\chi ,x^{\star },{\tau }^{\star },\rho ),$$

where $x^{\star }$ is a fully instantiated scenario, ${\tau }^{\star }\in {\mathsf{Replay}}_{\kappa }(c,x^{\star })$ is a concrete replay trace (or an aligned tuple of traces for a relational property), and ρ is a referee-generated reproducibility package that records every choice needed for deterministic re-execution under κ. In particular, if the harness admits multiple traces for a scenario-level witness, then acceptance requires the referee to refine it to some fully instantiated $x^{\star }$ before reporting.

This distinction ensures that a reported finding is backed not merely by the existence of some violating replay within an admissible class, but by an explicit replay witness that can be re-executed independently.

3.3. Executable Property

Let $\Phi$ denote the finite set of executable property instances considered during an audit. A property instance is not merely a vulnerability label; it is a concrete predicate together with the harness assumptions under which it is interpreted. Each $\chi \in \Phi$ therefore includes

$$\chi =(\mathsf{name},\mathsf{kind},\mathsf{pred},\mathsf{obs},\mathsf{scope}),$$

where $\mathsf{name}$ identifies the property family, $\mathsf{kind}\in \{\mathsf{trace},\mathsf{rel}\}$ specifies whether the property is unary or relational, $\mathsf{pred}$ is the executable predicate, $\mathsf{obs}$ specifies the observable state projected to the predicate, and $\mathsf{scope}$ records the assumptions required for valid interpretation.

3.3.1. Trace Properties

A trace property is evaluated over one concrete replay trace:

$$〚\chi {〛}_{\kappa }:{\mathsf{Trace}}_{\kappa }\to \{\top ,\perp \}.$$

Typical examples include assertions over access-control state transitions, unauthorized balance transfer, invariant-preserving storage updates, prohibited event/state mismatches, or reentrant interference under a specified adversarial callee harness. A trace $\tau$ violates $\chi$ iff $〚\chi {〛}_{\kappa }\left(\tau \right)=\perp$.

3.3.2. Relational Properties

A relational property compares multiple aligned traces:

$$〚\chi {〛}_{\kappa }:{\mathsf{Trace}}_{\kappa }^m\to \{\top ,\perp \},$$

for some arity $m\ge 2$. A relational property instance therefore includes an alignment relation

$${\mathsf{Align}}_{\chi }(x_1,\dots ,x_m),$$

which specifies which executions are meaningfully comparable, and an equivalence projection

$${\mathsf{ObsEq}}_{\chi }({\tau }_1,\dots ,{\tau }_m),$$

which identifies what aspects of behavior must coincide. This is necessary because differential discrepancies are not vulnerabilities by default. The property itself must specify the permitted differences and the observations that should remain equivalent. In upgrade analysis, for example, ${\mathsf{Align}}_{\chi }$ may require equal caller roles and normalized call contexts, while ${\mathsf{ObsEq}}_{\chi }$ may compare externally visible return values, state deltas on preserved storage slots, and emitted events after excluding fields intentionally changed by the upgrade specification.

3.3.3. Executable Property Templates

The property library exposed to the referee is a set of executable templates. A template is a parameterized family of property instances whose semantics is fixed by code, not by natural-language interpretation. Each template specifies:

• The required observables and instrumentation;
• The admissible harness assumptions;
• The witness shape expected by replay;
• The violation predicate;
• The scope conditions under which the template may be instantiated.

This design prevents the framework from claiming coverage over a vulnerability category more broadly than the executable predicate actually supports. For example, a template that detects role-conditioned unauthorized state mutation under a bounded transaction prefix should be reported as such; it should not be described as complete coverage of all access-control flaws.

3.3.4. Audit Objective and Deduplication

Let B denote the total audit budget, and let $\mathcal{F}$ denote the set of validated findings reported by the referee. The operational objective of SEMA is to maximize the number and diversity of deduplicated findings subject to budget B. Two findings are deduplicated only if they concern the same target component set, the same instantiated property template, and materially the same fault site or semantically equivalent replay witness after normalization. This equivalence is implementation-defined but must be explicit and stable within an experiment. Deduplication is therefore a practical reporting operation, not a semantic guarantee about root-cause identity.

3.4. SEMA Architecture

We define SEMA as the tuple

$$\mathfrak{S}=\langle \mathcal{A},\mathfrak{O},\mathfrak{K},\mathfrak{J}\rangle ,$$

where $\mathcal{A}$ is a set of testing agents, $\mathfrak{O}$ is the orchestrator, $\mathfrak{K}$ is the shared knowledge base, and $\mathfrak{J}$ is the referee.

3.4.1. Testing Agents

Each agent ${\mathfrak{A}}_i\in \mathcal{A}$ is a heuristic analysis module specialized to a search regime. A job assigned to ${\mathfrak{A}}_i$ has the form

$$j=(c,\chi ,{\mathfrak{A}}_i,{\Theta }_i,b,\nu ),$$

where $c\in \mathcal{C}$ is the target, $\chi \in \Phi$ is the property instance or family currently emphasized, ${\Theta }_i$ is an agent-specific configuration, b is a reserved budget slice, and $\nu$ is the snapshot version of the knowledge base consumed by the job. The agent interface is

$${\mathsf{Run}}_i(c,\chi ,{\Theta }_i,{\mathfrak{K}}^{\left(\nu \right)},b)\mapsto ({\widehat{\mathcal{W}}}_i,\Delta {\mathfrak{K}}_i,{\mathsf{Stat}}_i),$$

where ${\widehat{\mathcal{W}}}_i$ is a set of scenario-level witnesses, $\Delta {\mathfrak{K}}_i$ a set of candidate artifact updates, and ${\mathsf{Stat}}_i$ execution statistics used only for heuristic adaptation. Agents may invoke LLMs for advisory tasks such as target prioritization, transaction-sketch construction, state summarization, or artifact ranking. Such outputs are never treated as accepted evidence.

3.4.2. Orchestrator

The orchestrator $\mathfrak{O}$ instantiates jobs, reserves budget, controls parallelism, maintains utility estimates, and mediates all writes to the shared knowledge base. To avoid concurrency ambiguity, running jobs do not mutate $\mathfrak{K}$ directly. Each job reads a versioned snapshot ${\mathfrak{K}}^{\left(\nu \right)}$ and emits a delta $\Delta {\mathfrak{K}}_i$; the orchestrator serializes commits in commit order after compatibility and dominance checks. This design makes the effective search state well defined even under parallel execution.

3.4.3. Knowledge Base

The knowledge base $\mathfrak{K}$ stores typed artifacts together with scope, provenance, and consumption history. It is the sole medium through which earlier jobs influence later ones, and thus the operational mechanism of self-evolution.

3.4.4. Referee

The referee $\mathfrak{J}$ is the only component authorized to produce validated findings. Given a scenario-level witness, it reconstructs a fully specified replay witness, replays it under $\kappa$, evaluates the declared property, and emits a finding only if violation is reproduced concretely. It also records negative outcomes, such as mismatch between the declared property and the replayed behavior, so that failed witnesses can still improve later search without entering the report.

3.5. Agent Roles

3.5.1. Symbolic Execution Agent

The symbolic execution agent searches for feasible transaction prefixes and state evolutions satisfying agent-selected target predicates. It is appropriate when the vulnerability depends on narrow feasibility regions or latent state preconditions. The agent reasons under an explicit model of environmental and external-call nondeterminism supplied by the harness. Any summary used for an external callee, proxy dispatch path, or storage aliasing relation must therefore be part of the harness scope recorded by the witness. LLM assistance, when present, is restricted to advisory tasks such as ranking suspicious branches or proposing transaction goals; satisfiability and model extraction remain delegated to the symbolic engine.

3.5.2. Fuzzing Agent

The fuzzing agent mutates transaction sequences to maximize behavioral novelty subject to the current property focus and budget. It may use ABI-conformant mutations, grammar-based argument generation, direct low-level calls, or sequence-level template mutation depending on the target scope. Unlike a purely monolithic greybox fuzzer, it can retrieve typed seeds, infeasible prefixes, storage predicates, and transaction skeletons from $\mathfrak{K}$, thereby biasing exploration with information discovered by other agents. However, all such guidance is advisory: the referee still validates only concrete replay witnesses.

3.5.3. Invariant-Testing Agent

The invariant-testing agent infers candidate regularities over observed state transitions, event/state correspondences, role-conditioned updates, or temporal relations across bounded transaction sequences. Inferred invariants are treated as provisional specifications. A violated invariant becomes audit-relevant only through a two-stage filter: first, it must be mapped to a property template whose semantics is explicit; second, the referee must validate a replay witness for that instantiated property. This prevents the framework from treating every learned regularity as a security claim.

3.5.4. Differential Analysis Agent

The differential analysis agent searches for violations of relational property instances. It compares executions only when an explicit alignment relation is available, such as a declared proxy–logic pairing, versioned upgrade relation, compiler-configuration comparison, or annotated function correspondence. The agent must therefore specify: (i) the compared targets or call paths; (ii) the normalization applied to caller, value, and environment; (iii) the observable projection on which equivalence is expected; and (iv) any allowed differences. This avoids interpreting arbitrary behavioral divergence as a vulnerability.

The framework does not require exactly these four agents, but the interfaces above characterize the intended instantiation and the form of evidence each agent may contribute.

3.6. Self-Evolving Knowledge Base

A central contribution of SEMA is the shared knowledge base $\mathfrak{K}$, which turns local analysis outcomes into globally reusable search guidance.

3.6.1. Typed Artifacts

Artifacts belong to the disjoint union

$$\mathsf{Art}=\mathsf{Seed}\uplus \mathsf{Constr}\uplus \mathsf{Inv}\uplus \mathsf{Trace}\uplus \mathsf{Summary}\uplus \mathsf{Template}\uplus \mathsf{Regression}\uplus \mathsf{Rejection}.$$

The types denote, respectively, concrete or parameterized transaction seeds; feasibility or infeasibility constraints; provisional invariants; concrete replay traces or normalized trace abstractions; behavioral summaries; reusable transaction templates; accepted replay witnesses preserved for regression and deduplication; and structured records of referee rejection outcomes.

Each artifact a is stored with metadata

$$m\left(a\right)=(\mathsf{type}, \mathsf{scope}, \mathsf{prov}, \mathsf{producer}, \mathsf{version}, \mathsf{region}, \mathsf{score}, \mathsf{usehist}),$$

where $\mathsf{scope}$ records the conditions under which reuse is admissible, $\mathsf{prov}$ records derivation provenance, $\mathsf{version}$ identifies the knowledge-base version at commit time, $\mathsf{region}$ identifies the relevant target component or code region, $\mathsf{score}$ is an advisory utility estimate, and $\mathsf{usehist}$ records downstream consumption outcomes.

3.6.2. Compatibility

Artifacts are never globally reusable by default. Let q denote a consumer query induced by a job. Reuse is controlled by a compatibility predicate

$$\mathsf{Compat}(a,q)\in \{\top ,\perp \},$$

which checks consistency between the artifact scope and the consumer context. At minimum, compatibility requires agreement on the target or target relation, relevant code version, harness assumptions, property family, and any prefix-state or environment conditions on which the artifact depends. For artifacts derived from proxy or upgrade analysis, compatibility additionally requires consistency of the proxy-resolution and storage-layout assumptions recorded in scope. If compatibility fails, the artifact is not retrieved.

3.6.3. Versioned Retrieval

A running job observes only the snapshot ${\mathfrak{K}}^{\left(\nu \right)}$ assigned at launch. Retrieval is therefore a pure function  

$$\mathsf{Retrieve}({\mathfrak{K}}^{\left(\nu \right)},q)\to \{a_1,\dots ,a_r\},$$

where each returned artifact satisfies $\mathsf{Compat}(a_i,q)=\top$. This snapshot discipline eliminates ambiguity about whether a job was influenced by artifacts committed concurrently after the job began.

3.6.4. Commit and Conflict Handling

After a job completes, the orchestrator validates each proposed update in $\Delta {\mathfrak{K}}_i$ before committing it to the next version of the knowledge base. Commits are serialized. If two artifacts conflict, the orchestrator does not collapse them silently. Instead, it either records both with distinct provenance and scope, or discards one according to an explicit artifact-type-specific dominance rule. This is important for summaries and provisional invariants, whose meaning may differ across scopes even when their surface form is similar.

3.6.5. Type-Specific Dominance

For each artifact type t, the knowledge base maintains a preorder

$${⪯}_t\subseteq {\mathsf{Art}}_t\times {\mathsf{Art}}_t$$

that captures operational redundancy under compatible scope. Intuitively, $a_1{⪯}_t{a}_2$ means that retaining $a_2$ renders $a_1$ unnecessary for intended downstream consumers of type t. Dominance is a practical utility relation, not a semantic implication relation. For example, a stronger path constraint may be logically informative but less reusable than a weaker one. Dominance therefore controls storage and retrieval policy, not soundness.

3.6.6. Feedback Transformations

Self-evolution is realized by explicit transformations from local outcomes to reusable artifacts:

• Accepted replay witness → regression artifacts, region-localized templates, and targeted summaries;
• Referee rejection → rejection artifacts explaining why the candidate failed under replay, such as unsatisfied preconditions, mismatched property declarations, or stale scope assumptions;
• Infeasible symbolic prefix → infeasible-prefix constraints or mutation guards;
• Repeated trace motif → normalized summaries or reusable transaction templates;
• Stable differential discrepancy under aligned replays→ refined relational hypotheses for subsequent confirmation.

These transformations define the operational meaning of self-evolution: later jobs are informed by validated outcomes and structured failures from earlier rounds.

3.7. Orchestration

Given input targets ${\mathcal{C}}_{\mathsf{in}}\subseteq \mathcal{C}$, property set $\Phi$, an initial knowledge-base state ${\mathfrak{K}}^{\left(0\right)}$, and budget B, the orchestrator repeatedly instantiates jobs, reserves budget, executes selected jobs under bounded parallelism, commits their knowledge-base deltas, and streams candidate witnesses to the referee.

3.7.1. Job Instantiation

For each target c and current knowledge state ${\mathfrak{K}}^{\left(\nu \right)}$, the orchestrator constructs candidate jobs via

$$\mathsf{Instantiate}(c,{\mathfrak{K}}^{\left(\nu \right)},\Phi )\to \left\{(c,\chi ,\mathfrak{A},\Theta ,b,\nu )\right\}.$$

Instantiation may specialize a job to a function group, contract component, path prefix, transaction skeleton, property family, or relational comparison target suggested by prior artifacts.

3.7.2. Budget Reservation and Accounting

The budget B may represent wall-clock time, execution steps, solver calls, token usage, or a composite cost metric fixed before the audit begins. To avoid ambiguity in parallel execution, the orchestrator reserves a budget slice b when scheduling a job and later reconciles reserved and realized cost after completion. Utility estimates may influence allocation, but the accounting rule itself is deterministic.

3.7.3. Utility Estimation

Each candidate job j is assigned a heuristic score

$$U(j\mid {\mathfrak{K}}^{\left(\nu \right)})=\alpha \widehat{\Delta \mathsf{Cov}}\left(j\right)+\beta \widehat{\mathsf{Acc}}\left(j\right)+\gamma \widehat{\mathsf{Novel}}\left(j\right)+\delta \widehat{\mathsf{Art}}\left(j\right)-\lambda \widehat{\mathsf{Cost}}\left(j\right),$$

where $\widehat{\Delta \mathsf{Cov}}$ estimates behavioral-coverage gain, $\widehat{\mathsf{Acc}}$ the likelihood of yielding an accepted replay witness, $\widehat{\mathsf{Novel}}$ the expected non-redundancy of the resulting finding, $\widehat{\mathsf{Art}}$ the expected downstream utility of artifacts, and $\widehat{\mathsf{Cost}}$ the expected resource consumption. This score is explicitly heuristic: it is a scheduling policy, not a claim of optimality.

3.7.4. Diversity-Aware Selection

Purely greedy allocation risks over-concentration on one search regime. The orchestrator therefore enforces diversity across agent types, target regions, and property families when selecting a batch. This reflects the portfolio character of SEMA and prevents the self-evolving state from collapsing prematurely onto a narrow class of hypotheses.

3.7.5. Asynchronous Referee Interaction

Candidate witnesses are submitted to the referee as they are produced. Accepted witnesses can therefore be turned into regression and template artifacts immediately, while rejections can de-emphasize unproductive search directions before the current audit round ends. Importantly, referee outcomes are committed to the knowledge base only through the orchestrator’s serialized commit path; they do not mutate running jobs retroactively.

3.7.6. Termination

The audit terminates when the total budget is exhausted or when no pending job satisfies the configured minimum utility threshold. Additional practical stopping criteria, such as sustained stagnation in deduplicated validated findings or exhaustion of unexplored compatible seeds, are permitted but must be fixed before evaluation.

3.7.7. Operational Workflow

Algorithm 1 summarizes the orchestration loop.

Algorithm 1 Orchestration loop of SEMA

Require: Contract set ${\mathcal{C}}_{\mathsf{in}}$, property set $\Phi$, initial knowledge base ${\mathfrak{K}}^{\left(0\right)}$, budget B       1: $\nu \leftarrow 0$, $\mathcal{F}\leftarrow Ø$, $B_{\mathrm{rem}}\leftarrow B$       2: while $B_{\mathrm{rem}}>0$ and stopping criterion is false do       3:       $J\leftarrow {\bigcup }_{c\in {\mathcal{C}}_{\mathsf{in}}}\mathsf{Instantiate}(c,{\mathfrak{K}}^{\left(\nu \right)},\Phi )$       4:       assign each $j\in J$ a utility score $U(j\mid {\mathfrak{K}}^{\left(\nu \right)})$       5:       select a diversity-aware batch $J^{\star }\subseteq J$ under remaining budget and parallelism constraints       6:       reserve budget slices for all jobs in $J^{\star }$       7:       for all $j=(c,\chi ,\mathfrak{A},\Theta ,b,\nu )\in J^{\star }$ in parallel do       8:             $(\widehat{\mathcal{W}},\Delta \mathfrak{K},\mathsf{Stat})\leftarrow {\mathsf{Run}}_{\mathfrak{A}}(c,\chi ,\Theta ,{\mathfrak{K}}^{\left(\nu \right)},b)$       9:             emit $(\widehat{\mathcal{W}},\Delta \mathfrak{K},\mathsf{Stat})$ to orchestrator     10:       end for     11:       for all completed job outputs in orchestrator commit order do     12:             validate and merge $\Delta \mathfrak{K}$ into $\mathfrak{K}$ using scope checks and type-specific dominance rules     13:             for all $\widehat{w}\in \widehat{\mathcal{W}}$ do     14:                    submit $\widehat{w}$ to referee $\mathfrak{J}$     15:                    if $\mathfrak{J}$ accepts $\widehat{w}$ and returns replay witness $w^{\star }$ then     16:                          add $w^{\star }$ to $\mathcal{F}$     17:                          transform $w^{\star }$  into regression and template artifacts and commit them to $\mathfrak{K}$     18:                    else     19:                          commit structured rejection artifacts derived from referee feedback to $\mathfrak{K}$     20:                    end if     21:             end for     22:             update utility estimates using $\mathsf{Stat}$ and referee outcomes     23:             reconcile reserved and realized cost; update $B_{\mathrm{rem}}$     24:             $\nu \leftarrow \nu +1$     25:       end for     26: end while     27: return $\mathcal{F}$

3.8. Referee Semantics

The referee $\mathfrak{J}$ is the trust anchor of SEMA. Given a scenario-level witness

$$\widehat{w}=(c,\chi ,x,\eta ),$$

it performs four steps:

• Reconstruct the scenario under the pinned configuration $\kappa$;
• Refine the scenario, if necessary, into a fully instantiated replay scenario $x^{\star }$ by fixing every remaining admissible harness choice relevant to reproduction;
• Execute $x^{\star }$ under ${\mathcal{M}}_{c,\kappa }$ to obtain the replay result;
• Evaluate the executable property instance $\chi$ on that result.
  Let

  $${\mathsf{Replay}}_{\kappa }^{\chi }(c,x^{\star })\to {\mathcal{R}}_{\chi }$$

denote the replay result domain for property $\chi$. If $\chi$ is a trace property, then ${\mathcal{R}}_{\chi }$ is a singleton trace. If $\chi$ is relational, then ${\mathcal{R}}_{\chi }$ is an aligned tuple of traces satisfying the alignment relation declared by $\chi$. A candidate is acceptable only after the referee has fixed enough of the environment to obtain such a fully specified result.

Definition 3 

(Acceptance). Let $\widehat{w}=(c,\chi ,x,\eta )$ be a scenario-level witness. The referee accepts $\widehat{w}$, written

$${\mathsf{Acc}}_{\mathfrak{J}}(\widehat{w};c,\chi ,\kappa )=\top ,$$

iff the referee constructs a fully instantiated scenario $x^{\star }$ and a replay witness

$$w^{\star }=(c,\chi ,x^{\star },{\tau }^{\star },\rho )$$

such that one of the following holds:

1. 

If χ is a trace property, then

$${\tau }^{\star }={\mathsf{Replay}}_{\kappa }^{\chi }(c,x^{\star })\mathit{and}〚\chi {〛}_{\kappa }\left({\tau }^{\star }\right)=\perp ;$$

2. 

If χ is a relational property of arity m, then

$${\tau }^{\star }=({\tau }_1^{\star },\dots ,{\tau }_m^{\star })={\mathsf{Replay}}_{\kappa }^{\chi }(c,x^{\star })\mathit{and}〚\chi {〛}_{\kappa }({\tau }_1^{\star },\dots ,{\tau }_m^{\star })=\perp .$$

The reproducibility package ρ must record every binding required for deterministic re-execution under κ, including the final harness choices used to instantiate $x^{\star }$.

Definition 4 

(Validated Finding). A validated finding is a replay witness

$$f=(c,\chi ,x^{\star },{\tau }^{\star },\rho )$$

emitted by the referee after acceptance. The report may additionally include advisory metadata, such as an analyzer rationale or a reduced explanation trace, but these annotations are not part of the acceptance criterion.

Witness Reduction

After acceptance, the referee may invoke a semantics-preserving reduction procedure to simplify the witness while preserving violation under $\kappa$. Reduction may remove irrelevant transactions, shrink arguments, simplify initialization, or tighten environmental bindings. Because exact minimality is generally expensive and may be undefined under composite harnesses, SEMA does not require globally minimal witnesses. Instead, the reproducibility package records whether the reported witness is unreduced, reduced by a specified reduction procedure, or proven minimal with respect to that procedure.

The principal reporting guarantee is immediate.

Lemma 1 

(Replay-based Reporting Soundness). Assume that replay under κ faithfully implements the pinned execution semantics for the audited target and harness, that the executable property instance χ is correctly implemented, and that SEMA reports only referee-emitted validated findings. Then every reported finding corresponds to a concrete replay witness admissible under the pinned model that violates the reported property instance.

Proof. 

Let $f=(c,\chi ,x^{\star },{\tau }^{\star },\rho )$ be a reported finding. By definition, f is emitted only after acceptance by the referee. Acceptance requires the existence of a fully instantiated replay scenario $x^{\star }$ such that replay under $\kappa$ yields ${\tau }^{\star }$ and the property evaluator returns ⊥ on that replay result. Hence f corresponds to a concrete violating execution admissible under the pinned model.    □

Proposition 1 

(Heuristic Non-Interference at the Reporting Boundary). Under the assumptions of the lemma above, unsound analyzer heuristics, LLM suggestions, retrieval errors, or scheduling mistakes cannot by themselves produce a false reported finding with respect to the pinned replay model.

Proof. 

Such components influence only candidate generation, prioritization, and artifact reuse. Reporting remains restricted to referee acceptance, which depends only on replay under $\kappa$ and evaluation of the executable property instance. Therefore, heuristic errors may waste budget, reduce recall, or bias the search toward unproductive regions, but they cannot by themselves create a reported finding lacking a concrete violating replay witness under the pinned model.    □

These guarantees are intentionally narrow. They do not establish that the encoded property is the complete security specification of the contract, that the harness captures all relevant external behavior, or that a violating replay witness always corresponds to a practically exploitable attack in deployment. They establish only the following claim, which is the strongest defensible guarantee of the framework itself: SEMA separates heuristic exploration from trusted reporting, and every reported finding is backed by an explicit replay witness under a fixed and auditable execution model.

4. SEMA in Practice

This section describes how the formal framework of Section 3 is instantiated in a concrete auditing system and provides an illustrative end-to-end case study.

4.1. Implementation

The current implementation of SEMA instantiates the formal architecture as a coordinator process that manages four specialized testing agents, a shared knowledge base, an orchestrator, and a Foundry-based referee pipeline.

4.1.1. Agent Implementation

Each agent is implemented as an independent reasoning loop backed by a large language model (Qwen3:8b [41]) served via Ollama. Agents receive structured prompts comprising: (i) the contract source code; (ii) the target vulnerability category and associated property-template specification; and (iii) relevant artifacts retrieved from the shared knowledge base. The four agent types correspond to the specializations defined in Section 3.5: a symbolic-analysis agent that focuses on path-sensitive invariant reasoning, a fuzzing agent that generates mutation-driven transaction sequences, an invariant-testing agent that synthesizes and evaluates contract-level invariant assertions, and a differential-analysis agent that compares behavioral profiles across related code variants or historical versions.

4.1.2. Knowledge Base Construction

The shared knowledge base is implemented as a typed artifact store indexed by target contract, vulnerability category, and artifact type. Artifact types include candidate invariants, transaction templates, refuted hypotheses, coverage summaries, and validated witnesses. Read and write operations are logged with provenance metadata. Compatibility checks prevent redundant storage, and type-specific dominance rules allow newer artifacts to supersede weaker predecessors.

4.1.3. Orchestrator Scheduling

The orchestrator implements round-robin scheduling with utility-informed priority adjustment under the fixed 300 s wall-clock budget. At each scheduling decision, the orchestrator evaluates agent progress using the metrics accumulated in the knowledge base and allocates the next time slice to the agent with the highest estimated marginal utility.

4.1.4. Referee Pipeline

The referee is implemented as a separate Foundry-based execution process. For each candidate finding produced by an agent, the referee pipeline: (1) constructs a Foundry test script encoding the candidate transaction sequence and the associated property assertion; (2) invokes the Foundry test runner as an external subprocess, which performs Solidity compilation, contract deployment, transaction replay, and assertion evaluation under the pinned EVM configuration; and (3) checks the exit status and assertion result of the Foundry process. A candidate is accepted if and only if the Foundry test runner reports successful execution with the property violation confirmed by the assertion.

It is important to note that the execution time of the Foundry subprocess is not counted in the referee’s logged decision time. The sub-millisecond decision times visible in the trace data reflect only the orchestrator-side bookkeeping of checking the Foundry exit status. The actual EVM execution, compilation, and assertion evaluation occur within the Foundry process and are separate from the SEMA orchestrator’s timing infrastructure. This architectural separation means that the 100% acceptance rate observed in the trace data does not indicate a trivial referee: candidates that fail to produce a valid property violation are rejected at the Foundry execution stage and never reach the acceptance decision point. The referee therefore filters candidates through genuine EVM execution, but the acceptance boundary is at the Foundry process level rather than at a separately maintained replay engine.

4.1.5. Technology Stack

The system is implemented in a coordinator architecture with the following components: Foundry (forge) for EVM replay and property-assertion evaluation, Ollama for LLM serving, and a coordinator process managing agent lifecycles, knowledge-base operations, and referee invocation. All components communicate through local interfaces.

4.2. Case Study

To illustrate how the framework operates in practice, we trace the auditing of reentrancy_dao.sol from SmartBugs-Curated, which contains a classic reentrancy vulnerability.

4.2.1. Round 1: Initial Exploration

The orchestrator assigns the contract to the symbolic-analysis agent. The agent analyzes the contract source and identifies a check-effects-interaction pattern in the withdraw() function: the balance check precedes the external call, but the state update follows the call. The agent produces a candidate invariant artifact: “The balance mapping must be updated before any external call in withdraw().” This artifact is stored in the knowledge base with provenance metadata.

4.2.2. Round 2: Cross-Agent Reuse

The fuzzing agent is scheduled next. It retrieves the invariant artifact from the knowledge base and uses it to guide transaction-sequence generation. The agent constructs a two-transaction scenario: (1) deposit funds via deposit() from an attacker contract; (2) invoke withdraw() with a fallback that re-enters withdraw(). This candidate transaction template is stored as a new artifact.

4.2.3. Round 3: Referee Validation

The referee pipeline constructs a Foundry test script encoding the candidate reentrancy scenario. The script deploys the target contract, executes the attack transactions, and asserts that the attacker extracts more funds than their legitimate balance. The Foundry subprocess executes this script, and the reentrancy succeeds: the assertion confirms that the property violation is witnessed. The referee accepts the finding.

4.2.4. Outcome

The accepted finding includes the concrete transaction sequence, the property violation (balance invariant broken by reentrant withdrawal), and the Foundry test script as the reproducibility package. The full audit completes in under 60 s for this contract, well within the 300 s budget.

5. Evaluation

We evaluate SEMA with a focus on three research questions:

• RQ1: How effective is the standard SEMA configuration on the benchmark overall and across vulnerability categories?
• RQ2: How much does the full multi-agent design improve over single-agent baselines built from the same backend model family and budget?
• RQ3: How much of the observed gain is attributable to feedback-driven self-evolution and cross-agent artifact reuse?
• RQ4: How does SEMA compare against established traditional analysis tools on the same benchmark?

5.1. Experimental Setup

5.1.1. Execution Environment

All experiments in this section use Qwen3:8b [41] as the backend model for agent reasoning under a fixed 300 s wall-clock budget per contract. All experiments were executed on a dedicated GPU server equipped with a Core Ultra 9 Processor 285K, 64 GB of DDR5 RAM, and an NVIDIA RTX 5090 GPU. The implementation instantiates the architecture described in Section 3: a symbolic-execution agent, a fuzzing agent, an invariant-testing agent, a differential analysis agent, a shared knowledge base, an orchestrator, and a replay-based referee. Each reported finding is produced only after referee validation under the pinned replay configuration.

5.1.2. Dataset

We use SmartBugs-Curated https://github.com/smartbugs/smartbugs-curated (accessed on 13 May 2026) as the evaluation dataset, i.e., the curated benchmark of annotated vulnerable smart contracts provided by the SmartBugs project [42]. SmartBugs-Curated spans ten vulnerability families and serves as the source of the benchmark contracts and dataset-level annotations used throughout this section. However, SmartBugs-Curated annotations and SEMA outputs are not identical objects: the dataset labels vulnerability instances at the benchmark level, whereas SEMA reports executable property violations witnessed under a pinned replay configuration. To make this comparison meaningful, we retain only those SmartBugs-Curated instances that can be mapped to executable property templates supported by the instantiated referee.

5.1.3. Compared Configurations

We compare the following configurations.

• Full SEMA: The standard configuration with multiple specialized agents, dynamic shared-knowledge evolution, and cross-agent artifact reuse.
• Single-Agent (Symbolic): Retains only the symbolic-analysis agent, removing heterogeneous specialization and shared coordination.
• Single-Agent (Fuzzing): Retains only the fuzzing-oriented agent, likewise removing heterogeneous specialization and shared coordination.
• Frozen-K: Preserves the initial configuration of the shared knowledge base but disables the actual insertion of newly produced artifacts during auditing. Agent write attempts are dropped rather than committed to the store, but they are still counted in the metrics as attempted writes. This isolates the effect of feedback-driven self-evolution while keeping the agent set, orchestration policy, and search budget unchanged.
• No Cross-Agent Reuse: Allows multiple agents to execute and permits artifact production, but prevents one agent from consuming artifacts produced by another. This tests whether the benefit of the full system arises from reusable artifact exchange rather than mere parallel execution.

5.1.4. Metrics

We report the following metrics.

• Instance recall: Fraction of retained benchmark vulnerability instances recovered by accepted findings.
• Success rate: Fraction of benchmark contracts for which at least one retained benchmark vulnerability instance is recovered.
• Macro-average category recall: Unweighted average of per-category recall across vulnerability categories.
• Benchmark-aligned validated precision: Fraction of accepted findings that correspond to retained benchmark vulnerability instances.
• Deduplicated findings: Number of accepted findings after deduplication.
• Mean time to first finding: Average time until the first accepted finding is produced.

We use the term benchmark-aligned validated precision deliberately. Because vulnerability benchmarks may be incomplete, accepted findings that do not match retained annotations are not necessarily false positives. The metric should therefore be interpreted as agreement with the retained benchmark subset, not as an absolute estimate of real-world precision.

5.2. Results and Analysis

5.2.1. RQ1: Performance of SEMA

Table 1. RQ1: Overall and category-level performance of the standard SEMA configuration on SmartBugs-Curated. All experiments use Qwen3:8b and a 300 s wall-clock budget per contract.

Group	Metric	Full SEMA
Overall	Instance recall	0.9469
	Success rate	0.9441
	Macro-average category recall	0.9445
	Benchmark-aligned validated precision	0.8116
	Deduplicated findings	207
	Mean time to first finding (s)	28.5688
Category recall	Access control	0.9167
	Arithmetic	1.0000
	Bad randomness	1.0000
	Denial of service	0.8571
	Front running	1.0000
	Other	0.8000
	Reentrancy	0.9375
	Short addresses	1.0000
	Time manipulation	1.0000
	Unchecked low-level calls	0.9333

summarizes the performance of the standard SEMA configuration. Under the common 300 s budget, Full SEMA achieves an instance recall of $0.9469$, a success rate of $0.9441$, and a macro-average category recall of $0.9445$. Taken together, these results indicate that the standard configuration recovers most retained benchmark instances, succeeds on most benchmark contracts, and performs well across categories rather than only on the dominant ones.

The benchmark-aligned validated precision is $0.8116$, with 207 deduplicated findings and a mean time to first validated finding of $28.5688$ s. We interpret this precision value jointly with recall rather than in isolation: the full system is designed to search broadly and then filter reports through referee validation, so benchmark agreement is expected to trade off against broader discovery.

At the category level, Full SEMA reaches recall $1.0000$ on arithmetic, bad randomness, front running, short addresses, and time manipulation, while remaining strong on access control ($0.9167$), reentrancy ($0.9375$), unchecked low-level calls ($0.9333$), and denial of service ($0.8571$). The weakest category is other at $0.8000$. These numbers suggest broad coverage across heterogeneous bug classes within the retained executable subset of the benchmark. At the same time, categories such as front running and time manipulation should be interpreted with care: the reported results apply to the concrete executable property templates and harness assumptions used by the referee, not to every broader economic or protocol-level formulation of those classes.

5.2.2. RQ2: Comparison with Single-Agent Baselines

Table 2. RQ2: Comparison between Full SEMA and single-agent baselines on SmartBugs-Curated. The best performance is bolded.

Configuration	Inst. Recall	Success Rate	Macro Cat.	BAV Precision	Dedup.	Time-to-1st (s)
Full SEMA	0.9469	0.9441	0.9445	0.8116	207	28.5688
Single-Agent (Symbolic)	0.7681	0.7552	0.6632	0.9474	114	12.1236
Single-Agent (Fuzzing)	0.6570	0.6993	0.4878	0.9174	109	11.3370

compares Full SEMA against the symbolic-only and fuzzing-only single-agent baselines under the same backend model family and wall-clock budget. Full SEMA achieves the best instance recall ($0.9469$), success rate ($0.9441$), and macro-average category recall ($0.9445$), substantially exceeding both single-agent variants. Relative to Single-Agent (Symbolic), the full system improves instance recall by $0.1788$, success rate by $0.1889$, and macro-average category recall by $0.2813$. Relative to Single-Agent (Fuzzing), the gains are $0.2899$, $0.2448$, and $0.4567$, respectively.

These differences support the claim that the benefit of Full SEMA does not arise from the underlying model alone. All three configurations use the same backend family and budget, yet the heterogeneous multi-agent design recovers substantially more retained benchmark instances. The symbolic-only baseline remains competitive on categories with strong structural cues, including perfect recall on arithmetic and bad randomness, but degrades sharply on front running ($0.2857$), short addresses ($0.0000$), and reentrancy ($0.5938$). The fuzzing-only baseline is weaker still on arithmetic ($0.7391$), bad randomness ($0.6471$), denial of service ($0.6429$), time manipulation ($0.5714$), and short addresses ($0.0000$). This pattern is consistent with the intended role of specialization: different vulnerability families reward different search biases, and the full system benefits from combining them.

The single-agent baselines achieve higher benchmark-aligned validated precision ($0.9474$ for symbolic-only and $0.9174$ for fuzzing-only) and lower time to first finding ($12.1236$ and $11.3370$ s, respectively) than Full SEMA. We therefore do not claim that the full system is uniformly superior on every axis. Rather, the evidence indicates a search trade-off: the single-agent variants are more conservative and produce earlier accepted findings, while the full system recovers a materially larger fraction of retained benchmark vulnerabilities and produces more deduplicated findings (207 versus 114 and 109). For a vulnerability-discovery setting, this broader recovery is the primary objective.

The interaction statistics reinforce this interpretation. The single-agent baselines eliminate the heterogeneous mechanisms that the full system exploits: symbolic-only performs no mutation, invariant synthesis, or differential pairing, while fuzzing-only performs mutation-driven exploration without invariant synthesis or differential analysis. The empirical gains of Full SEMA are therefore consistent with the joint effect of heterogeneous specialization and artifact-mediated coordination.

5.2.3. RQ3: Ablation Study

Table 3. RQ3: Comparison between Full SEMA and multi-agent ablations on SmartBugs-Curated. For Frozen-K, the reported write count denotes attempted writes that were dropped rather than committed to the shared store. The best performance is bolded.

Configuration	Inst. Recall	Success Rate	Macro Cat.	BAV Precision	Dedup.	Time-to-1st (s)	KB Reads	KB Write Attempts
Full SEMA	0.9469	0.9441	0.9445	0.8116	207	28.5688	839	518
Frozen-K	0.9275	0.9371	0.9096	0.8364	214	27.4931	0	524
No Cross-Agent Reuse	0.9324	0.9301	0.8772	0.8390	205	28.5522	229	515

compares Full SEMA with two multi-agent ablations designed to isolate the contribution of dynamic self-evolution and cross-agent reuse. Relative to Frozen-K, Full SEMA improves instance recall from $0.9275$ to $0.9469$, success rate from $0.9371$ to $0.9441$, and macro-average category recall from $0.9096$ to $0.9445$. Relative to No Cross-Agent Reuse, Full SEMA improves instance recall from $0.9324$ to $0.9469$, success rate from $0.9301$ to $0.9441$, and macro-average category recall from $0.8772$ to $0.9445$.

The comparison with Frozen-K isolates the value of feedback-driven self-evolution during the audit. In this ablation, the initial knowledge-base configuration is preserved, but newly generated artifacts are not inserted into the shared store. The recorded write count for Frozen-K therefore reflects attempted writes rather than committed updates. Under this setting, the loss of dynamic store evolution reduces recall and category coverage even though the agent set, budget, and orchestration structure remain unchanged. The largest visible category differences occur in reentrancy ($0.8750\to 0.9375$) and time manipulation ($0.7143\to 1.0000$), which is consistent with the intuition that these classes benefit from iterative refinement and accumulation of intermediate evidence.

The comparison with No Cross-Agent Reuse isolates the value of reusable artifact exchange across specialized agents. This ablation still permits multi-agent execution and artifact production, but blocks one agent from consuming artifacts produced by another. Full SEMA nevertheless achieves higher instance recall, success rate, and macro-average category recall, with especially large gains in other ($0.2000\to 0.8000$), access control ($0.8750\to 0.9167$), and reentrancy ($0.9063\to 0.9375$). These differences support the interpretation that the full system benefits from coordination through reusable intermediate artifacts rather than from parallel search alone.

Both ablations yield slightly higher benchmark-aligned validated precision than Full SEMA ($0.8364$ for Frozen-K and $0.8390$ for No Cross-Agent Reuse, versus $0.8116$ for the full configuration), again indicating that the standard system searches more aggressively. However, the full configuration converts that broader search into higher recall and broader category coverage. Notably, Frozen-K produces more deduplicated findings than Full SEMA (214 versus 207) while still achieving lower benchmark-aligned recovery. This reinforces an important point for evaluation: raw finding volume is not the same as benchmark-aligned vulnerability recovery.

The interaction statistics are also informative when interpreted correctly. Full SEMA performs 839 knowledge-base reads, whereas Frozen-K performs none because dropped writes leave no newly committed artifacts to consume during the audit. At the same time, Frozen-K still records 524 attempted writes, precisely because agent write attempts are tracked even when insertion is disabled. No Cross-Agent Reuse retains some knowledge-base activity (229 reads and 515 writes), but its performance remains below the full configuration. Taken together, these results indicate that the benefit of Full SEMA is not attributable solely to the concurrent execution of multiple agents; it arises from the combination of evolving shared knowledge and reusable artifact exchange across specialized agents.

5.2.4. RQ4: Comparison with Traditional Analysis Tools

To contextualize SEMA’s performance relative to the established auditing ecosystem,

Table 4. RQ4: Detection rates of SEMA configurations and traditional analysis tools on SmartBugs-Curated. All entries are reported as detected/total. Traditional tools: contract-level detection over 150 contracts. SEMA configurations: instance-level detection over the 207-instance retained subset.

Category	Slither [43]	CCC [45]	Securify2 [47]	Solhint [48]	Full SEMA	Frozen- K	No Reuse
Access control	11/18	7/18	0/18	3/18	22/24	22/24	21/24
Arithmetic	1/16	12/16	0/16	0/16	23/23	23/23	23/23
Bad randomness	0/8	0/8	0/8	0/8	17/17	17/17	17/17
Denial of service	1/8	1/8	0/8	0/8	12/14	12/14	12/14
Front running	0/7	1/7	0/7	0/7	7/7	7/7	7/7
Other	0/5	0/5	0/5	0/5	4/5	4/5	1/5
Reentrancy	30/32	28/32	0/32	5/32	30/32	28/32	29/32
Short addresses	0/1	1/1	0/1	0/1	3/3	3/3	3/3
Time manipulation	5/7	5/7	0/7	0/7	7/7	5/7	7/7
Unchecked low-level calls	33/48	9/48	0/48	24/48	70/75	70/75	70/75
Overall ‡	81/150	64/150	0/150	32/150	196/207	192/207	193/207

^‡ Traditional tools: contract-level detection rate. SEMA: instance-level detection rate on the retained executable subset.

compares Full SEMA and its two multi-agent ablations (Frozen-K and No Cross-Agent Reuse) against four traditional analysis tools run on the same SmartBugs-Curated benchmark using the SmartBugs framework [42]: Slither (v0.11.3) [12,43], CCC [44,45], Securify2 [46,47], and Solhint (v6.0.0) [48]. All results are reported as detection rates in the form detected/total. For the traditional tools, a contract is counted as detected when the tool produces at least one finding whose type is semantically aligned with the ground-truth category, and the denominator is the number of contracts in the category ($N=150$ total). For the SEMA configurations, detection counts recovered benchmark-aligned vulnerability instances on the retained executable subset ($N=207$ total instances across 143 contracts).

The results reveal a substantial performance gap between the two paradigms. The best-performing traditional tool, Slither, achieves an overall detection rate of $81/150$, whereas Full SEMA achieves $196/207$ on the retained executable subset. Even the weakest SEMA ablation, No Cross-Agent Reuse, achieves $193/207$, which exceeds every traditional tool by a wide margin. At the category level, the contrast is most striking on categories that require deeper semantic reasoning: on front running, bad randomness, short addresses, and other, all four traditional tools achieve near-zero detection, while Full SEMA achieves detection rates $\ge 4/5$ on every category. For reentrancy, Slither achieves the strongest traditional result ($30/32$), which matches Full SEMA’s detection of $30/32$; however, Slither’s reentrancy detections are pattern-matched warnings rather than replay-validated property violations. On arithmetic, CCC achieves $12/16$, while Full SEMA achieves perfect detection ($23/23$). Securify2 fails to analyze any contract due to Solidity compiler version incompatibilities, and Solhint operates primarily as a linting tool with limited semantic coverage.

The comparison also highlights the contribution of SEMA’s self-evolution and cross-agent reuse mechanisms. Frozen-K and No Cross-Agent Reuse both outperform all traditional tools on every category, but they fall short of Full SEMA on categories where iterative refinement and artifact exchange are most beneficial. For example, No Cross-Agent Reuse drops to $1/5$ on the other category (versus $4/5$ for Full SEMA), and Frozen-K drops to $5/7$ on time manipulation (versus $7/7$ for Full SEMA). These gaps indicate that self-evolution and cross-agent reuse are not only improvements over simpler SEMA ablations, but also widen the already large advantage over traditional tools.

We note that this comparison is not strictly like-for-like: traditional tools perform pattern-matching or heuristic analysis and report warnings, whereas SEMA reports replay-certified property violations under a pinned execution configuration. The denominators also differ: traditional tools are evaluated on all 150 contracts, while SEMA is evaluated on the 207 vulnerability instances across the 143-contract retained subset. Nevertheless, the magnitude of the performance gap is sufficient to demonstrate the benefit of the multi-agent architecture with replay-certified validation.

6. Discussion

The evaluation shows that SEMA improves validated vulnerability discovery under a fixed audit budget, but the significance of this result lies in how the improvement is obtained. The framework is not a monolithic predictor of vulnerability labels. Instead, it combines heterogeneous testing agents, an evolving shared knowledge base $\mathfrak{K}$, orchestration by $\mathfrak{O}$, and a replay-based referee $\mathfrak{J}$ to turn heuristic exploration into replayable, accepted findings. The results should therefore be read as evidence that the architectural choices introduced in Section 3 lead to better recovery of benchmark-aligned executable violations than simpler single-agent or weakened multi-agent alternatives.

6.1. Interpretation of the Evaluation Results

The comparison with the single-agent baselines suggests that the main gain comes from structured coordination rather than from backend model usage alone. All compared systems operate under the same time budget, yet the full configuration achieves stronger instance recall, success rate, and category coverage. This pattern is consistent with the design principles in Section 3: smart contract auditing benefits from combining distinct search modes, including symbolic reasoning, concrete execution, invariant-oriented analysis, and differential comparison, because different vulnerability classes expose evidence through different program views and execution behaviors.

The ablation results sharpen this interpretation. Frozen-K removes effective shared-store evolution, while No Cross-Agent Reuse restricts agents from benefiting from artifacts produced by others. Both changes reduce benchmark-aligned recovery even though they preserve much of the overall pipeline. This indicates that the value of self-evolution does not come from running several agents concurrently in isolation; it comes from the typed production, filtering, and reuse of intermediate artifacts described in the shared-knowledge design. In other words, SEMA benefits from stateful collaboration, not merely from parallel search.

The precision-like metrics should be interpreted together with this broader search behavior. Full SEMA does not maximize benchmark-aligned validated precision, and it is not always the fastest configuration to a first accepted finding. This is compatible with the intended objective. The system is designed to allocate budget toward broader counterexample discovery and then rely on the referee semantics to enforce acceptance discipline. The trade-off observed in evaluation is therefore a consequence of the framework’s discovery-oriented design rather than a contradiction of it.

6.2. Implications for the Framework Design

A central implication of the results is that the separation between exploration and reporting is practically useful. In SEMA, testing agents may generate hypotheses, candidate invariants, trace fragments, and differential hints, but none of these artifacts are reportable on their own. Acceptance is delegated to $\mathfrak{J}$, which checks executable property violations under the pinned replay configuration. This boundary gives the framework a clearer semantics than conventional LLM-assisted auditing pipelines, where plausible but non-reproducible reasoning can leak directly into the final report.

The knowledge base $\mathfrak{K}$ also emerges as more than a cache. Its role is to maintain scoped, typed, and reusable artifacts that can guide later tasks without allowing low-confidence heuristics to bypass the referee. The measured drop in performance under Frozen-K and No Cross-Agent Reuse indicates that artifact reuse is beneficial only when it is disciplined by compatibility checks, provenance, and type-specific dominance. This observation supports the design choice to treat artifact management as a core part of the auditing framework rather than as an implementation convenience.

These implications place SEMA in a useful middle ground between purely heuristic LLM auditing and full formal verification. The framework does not attempt to prove complete contract correctness, but it also does not present free-form model judgments as audit conclusions. Its contribution is a structured process for discovering executable counterexamples under explicit assumptions and reporting only those findings that survive replay-based validation.

6.3. Scope and Limitations

The guarantees of SEMA remain conditional on the execution and property models defined in Section 3. Reporting soundness is relative to the pinned replay environment, the encoded property templates, and the correctness of harness construction. If proxy behavior, external protocol interactions, oracle assumptions, or environment-dependent semantics are modeled incompletely, then accepted findings remain sound only with respect to that abstraction. The framework therefore certifies replayed violations under an explicit model; it does not establish model-independent exploitability.

The system is also a resource-bounded search procedure rather than a completeness method. A failed search does not imply the absence of vulnerabilities, and stronger recovery depends on the quality of scheduling, artifact reuse, and the diversity of agent behaviors under the available budget. The same qualification applies to intermediate artifacts. Inferred invariants, summaries, and LLM-generated hints may be noisy or misleading. The architecture is designed so that such artifacts can degrade efficiency or misdirect search, but cannot by themselves create an accepted finding without replay validation.

Another practical limitation is that a validated property violation is not identical to real-world impact. Some accepted findings may correspond to policy inconsistencies, unsafe edge cases, or behavior that is security-relevant but not directly monetizable. Conversely, some high-impact attacks may depend on cross-contract, market, or governance conditions that are difficult to encode within the present replay model. The evaluation should therefore be interpreted as measuring validated vulnerability discovery on the retained executable subset, not as a universal measure of economic severity or exhaustive audit quality.

6.4. Validity of the Empirical Claims

The empirical claims are subject to several validity constraints. At the construct level, the benchmark provides vulnerability annotations, whereas SEMA reports accepted executable violations under referee-defined property semantics. The reported metrics therefore capture benchmark-aligned validated recovery on the executable subset rather than unrestricted vulnerability classification. This distinction is especially important when interpreting benchmark-aligned validated precision.

At the internal level, the compared configurations differ in search behavior as well as outcomes. The full system explores more aggressively, which affects the balance between recall-oriented and precision-oriented metrics. The ablation measurements are still informative because they isolate specific architectural changes, but repeated-seed experiments and statistical confidence estimates would strengthen the causal interpretation of the observed gains if nondeterministic decoding or scheduling is enabled. All reported experiments use a single random seed (seed = 42). We acknowledge that the observed ablation gaps (e.g., $+0.0194$ instance recall vs. Frozen-K) cannot be confirmed as statistically significant from a single run. We plan to conduct multi-seed experiments in future work to provide mean ± standard deviation estimates and strengthen the causal interpretation. The current results should be interpreted as indicative of the architectural contribution rather than as statistically confirmed effect sizes.

At the external level, the conclusions are tied to the current benchmark, the retained executable subset, the instantiated property templates, the selected backend model family, and the fixed 300 s budget. The present results therefore establish effectiveness for a concrete evaluation regime rather than universal superiority across all smart contract auditing settings. A fully auditable replication package should include the benchmark split, property-template mapping rules, replay configurations, prompting and decoding settings, orchestration parameters, and metric computation scripts. Evaluation on a single benchmark (SmartBugs-Curated) limits the external validity of these conclusions. Extending the evaluation to additional benchmarks and real-world contract corpora is an important direction for future work.

7. Conclusions

This paper presented SEMA, a self-evolving multi-agent framework for smart contract auditing that combines heterogeneous vulnerability discovery with replay-certified validation. The key idea is to decouple exploration from reporting: specialized agents generate hypotheses, transaction candidates, and reusable artifacts, while a referee accepts a finding only if the claimed violation can be replayed under a pinned execution configuration and verified against an executable security property. This design improves both search diversity and reporting reliability.

The evaluation shows that SEMA consistently outperforms symbolic-only and fuzzing-only baselines under the same 300 s budget, achieving 0.9469 instance recall, 0.9441 success rate, and 0.9445 macro-average category recall on the retained executable subset. Ablation results further show that the gains come not only from multi-agent parallelism, but also from dynamic knowledge evolution and cross-agent artifact reuse, which are central to the effectiveness of the framework.

These results should be interpreted within the scope of the current methodology. SEMA is evaluated on benchmark instances that can be expressed as executable referee-supported properties, and accepted findings are limited to violations reproducible within the pinned replay model. Accordingly, the paper does not claim completeness or full coverage of broader real-world attack conditions.

Overall, SEMA demonstrates that self-evolving multi-agent auditing is a practical and effective direction for smart contract security. By combining heterogeneous analysis with replay-grounded validation, it moves LLM-assisted auditing toward a more rigorous, reproducible, and trustworthy paradigm. Future work includes extending the property language, supporting richer cross-contract behaviors, and further strengthening the adaptivity and formal guarantees of the framework.